Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?


Migrate Data from NorthStar to Paragon Automation

You can migrate DeviceProfile, Cassandra DB, and Analytics (ES DB) data from an existing NorthStar Release 6.x setup to a Paragon Automation Release 21.3 setup.

SUMMARY Use the steps described in this topic to migrate date from NorthStar to Paragon Automation.


  • Ensure that both the NorthStar and Paragon Automation setups are up and running.
  • Cassandra must be accessible from Paragon Automation. Set the rpc_address parameter in the /opt/northstar/data/apache-cassandra/conf/cassandra.yaml path to an address to which the Paragon Automation setup can connect. After setting the address, restart Cassandra for the configuration changes to take effect:
  • Ensure that both NorthStar and Paragon Automation have sufficient disk space to migrate the Cassandra DB. The Cassandra migration exports all data to CSV files and sufficient space must be available for the migration operation. To ensure that sufficient space is available:

    1. Log in to NorthStar and check the current disk usage by Cassandra. For a multisite setup, issue the following command on all nodes in the setup and add them to calculate total disk usage:

    2. Ensure that the available disk space on both NorthStar and Paragon exceeds the total Cassandra disk usage by at least a factor of 2. For Paragon Automation, this amount of space must be available on every node that has scheduling enabled on the device used for the /var/local directory. For NorthStar, only the node from which data is exported must have the available disk space.

      For example, on a Paragon Automation node that has a large root partition '/' without an optional partition for '/var/local':

      See Disk Requirements for more information on partition options.

      On NorthStar:

Follow this procedure to migrate data from NorthStar to Paragon Automation.

Create the nsmigration Task Pod

  1. Log in to the Paragon Automation primary node.
  2. Create the nsmigration task pod.
  3. Log in to the nsmigration task pod.

Export Cassandra DB Data to CSV Files

For the migration procedure, you must export the contents of the Cassandra database in NorthStar to CSV files and copy those files to Paragon Automation.
  1. Copy the opt/northstar/thirdparty/dsbulk-1.8.0.tar.gz file and /opt/northstar/util/db/export_csv/ from the nsmigration container in Paragon Automation to the target NorthStar installation:

    Copy the files locally to the current node:

    Copy the files to the target NorthStar installation.

  2. Log in to the NorthStar instance, and install the migration utils by extracting the dsbulk-1.8.0.tar.gz file,
  3. Export the contents of the Cassandra database to CSV files by running the script. The --skip-historical-data option can be passed to this script to skip the export of historical event date. For more information, see Table 1.

    Source the NorthStar environment file.

    Run the export script.

    Table 1: Historical Event Data Tables
    keyspace table
    taskscheduler taskstatus
    pcs topology, lsp_topo, lsp_link, ntad, messages, pcs_lsp_event, link_event, node_event
    pcs_provision provision

    Running the script exports the contents of the Cassandra database (according to db_schema.json) to the export_csv folder in the current working directory. The script pipes the progress output from the dsbulk invocations to stdout. Each table has its own sub-directory with one or more CSV files. The procedure may take a long time for larger databases.


    The exported CSV files also serve as a backup of the Cassandra DB data. We recommend archiving the files in case data needs to be restored in the future.

  4. Copy the export_csv folder to the Paragon Automation node where the nsmigration pod is running.

    Copy the exported files to the correct directory on worker3 node.

Migrate DeviceProfile and Cassandra DB

  1. Run the -a -sp -dp script from the nsmigration task pod. The complete command syntax is ./ -a ns-app-server-ip -su root -sp ns-app-user-ssh-password -dh cassandra-db-host -du cassandra -dp cassandra-password -dcsv /opt/northstar/ns_db_migration/export_csv -pu postgres-user -pp postgres-password -ph postgres-host -po postgres-port -pah vip-of-ingress-controller-or-hostname-of-main-web-application -pau paragon-web-ui-login -pap paragon-web-ui-password -dr 1.
    For example:

    You must specify the following parameters while running the script.

    • -a APP, --app APP—IP address or hostname of the application server
    • -su SSHUSER, --sshuser SSHUSER—SSH username (default is root)
    • -sp SSHPASS, --sshpass SSHPASS—SSH password
    • -so SSHPORT, --sshport SSHPORT—SSH port (default is 22)
    • -du DBUSER, --dbuser DBUSER—Cassandra DB username (default is cassandra)
    • -dp DBPASS, --dbpass DBPASS—Cassandra DB password
    • -do DBPORT, --dbport DBPORT—Cassandra DB port (default is 9042)
    • -dh DBHOST, --dbhost DBHOST—Comma-separated host IP addresses of Cassandra DB
    • -pu PGUSER, --pguser PGUSER—Postgres DB username (default is northstar)
    • -dcsv DBCSV, --dbCsvPath DBCSV—The path with CSV data exported from Cassandra
    • -pp PGPASS, --pgpass PGPASS—Postgres DB password
    • -ph PGHOST, --pghost PGHOST—Postgres DB host (default is atom-db.common)
    • -po PGPORT, --pgport PGPORT—Postgres DB port (default is 5432)
    • -pah PARAGONHOST, --paragonHost PARAGONHOST—Virtual IP (VIP) address of Paragon Automation Web UI
    • -pau PARAGONUSER, --paragonUser PARAGONUSER—Paragon Automation Web UI username
    • -pap PARAGONPASSWORD, --paragonPassword PARAGONPASSWORD—Paragon Automation Web UI user password
    • -dr DISCOVERYRETRIES, --discoveryRetries DISCOVERYRETRIES—Device discovery retries (default is 2).

      You use the dr DISCOVERYRETRIES option for DeviceProfile migration when Paragon Automation fails to discover devices at the first attempt. There are multiple reasons for discovery failure, such as devices not being reachable or device credentials being incorrect. Despite discovery failure for devices with incorrect information, Paragon Automation discovers devices with correct information. Partial failure for a subset of devices while discovering multiple devices at a time is possible. To determine the exact reason of failure, see the Monitoring > Jobs page in the Paragon Automation Web UI.

      If the dr option is set to more than 1, on getting a discovery failure, the script retries the discovery for all the devices. This attempt does not impact the devices that are already discovered. However, the chances of successfully discovering devices in subsequent attempts for any failed device discovery is minimal. We recommend that you set the maximum value for the dr option to 2, which is the default value. If there are too many devices in the network, then use a value of 1 to avoid unnecessary retries.


    When migrating Cassandra DB data from NorthStar to Paragon Automation, large tables with millions of rows might cause the migration to proceed very slowly and take a long time. Often these large tables contain historical event data that you can discard during migration. To skip migrating this data, you can manually set the '--dbSkipHistoricalData' flag while calling the '' script. This means that the data in the historical event tables listed in Table 1 is not available in Paragon Automation. This data is permanently lost if not backed up once the NorthStar instance is removed.

  2. Verify the DeviceProfile data.

    Log in to Paragon Automation Web UI and navigate to Configuration > Device. Verify that all the devices are discovered and present. Also, verify that the configuration information is the same as that in the NorthStar device profile.

    To view the device discovery result, go to the Monitoring > Jobs page in the Paragon Automation Web UI.

  3. Verify Cassandra DB data.
    The log output of the script indicates whether there were any problems migrating data from Cassandra. You can also run a script to verify the data in Paragon Automation against the exported CSV files. Note, this may take a long time for larger databases. From the nsmigration container, run:

    The script outputs (rows verified)/(rows checked) in each table (see lines beginning with "Verify") to stdout and debug_migration.log. Note that some rows may have been updated after the data was imported but before it was verified, so 'rows verified' may not always equal 'rows checked'. The exported CSV files can be removed once the migration is complete by simply removing the /var/local/ns_db_migration/export_csv directory on the relevant node.

(Optional) Migrate Analytics Data

If you have installed Analytics, perform the following steps to migrate analytics data from NorthStar ES DB to Paragon Automation Influx DB:
  1. Log in to the nsmigration task pod, and run the -a script.
    You must specify the following script options:.
    • Statistics type—By default, supports interface, label-switched path (LSP), and link-latency statistics data. You can select a specific type by using the --type option.

    • Rollups type—By default, supports daily and hourly time periods. You can select a specific type by using the --rollup_type option.

    • Migration schema—The mapping of ES DB to Influx DB schema is defined in the /opt/northstar/util/db/nsmigration/es_influx_mapping.json file.

    • Rollup ages—By default, fetches hourly and daily data for the last 180 days and 1000 days, respectively. You can change the ages by using the --hourly_age and --daily_age options.

    • ETL parallel worker process—By default, uses four ETL parallel worker processes. You can change the worker process by using the --wp option.

    • Execution time—The script execution time varies based on data volume, the number of days, and the number of ETL parallel worker processes. For example, if four ETL workers use a migration rate of 1500, then:

      • 25,000 LSP statistics of 180 days hourly can take 5 hours

      • 50,000 LSP statistics of 180 days hourly can take 10 hours

    For more information about script arguments, see help ' -h'.

  2. Verify Influx DB data using the following commands.
    • To query all tunnel traffic data for the last 30 days in Influx DB, run the /opt/pcs/bin/ script inside the dcscheduler pod:
    • To query all egress interface traffic data for the last 30 days in Influx DB, run the /opt/pcs/bin/ script inside the dcscheduler pod:

(Optional) Migrate NorthStar Planner Data

If you want to use saved NorthStar Planner models on the NorthStar application server file system in Paragon Automation, copy the models using the following steps:
  1. Log in to the NorthStar server.
  2. Use scp and copy the directory (/opt/northstar/data/specs) where your Planner models are saved to the Paragon Automation primary node (/root/ns_specs). For example:
  3. Log in to the Paragon Automation primary node.
  4. Copy the /root/ns_specs folder to the NorthStar Planner pod at /opt/northstar/data/specs using the kubectl command. For example:
  5. Verify that the NorthStar Planner models are copied inside the NorthStar Planner pod at /opt/northstar/data/specs/ns_specs.