Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
ON THIS PAGE
 

Back Up and Restore Paragon Automation

This topic describes the backup and restore functionality available for Paragon Automation.

You can use the backup and restore functionality available in Paragon Shell to back up and restore your Paragon Automation cluster and application configuration data.

Back Up Using Paragon Shell

You can back up your current Paragon Automation network configuration using Paragon Shell CLI. When you run the backup command, all the application configuration information stored in PostgreSQL, ArangoDB, and Airflow configuration database systems, LLM connector secrets keys, and available software images are backed up. The backup command also backs up telemetry information stored in OpenSearch, TimescaleDB, and VictoriaMetrics database systems. The backup procedure can be performed while the microservices and applications are running and does not affect the operation of the network. However, we recommend that you do not perform a backup during configuration changes such as device onboarding.

To back up your Paragon Automation configuration state.

  1. Log in as root user to any of the Paragon Automation nodes.
  2. Execute the request paragon backup command to back up the configuration. For example:

    The backup job runs in the background and you are returned to the command prompt. In this example, 20250318-090214 is the backup ID in the yyyymmdd-hhmmss format. The backup data is stored in a folder named as the backup ID.

    Alternatively, you can use the request paragon backup config command to back up only configuration information or request paragon backup telemetry command to back up only telemetry data.

    The backup command checks the space available in the local file system before taking a backup. If the node does not have enough storage space, the backup process will display an error and fail.

  3. (Optional) View progress of the backup job. The backup process takes a few minutes to complete. In order to not block the terminal for use for the whole duration of the backup job, you are returned to the command prompt before the backup job is complete. You can view the progress status of the backup job by using the show paragon backup status backup-id backupID command.

    For example:

  4. (Optional) If you want to store the backup is in a remote location, upload the backup folder to the remote location using the following command:

    Where:

    remote-path is the path to the remote backup location.

    username and password are the login credentials to the remote server.

Upon completion of the backup process, the backup folder is stored in the local persistent /export/paragon-shell/backup folder on the node. You'll have to exit out of Paragon Shell to the Linux root shell to navigate to the folder where the backups are saved.

Each backup folder contains the following folders with backup-related information.

  • airflow—Backup of airflow secrets and DAGs

  • arango—Backup of ArangoDB database

  • ask_paragon—Configured LLM connector secrets (in encrypted format)

  • postgres—Backup of PostgresDB database

  • software_images—Backup of available devices images

  • system_config—Backup of system configuration; this is for reference only

Telemetry data backups are stored in the nodes where the database pods are running.

Caveats of the backup process

  • Application configurations (such as devices, sites, service orders, and so on) are backed up, but certificates and infrastructure services configurations are not backed up. This information must be kept unchanged before you perform a restore.

  • The backup process captures the current infrastructure configurations, but information is used for reference only. The same configuration can be used to instantiate a new setup.

    For example, if monitoring was enabled on the cluster before performing a backup. The configuration related to monitoring is stored in the /export/paragon-shell/backup/backupID/system_config/config.cmgd file. Post-restore, use the information in the monitoring section of the config.cmgd file to reconfigure the monitoring commands on the new setup.

  • Telemetry backups are not version controlled, that is, only the latest copy of a backup is available at any given point of time. A new backup command will overwrite the existing telemetry backup with the delta of the data from the last backup.

    To maintain multiple and periodic copies of telemetry data, you can upload the telemetry backup to a remote location. To upload a backup to a remote location, use the following command.

    request paragon backup upload backup-id backupID storage-location remote-path user username password password

    You can re-upload the data to the same remote location folder every time you choose to back up telemetry data to maintain incremental copies.

Restore Using Paragon Shell

You can restore your Paragon Automation network configuration from a backup configuration folder. To restore from a backup configuration folder, all microservices and applications must be stopped, and the cluster is not functional until the databases are restored. Once the databases are flushed and restored to the backed-up configuration, the applications must be restarted, and configuration restored from the databases must be reparsed.

To restore your Paragon Automation configuration from a specific backup configuration folder.

  1. Log in as root user to the node where the backup folder is located.
  2. Execute the following command to uninstall and stop all running application services.
    root@Primary1> request paragon service destroy

    This command runs in the background and takes some time to complete. You must wait until all the applications are shut down before proceeding to the next step.

    Monitor the progress of the command using the monitor start /epic/config/log command.

  3. Clear S3 bucket.

    1. Type exit to exit to the Linux root shell.

    2. Clear S3 bucket.

    3. After the script completes, type cli to log in again to Paragon Shell.

  4. List all the backup directories available.

    • If your backup directory is stored in the cluster nodes, use the show paragon backup command. Determine the location of the backup directory and log in to the cluster node as root user.

      For example:

    • If your backup folder is stored in a remote location, view the backup folders, determine the backup ID and download the backup folder locally to the cluster node, using the following commands.

      root@Primary1> show paragon backup remote storage-location remote-path user username password password

      root@Primary1> request paragon backup download backup-id backupID storage-location remote-path user username password password

      Where:

      remote-path is the path to the remote backup folder.

      username and password are the login credentials to the remote server.

  5. Restore the applications configuration from the backup folder.
    root@Primary1> request paragon restore backup-id backupID

    You must perform the restore operation only on the node on which the required backup folder is located.

  6. (Optional) View progress status of the restore job.
    root@Primary1> show paragon restore status backup-id backupID
  7. Reinstall all the application services.
    root@Primary1> request paragon service start

    Monitor the progress of the command using the monitor start /epic/config/log command.

    Note:

    When you run the request paragon service start command, sometimes the command may fail because the config.yml is empty. Verify that the config.yml is empty using the file show /epic/config/config.yml command. Perform the steps detailed in the Release Notes: Installation and Upgrade section to repopulate the config.yml file and reinstall the application services.

  8. Ensure that all devices and links are visible in the topology map. Perform the following steps.

    1. Type exit to exit to the Linux root shell.

    2. Delete the pf-org-id namespace.

      root@Primary1:~# kubectl delete namespace $(kubectl get namespaces -o jsonpath='{.items}' | jq -r '.[]|select(.metadata.name | startswith("pf-"))|.metadata.name')

    3. Restart configmonitor.

      root@Primary1:~# kubectl -n northstar rollout restart deployment ns-configmonitor

    4. Type cli to log in again to Paragon Shell.

  9. Reparse and synchronize the restored configuration.
    root@Primary1> request paragon restore sync backup-id backupID

Caveats of the restore process

  • When you perform the restore operation, the network configuration is returned to the configuration present in the backup folder. From the time the backup was taken, if the network configuration has changed due to new devices being onboarded or new service orders being executed, the network configuration in Paragon Automation might be different from the actual network state. To ensure that the network configuration in Paragon Automation and the actual network state have minimal mismatch post a restore operation, we recommend that you take regular periodic backups or specific backups after every network intent change.

  • You cannot restore data from a release different from the current installed release of Paragon Automation.

  • Since a backup does not store the certificates and infrastructure services configurations, that information must be kept unchanged during restoration.

  • Resources allocated to the network won’t be preserved after a restore and you must ensure that you release the allocated resources during the window between taking a backup and performing a restore.

  • Performing a restore operation requires a maintenance window. You must expect that all functionality, including access to the GUI, is unavailable during this time frame.

View or Delete Backup Files

To view a list of all backup folders across all nodes, use the following command:

root@Primary1> show paragon backup

The node connects to all the other nodes in the Paragon Automation cluster using SSH and displays a list of all backup folder names along with the IP address of the node on which the folder is located.

To view a list of backup folders along with a list of failed backup attempts, use the following command:

root@Primary1> show paragon backup include-failure true

To list the available backups from a remote location use the following command.

root@Primary1> show paragon backup remote storage-location remote-path user username password password

Use this command to determine the backup that you want to download from the remote location. You can view the backup directories only in the folder that you have specified in the path and not all the backups available in other folders on the remote server. You cannot delete a backup in a remote location since you don't have the necessary permissions to manage the remote server.

To delete a backup folder, use the following command.

root@Primary1> request paragon backup delete backup-id backupID

You can delete a backup folder that is located only on the node on which you execute the command.

Upload or Download Backup Files

Upload your backed-up folder to a remote location outside the Paragon Automation cluster, within the same network as the cluster or in a different network. To upload your backup folder to a remote location, use the following command:

root@Primary1>request paragon backup upload backup-id backupID storage-location scp://IP:port/remote-path user username password password

To view progress of the backup folder upload command, use the following command:

root@Primary1> show paragon backup upload status backup-id backupID

To download your backup folder from a remote location, use the following command:

root@Primary1>request paragon backup download backup-id backupID storage-location scp://IP:port/remote-path user username password password

To view progress of the backup folder download command, use the following command:

root@Primary1> show paragon backup upload status backup-id backupID

Related Documentation