Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
ON THIS PAGE
 

Restore Routing Director

This topic describes the restore functionality available for Routing Director.

You can use the restore functionality available in Deployment Shell to restore your backed-up Routing Director deployment cluster and application configuration data.

Restore Using Deployment Shell

You can restore your Routing Director network configuration from a backup configuration folder. To restore from a backup configuration folder, all microservices and applications must be stopped, and the cluster is not functional until the databases are restored. Once the databases are flushed and restored to the backed-up configuration, the applications must be restarted, and configuration restored from the databases must be reparsed.

To restore your Routing Director configuration from a specific backup configuration folder.

  1. Log in as root user to the node where the backup folder is located.
  2. Execute the following command to uninstall and stop all running application services.
    root@Primary1> request deployment service destroy

    This command runs in the background and takes some time to complete. You must wait until all the applications are shut down before proceeding to the next step.

    Monitor the progress of the command using the monitor start /epic/config/log command.

  3. Clear S3 bucket.

    1. Type exit to exit to the Linux root shell.

    2. Clear S3 bucket.

    3. After the script completes, type cli to log in again to Deployment Shell.

  4. List all the backup directories available.

    • If your backup directory is stored in the cluster nodes, use the show deployment backup command. Determine the location of the backup directory and log in to the cluster node as root user.

      For example:

    • If your backup folder is stored in a remote location, view the backup folders, determine the backup ID and download the backup folder locally to the cluster node, using the following commands.

      root@Primary1> show deployment backup remote storage-location remote-path user username password password

      root@Primary1> request deployment backup download backup-id backupID storage-location remote-path user username password password

      Where:

      remote-path is the path to the remote backup folder.

      username and password are the login credentials to the remote server.

  5. Restore the applications configuration from the backup folder.
    root@Primary1> request deployment restore backup-id backupID

    You must perform the restore operation only on the node on which the required backup folder is located.

  6. (Optional) View progress status of the restore job.
    root@Primary1> show deployment restore status backup-id backupID
  7. Reinstall all the application services.
    root@Primary1> request deployment service start

    Monitor the progress of the command using the monitor start /epic/config/log command.

  8. Restart Active Assurance metrics-service.

    1. Type exit to exit to the Linux root shell.

    2. Restart the metrics-service in the paa Kubernetes namespace .

      root@Primary1:~# kubectl rollout restart deployment paa-metrics -n paa
  9. Ensure that all devices and links are visible in the topology map. Perform the following steps.

    1. Delete the pf-org-id namespace.

      root@Primary1:~# kubectl delete namespace $(kubectl get namespaces -o jsonpath='{.items}' | jq -r '.[]|select(.metadata.name | startswith("pf-"))|.metadata.name')

    2. Restart configmonitor.

      root@Primary1:~# kubectl -n northstar rollout restart deployment ns-configmonitor
  10. Reparse and synchronize the restored configuration.

    1. Type cli to log in again to Deployment Shell.

    2. Reparse the configuration.

      root@Primary1> request deployment restore sync backup-id backupID

Caveats of the Restore Process

  • When you back up a cluster and restore it, use the same VIP addresses as the cluster that was backed up. If the restored cluster uses a new set of VIP addresses, you must change the VIP addresses configured on the devices being managed by the Routing Director deployment cluster.

  • When you perform the restore operation, the network configuration is returned to the configuration present in the backup folder. From the time the backup was taken, if the network configuration has changed due to new devices being onboarded or new service orders being executed, the network configuration in Routing Director might be different from the actual network state. To ensure that the network configuration in Routing Director and the actual network state have minimal mismatch post a restore operation, we recommend that you take regular periodic backups or specific backups after every network intent change.

  • You cannot restore data from a release different from the current installed release of Routing Director.

  • Since a backup does not store the certificates and infrastructure services configurations, that information must be kept unchanged during restoration.

  • Resources allocated to the network won’t be preserved after a restore and you must ensure that you release the allocated resources during the window between taking a backup and performing a restore.

  • Performing a restore operation requires a maintenance window. You must expect that all functionality, including access to the GUI, is unavailable during this time frame.

Restore TimescaleDB Pod Failure Scenario

If the restore process fails, due to an error of the TimescaleDB pod, you must manually copy the TimescaleDB backup file to the correct node VM.

The TimescaleDB backup file is stored on the node where TimescaleDB pod is running. In some cases, if that pod is moved to another node, the restore functionality will fail. For example:

In such a case, you must manually copy the TimescaleDB pod backup file, /export/timescaledb_backup/paa_metrics_db.dump, from the node on which it has moved to the /export/paragon-shell/backup location on the node from which you want to restore Routing Director.

Note that, the backup file needs to be copied just once. Once the file is copied to the correct path, and you retry the restore function, the issue must not repeat.

Related Documentation