Preupgrade Paragon Shell Before Upgrade
Use this procedure only if you are an advanced user, or have upgraded an earlier release of your cluster successfully before, or have been directed by a Juniper Partner to preupgrade.
The upgrade command implicitly backs up the OpenSearch database. In releases earlier than release 2.4.0, the upgrade command backed up the whole OpenSearch database. If the data volume in OpenSearch is large, the upgrade becomes a time intensive process. In release 2.4.0, the upgrade command backs-up only critical OpenSearch data, thereby reducing the time taken to upgrade. However, to use the upgrade command supported in release 2.4.0, you must execute a preupgrade script to upgrade Paragon Shell on the installer primary node of your existing earlier release before you upgrade the release. You can then use the upgrade commands options supported in release 2.4.0 to perform the upgrade. Additionally, if you want to separately back up all OpenSearch data and not just critical data, the preupgrade script also enables you to manually take a back up of all the data before upgrading the cluster.
Perform the following steps to upgrade the Paragon Automation Release 2.2.0 or release 2.3.0 to release 2.4.0 after using the preupgrade script:
Upgrade Prerequisites—Ensure that all upgrade prerequistes are met
Perform Preupgrade—Use the preupgrade script to upgrade Paragon Shell on the installer primary node and manually back up OpenSearch data
Upgrade the Paragon Automation Cluster—Upgrade the cluster using either Upgrade using the local Option or Upgrade using the remote url Option
Upgrade Paragon Shell and the OVA System Files—Upgrade Paragon Shell and the OVA system files on all the cluster nodes
Post Cluster Upgrade Tasks—Update the base OS and recreate service and resources instances.
Upgrade Prerequisites
Before you upgrade the Paragon Automation cluster, ensure the following.
-
Paragon Shell is accessible and operational.
-
The cluster nodes have the following free disk space available:
-
The primary node from which the cluster was deployed must have 15% of the total disk space + three times the upgrade file size free.
-
The other two primary and worker nodes must have 15% of the total disk space + the same amount as the upgrade file size free.
-
The worker node must have 15% of the total disk space free.
-
-
Disable and delete previous OpenSearch backup files to free up space.
Disable OpenSearch backup.
root@primary1# kubectl patch cronjob opensearch-backup-cron -n common -p '{"spec": {"suspend": true}}'Delete the periodic backup job.
root@primary1# kubectl delete job -n common -l app=opensearch-backup-cron
Delete all existing OpenSearch backup files.
root@primary1# kubectl exec -i -n common -c opensearch-backup $(kubectl get po -n common -l app=opensearch-backup -o jsonpath={.items[0].metadata.name}) -- bash -c 'rm -rf /opt/paragon/opensearch-backup/*'
-
(Optional) Check the current build and OVA version of your existing setup from Paragon Shell using the
show paragon versioncommand.
Go to Perform Preupgrade to preupgrade the cluster.
Perform Preupgrade
Perform the following steps to download the preupgrade script file and prepare the cluster for upgrade.
Log in as root user to the primary node from which the current cluster was installed. You are logged in to Paragon Shell.
Type
exitto exit from Paragon Shell to the Linux root shell.Copy the preupgrade.sh file to the /root/epic/temp folder or any location in your primary node.
In air-gapped environments where your Paragon Automation installation does not have access to the Internet, you must ensure you are able to securely copy the script file to the primary node. You might need to download the preupgrade.sh file from the Juniper Software Download site to your local computer before copying it to the primary node.
Navigate to the download location and make the preupgrade.sh executable (if not already executable).
root@primary1:~# cd /root/epic/temp root@primary1:~/epic/temp# chmod +x preupgrade.sh
Execute the preupgrade.sh script to back up OpenSearch and upgrade Paragon Shell to release 2.4.0.
root@primary1:~/epic/temp# ./preupgrade.sh
Paragon Shell is upgraded to release 2.4.0 on the primary node only.
Type
clito log in to Paragon Shell.(Optional) Manually back up OpenSearch data. The upgrade command used to upgrade the cluster backs-up critical OpenSearch data. If you want to back up all OpenSearch data, use the following command before upgrading.
root@primary1> request paragon opensearch-backup
OpenSearch data is backed up. You can now upgrade the cluster. Go to Upgrade the Paragon Automation Cluster.
Upgrade the Paragon Automation Cluster
You can upgrade your Paragon Automation Release 2.2.0 or release 2.3.0 installation and all the applications running on it using any one of the following two options:
Upgrade using the local Option
Use this option when your Paragon Automation installation is in an air-gapped
environment with no access to the Internet. However, you need to be able to
copy the
upgrade_paragon-release-build-id.tgz
and
upgrade_paragon-release-build-id.tgz.psig
files to your primary node. To upgrade using the local
filename option:
Type
exitto exit to the Linux root shell from Paragon Shell of the installer primary node.Copy the upgrade_paragon-release-build-id.tgz and upgrade_paragon-release-build-id.tgz.psig files, of the version to which you want to upgrade, to the /root/epic/temp folder.
You might need to download the upgrade_paragon-release-build-id.tgz and upgrade_paragon-release-build-id.tgz.psig files from the Juniper Software Download site to your local computer before copying it to the primary node.
(Optional) Use the
gpg --verifycommand to validate the digital signature of the upgrade file. For example:root@primary1:~/epic/temp# gpg --verify upgrade_paragon-release-2.4.0.8952.gbef82aec6b.tgz.psig upgrade_paragon-release-2.4.0.8952.gbef82aec6b.tgz gpg: Signature made Tue Feb 23 01:00:09 2024 UTC gpg: using RSA key 4B7B22C9C4FE32CF gpg: Good signature from "Northstar Paragon Automation 2024 ca@juniper.net" [ultimate]
Here
primary1is the installer primary node. Validation takes a couple of minutes to complete.Type
clito re-enter Paragon Shell.Use the following command to upgrade the Paragon Automation cluster:
request paragon cluster upgrade local filename upgrade_paragon-release-build-id.tgzFor example:
root@primary1> request paragon cluster upgrade local filename upgrade_paragon-release-2.4.0.8952.gbef82aec6b.tgz Checking paragon cluster system health before proceeding with cluster upgrade. This will take a minute... ... <output snipped> ... ======================================================= Overall cluster status ======================================================= GREEN ======================================================= Paragon cluster is healthy. Proceed with Paragon cluster upgrade. Upgrade is in progress ... Updated to build: paragon-release-2.4.0.8952.gbef82aec6b Paragon Cluster upgrade is successful! Run 'request paragon health-check' command to check current system health with upgraded Paragon cluster. Please continue to primary host node to upgrade Paragon-shell and update OVA system files by: /root/epic/upgrade_paragon-shell_ova-system.sh
Here
primary1is the installer primary node. The upgrade command checks the health of the cluster before upgrading. If the cluster health check returns aGREENstatus, the cluster is upgraded requiring no further input. If the cluster health check returns aREDstatus, the cluster is not upgraded. If the cluster health check returns anAMBERstatus, you are prompted to choose to continue or stop the upgrade.no-confirm—Usage example:request paragon cluster upgrade local filename upgrade_paragon-release-build-id.tgz no-confirmUse the
no-confirmoption to ignore theAMBERstatus and continue with the upgrade without being prompted. However, theno-confirmoption does not ignore aREDstatus.detach-process—Usage example:request paragon cluster upgrade local filename upgrade_paragon-release-build-id.tgz detach-processAs the upgrade process takes over an hour to complete, you can let the upgrade run in the background and free up the CLI screen for any other tasks. The command runs the initial health checks and then proceeds with the upgrade. Once the upgrade process starts, the process is detached and moved into the background and you are returned to the command prompt. The upgrade output is logged in the /epic/temp/upgrade.log file. To monitor the status of the upgrade process and print the output onscreen, use the
monitor start /epic/temp/upgrade.logcommand. When the upgrade process completes, a success message similar to the following is displayed on all the cluster nodes:Paragon Cluster upgrade is successful! - Run 'request paragon health-check' command to check current system health with upgraded Paragon cluster. - Please continue to primary host node to upgrade Paragon-shell and update OVA system files by: /root/epic/upgrade_paragon-shell_ova-system.sh
If you get disconnected from the VM during the upgrade process, you can periodically check the upgrade log file for status on the upgrade.
input—Usage example:request paragon cluster upgrade local filename upgrade_paragon-release-build-id.tgz input input-stringUse the
inputoption to pass additional Ansible input parameters to the upgrade command. For example, if you want to enable verbose logging during upgrade, use the-voption.request paragon cluster upgrade local filename upgrade_paragon-release-build-id.tgz input "-v"
Your Paragon Automation installation and all the applications running on it are upgraded.
Note that, the upgrade process takes over an hour to complete.
Execute the
request paragon health-checkcommand to ensure that the upgraded cluster is healthy and operational.The
Overall Cluster Statusmust beGREEN.For example:
root@primary1> request paragon health-check Health status checking... ======================================================= Get node count of Kubernetes cluster. ======================================================= OK There are 4 nodes in the cluster. ... <output snipped> ... ======================================================= Overall cluster status ======================================================= GREEN
Upgrade Paragon Shell and the OVA system files. While Paragon Shell is upgraded on the installer primary node already, you must upgrade it on all the cluster nodes.
Upgrade using the remote url Option
Use this option if your Paragon Automation installation has access to the
Internet and the upgrade file is in a remote location. To upgrade using the
remote url option:
Use the following Paragon Shell command on the installer primary node to upgrade the Paragon Automation cluster:
request paragon cluster upgrade remote url "https://juniper.software.download.site/upgrade_paragon-release-build-id.tgz?query_string"For example:
root@primary1> request paragon cluster upgrade remote url "https://cdn.juniper.net/software/paragon-images/upgrade_paragon-release-2.3.0.8213.g458486e9da.tgz?query_string" Checking paragon cluster system health before proceeding with cluster upgrade. This will take a minute... ... <output snipped> ... ======================================================= Overall cluster status ======================================================= GREEN ======================================================= Paragon cluster is healthy. Proceed with Paragon cluster upgrade. Upgrading paragon cluster from https://cdn.juniper.net/software/paragon-images Downloading tarball file upgrade_paragon-release-2.4.0.8952.gbef82aec6b Download file size: 28,831,677,064 bytes Current disk Usage: Total: 263,622,004,736 bytes Used: 106,109,399,040 bytes Available: 145,685,159,936 bytes Please wait for current download to finish... (File is large. It may take a while.) Upgrade tarball file is downloaded. Upgrade is in progress ... Updated to build: eop-release-2.4.0.8952.gbef82aec6b Paragon Cluster upgrade is successful! Run 'request paragon health-check' command to check current system health with upgraded Paragon cluster. Please continue to primary host node to upgrade Paragon-shell and update OVA system files by: /root/epic/upgrade_paragon-shell_ova-system.shHere
primary1is the installer primary node from which your cluster was originally installed. The upgrade command checks the health of the cluster before upgrading. If the cluster health check returns aGREENstatus, the cluster is upgraded requiring no further input. If the cluster health check returns aREDstatus, the cluster is not upgraded. If the cluster health check returns anAMBERstatus, you are prompted to choose to continue or stop the upgrade.Additional upgrade command options:
You can also use any one or more of the following command options along with the upgrade command while upgrading:
no-confirm—Usage example:request paragon cluster upgrade remote url "https://juniper.software.download.site/upgrade_paragon-release-build-id.tgz?query_string" no-confirmUse the
no-confirmoption to ignore theAMBERstatus and continue with the upgrade without being prompted. However, theno-confirmoption does not ignore aREDstatus.detach-process—Usage example:request paragon cluster upgrade remote url "https://juniper.software.download.site/upgrade_paragon-release-build-id.tgz?query_string" detach-processAs the upgrade process takes over an hour to complete, you can let the upgrade run in the background and free up the CLI screen for any other tasks. The command runs the initial health checks and then proceeds with the upgrade. Once the upgrade process starts, the process is detached and moved into the background and you are returned to the command prompt. The upgrade output is logged in the /epic/temp/upgrade.log file. To monitor the status of the upgrade process and print the output onscreen, use the
monitor start /epic/temp/upgrade.logcommand. When the upgrade process completes, a success message similar to the following is displayed on all the cluster nodes:Paragon Cluster upgrade is successful! - Run 'request paragon health-check' command to check current system health with upgraded Paragon cluster. - Please continue to primary host node to upgrade Paragon-shell and update OVA system files by: /root/epic/upgrade_paragon-shell_ova-system.sh
disk-saving—Usage example:request paragon cluster upgrade remote url "https://juniper.software.download.site/upgrade_paragon-release-build-id.tgz?query_string" disk-savingUse this option to delete the upgrade_paragon-release-build-id.tgz file as soon as it is unzipped from the primary node. The upgrade command downloads the upgrade file from the remote location and extracts the contents of the file at the beginning of the upgrade process. This option deletes the downloaded file as soon as it is unzipped to free up space on the primary node.
The advantage of using this option is that you need lesser free space for the upgrade process. The default minimum free space required is 15% of the total disk space + three times the upgrade file size. With this option you need a minimum free space of 15% of the total disk space + two times the upgrade file size.
input—Usage example:request paragon cluster upgrade remote url "https://juniper.software.download.site/upgrade_paragon-release-build-id.tgz?query_string" input input-stringUse the
inputoption to pass additional Ansible input parameters to the upgrade command. For example, if you want to enable verbose logging during upgrade use the-voption.request paragon cluster upgrade remote url "https://juniper.software.download.site/upgrade_paragon-release-build-id.tgz?query_string" input "-v"
Your Paragon Automation installation and all the applications running on it are upgraded.
Note that, the upgrade process takes over an hour to complete.
Execute the
request paragon health-checkcommand to ensure that the upgraded cluster is healthy and operational.The
Overall Cluster Statusmust beGREEN.For example:
root@primary1> request paragon health-check Health status checking... ======================================================= Get node count of Kubernetes cluster. ======================================================= OK There are 4 nodes in the cluster. ... <output snipped> ... ======================================================= Overall cluster status ======================================================= GREEN
Upgrade Paragon Shell and the OVA system files. While Paragon Shell is upgraded on the installer primary node already, you must upgrade it on all the cluster nodes.
Upgrade Paragon Shell and the OVA System Files
When your Paragon Automation installation and all the applications running on it are successfully upgraded, you must upgrade Paragon Shell and the OVA system files.
Exit from the installer primary node Paragon Shell to the Linux root shell by typing
exit.Execute the Paragon Shell upgrade shell script.
root@primary1:~# bash /root/epic/upgrade_paragon-shell_ova-system.sh Upgrading paragon-shell... Updating paragon-shell for primary1...... Container paragon-shell Stopping Container paragon-shell Stopped Container paragon-shell Removing Container paragon-shell Removed paragon-shell Pulling ... <output snipped> ... primaryname update-status primary1 ok primary3 ok primary2 ok primary4 ok paragon-shell upgrade successful! Updating OVA system files... OVA system files update successful!
Paragon Shell and the OVA system files are upgraded.
(Optional) Check the build and OVA version of your upgraded setup from Paragon Shell.
root@primary> show paragon version ova: 20250226_1117_ova ova-patch: 20250226_0349 build: eop-2.4.0.8952.gbef82aec6b Client Version: v1.29.6 Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3 Server Version: v1.31.4+rke2r1
Now proceed to perform the post cluster upgrade tasks.
Post Cluster Upgrade Tasks
After upgrading the cluster and Paragon Shell OVA, perform the following tasks to complete the upgrade process.
Update the base OS. See Update the OS.
Upgrade the service designs, update the network implementation plan, and recreate the resource and service instances. See Update the Network Implementation Plan and Recreate Service Instances After Upgrade.