Upgrading Contrail Service Orchestration
The upgrade process supports n–1 release upgrade approach. For example, if you have CSO Release 3.3 and you intend to upgrade it to CSO Release 4.1.0, you must first upgrade to CSO Release 4.0.x and, then follow the same steps to upgrade to CSO Release 4.1.0.
If your CSO release version is lower than 3.2.1, you have to opt for fresh installation.
We recommend that you take a snapshot of your current configuration and VMs before you proceed with upgrade process.
The upgrade process is applicable to CSO Release 3.2.1 and later.
This upgrade procedure is independent of the deployment type (trial and production), environment type (non-HA and HA), infrastructure components and microservices used, and the hypervisor type (KVM or VMware ESXi).
Before you begin the upgrade:
Ensure that you are in Contrail Service Orchestration (CSO) Release 3.2.1 or later.
Ensure the installer Virtual Machine (VM) is up and running.
If you are using VMware ESXi VMs, you must create the
provision_vm.conf
file in theContrail_Service_Orchestration_4.1.0/confs/
directory.
To upgrade to CSO Release 4.1.0:
- Download the CSO Release 4.1.0 installer package from the CSO Download page to the local drive.
- Log in to the installer VM as root.
- Copy the installer package from your local folder to the
installer VM.
root@host:~/# scp Contrail_Service_Orchestration_4.1.0.tar.gz root@installer VM :/root
- On the installer VM, extract the installer package.
For example, if the name of the installer package is
Contrail_Service_Orchestration_4.1.0.tar.gz
,root@host:~/# tar –xvzf Contrail_Service_Orchestration_4.1.0.tar.gz
The contents of the installer package are extracted in a directory with the same name as the installer package.
- Navigate to the CSO Release 4.1.0 directory in the installer VM.
root@host:~/# cd Contrail_Service_Orchestration_4.1.0
root@host:~/Contrail_Service_Orchestration_4.1.0#
(Optional) You can view the list of files in the Contrail_Service_Orchestration_4.1.0.
root@host:~/Contrail_Service_Orchestration_4.1.0# ls
The Contrail_Service_Orchestration_4.1.0.tar.tz file includes the following scripts:
upgrade.sh
revert.sh
- Run the upgrade.sh script.
Warning Before you upgrade ensure that all ongoing jobs in Administration Portal and Customer Portal are stopped; otherwise, the upgrade process will fail. During the upgrade, you experience a downtime as CSO goes into maintenance mode.
This script upgrades CSO software to Release 4.1.0. The upgrade.sh script puts CSO in maintenance mode; takes a snapshot of running status of all VMs (optional); upgrades all microservices and infrastructure components, if required; performs health checks at various levels; validates whether all VMs, infrastructure components, and microservices are up and running; and puts the CSO in live mode. The upgrade.sh script takes a snapshot of all VMs by default.
Note The script does not take a snapshot of the Installer VM and Virtual Route Reflector (VRR) VM.
root@host:~/Contrail_Service_Orchestration_4.1.0# ./upgrade.sh
INFO Configuration Upgrade : success INFO System Health Check : success INFO CSO Health-Check Before Upgrade : success INFO CSO Maintenance Mode Enabled : success INFO Kernel Upgrade : NA INFO VM Snapshot : success INFO Selective Infra Components Upgrade : NA INFO Central Infra Upgrade : success INFO Regional Infra Upgrade : success INFO Microservices pre-deploy scripts execution : success INFO Central Microservices Upgrade : success INFO Regional Microservices upgrade : success INFO Microservices post-deploy scripts execution : success INFO CSO Health-Check after Upgrade : success INFO Enable CSO Services : success INFO Load Microservices Data : success INFO ============================================ INFO CSO is successfully upgraded to Release Contrail_Service_Orchestration_4.1.0 INFO ============================================
Note The password for each infrastructure component is displayed on the console after the upgrade is successful. You must note the password that is displayed on the console as they are not saved in the system. To enhance the password security, the length and pattern for each password is different, the password is encrypted, and passwords in the log file are masked.
The time taken to complete the upgrade process depends on the hypervisor type and the environment type. All VMs on KVM are shut down while a snapshot is being taken. All VMs on VMware ESXi are up and running while a snapshot is being taken.
If an error occurs, you must fix the error and rerun the upgrade.sh script. When you rerun the upgrade.sh script, the script continues to execute from the previously failed step.
You can view the following log files that are available at
root/Contrail_Service_Orchestration_4.1.0/logs
:upgrade_console.log
upgrade_error.log
upgrade.log
- (Optional) If you are unable to troubleshoot the error
you can roll back to your previous release. Run the revert.sh script.
Note You can roll back to the previous release only if you have taken snapshot of VMs before starting the upgrade process.
root@host:~/Contrail_Service_Orchestration_4.1.0# ./revert.sh
INFO revert revert.py Overall Revert Summary INFO revert revert.py =============================================== INFO revert revert.py Revert VM Snapshots : success INFO revert revert.py Revert Kernel Upgrade : NA INFO revert revert.py Revert Salt Master Configurations : success INFO revert revert.py Post-revert processes : success INFO revert revert.py CSO Health-Check after Revert : success INFO revert revert.py Start Kubernet pods : success INFO revert revert.py Enable CSO Services : success INFO revert revert.py ============================================================= INFO revert revert.py CSO successfully reverted to the previously installed version. INFO revert revert.py =============================================================
After a successful upgrade, CSO is functional and you can log in to Administrator Portal and Customer Portal.
After successfully upgrading the CSO software, ensure that you download the application signatures before installing signatures on the device. This is a one-time operation after the upgrade.