Upgrade Procedure
If you are upgrading from 2.34, please make sure you use the special upgrade procedure described in the document Upgrading Control Center from Version 2.34.
If you have previously upgraded from 2.34 and are now going to upgrade to 3.0 or later, you must begin by undoing a step performed in the 2.34 upgrade. Run this command:
sudo apt-mark unhold python-django python-django-common
You can then follow the instructions below.
Below are general instructions for upgrading Control Center. Note that for specific releases, additional actions may be required; separate instructions are then given in each case in what follows.
Be sure to refer to the current Paragon Active Assurance Installation Guide.
-
Disable the
apache2andnetrounds-callexecuterservices completely:sudo systemctl disable apache2 sudo systemctl disable netrounds-callexecuter
-
Stop all Paragon Active Assurance services:
sudo systemctl stop "netrounds-*" apache2 openvpn@netrounds
-
Make backups according to the Operations Guide, chapter Backing Up Product Data, starting with the section "Backing Up the PostgreSQL Database".
-
Verify the integrity of the tarball containing the new Control Center version:
# Compute the checksum for the tar file and verify that it is equal to the SHA256 # checksum provided on the download page export CC_BUILD=4.4.0.12 sha256sum paa-control-center_${CC_BUILD}.tar.gz -
Unpack the Control Center tarball:
tar -xzf paa-control-center_${CC_BUILD}.tar.gz -
Install new Control Center packages.
In the file
/etc/netrounds/netrounds.confyou can optionally configure theSPEEDTEST_ADDRESSsetting (if you are going to use Speedtest). This can either point to the same IP address thatSITE_URLresolves to, or it can have a hostname of its own.Warning:You will now be prompted about overwriting existing configuration files. Before proceeding, please read all the information about settings below.
Note:- We highly recommend that you first inspect the difference between your old configuration and the new one using the "D" choice. In most cases you will then want to keep your old settings by pressing "N" (do not overwrite).
- New optional and updated settings may be available in the example configuration files provided in the packages. We recommend that you review these and add new options as appropriate for your installation.
Warning:For the Apache configuration files found in
/etc/apache2/sites-available/
you need to press "Y", which is the "package maintainer's version".
If you have installed proper SSL certificates (as recommended) instead of the default snakeoil ones, you will have to modify the file again to point to the correct path in the
SSLCertificateFileandSSLCertificateKeyFilesettings after the Debian package installation has completed. See the Installation Guide, chapter Service Configuration, section "SSL Certificate Configuration".sudo apt-get update sudo apt-get install ./paa-control-center_${CC_BUILD}/*.deb -
Run the database migration:
Warning:If you have changed the database password from the default, make sure you also change this in the
db-passwordsetting in the/etc/netrounds/plugin.yamlfile before runningncc migrate. Otherwise, the command will fail.Note:- This is a sensitive command, and care should be taken when executing it on a remote
machine. In such a scenario it is strongly recommended that you use a program like
screen(generally installed by default on popular Linux distributions) ortmux(runsudo apt-get install tmuxto install) so that the migrate command will continue running even if the ssh session breaks. - This command takes considerable time to execute.
sudo ncc migrate
- This is a sensitive command, and care should be taken when executing it on a remote
machine. In such a scenario it is strongly recommended that you use a program like
-
Restart all Paragon Active Assurance services:
sudo ncc services restart
-
Install the new Test Agent repository and plugins.
The plugins are used by Test Agent Applications.
TA_APPLIANCE_BUILD=4.4.0.12 TA_APPLICATION_BUILD=4.4.0.12 PLUGIN_BUILD=4.4.0.24 # Compute checksums for the repositories and verify that they match the # SHA256 checksums provided on the download page sha256sum paa-test-agent_${TA_APPLIANCE_BUILD}_all.deb sha256sum paa-test-agent-application_${TA_APPLICATION_BUILD}_all.deb sha256sum paa-test-agent-plugins_${PLUGIN_BUILD}_all.deb # Start the installation sudo apt-get install ./paa-test-agent_${TA_APPLIANCE_BUILD}_all.deb sudo apt-get install ./paa-test-agent-application_${TA_APPLICATION_BUILD}_all.deb sudo apt-get install ./paa-test-agent-plugins_${PLUGIN_BUILD}_all.deb
-
Enable services as follows:
sudo ncc services enable apache2 sudo ncc services enable kafka sudo ncc services enable callexecuter
-
Restart all Paragon Active Assurance services:
Note:You must do this to get the services up and running again after the upgrade.
sudo ncc services restart
-
To activate the new configuration, you also need to run:
sudo systemctl reload apache2
-
Check that the system is up and running with the commands
ncc status sudo systemctl status "netrounds-*"
-
Run the following script to enable the latest version of all plugins in all accounts:
export PORT=49900 while true; do if netstat -lnt | grep ":$PORT " > /dev/null; then echo "$(date): Plugin service is listening on port $PORT" echo "Enabling latest plugins for all accounts" sudo ncc plugins edit enabled-version --all-plugins --latest-version --all-accounts --exit-on-failure=false --verbose break fi echo "Waiting for plugin service listening on $PORT" sleep 3 doneIf you encounter the following error after you run the script, create a support case by attaching the script output in order to troubleshoot the error.
2023-11-17T09:22:46Z ERR ../../app/api/handlers/update_plugin.go:246 > Failed to change enabled plugin version error="unknown account shortname account_name" host=ip-10-0-0-11 service=plugin-service short_name=account_name src=core
For more information on how to manage plugins using the Control Center CLI, see the in-app help under "Plugins".
- Log in to the Control Center GUI and go to the Test Agents view. Next to each Test Agent for which an upgrade is available, an up-arrow icon appears. Click that icon to go ahead with the upgrade.
Troubleshooting
- Password Authentication Failed For User
- Target WSGI Script Not Found
- Same Origin Policy Disallows Reading the Remote Resource
- Test Agent Appliance Does Not Come Online After Control Center Upgrade
Password Authentication Failed For User
If the ncc migrate command fails with an error message
Failed to connect to database error="pq: password authentication failed for user \"netrounds\"" db-host=localhost db-name=paa-plugins db-port=5432 ...
you must update the variable db-password in the
/etc/netrounds/plugin.yaml file as explained in the warning above. Edit this
file and then rerun ncc migrate.
Target WSGI Script Not Found
If you accidentally selected "N" for the Apache configuration files (see this step above) and got an error message like the one below
[wsgi:error] [pid 29401:tid 140567451211520] [client 127.0.0.1:37172] Target WSGI script not found or unable to stat: /usr/lib/python2.7/dist-packages/netrounds/wsgi.py
run the following commands to get back on track:
export CC_BUILD=4.4.0.12
dpkg-deb --fsys-tarfile paa-webapp_${CC_BUILD}_all.deb | tar -x --wildcards ./etc/apache2/sites-available/*.conf --strip-components 4
sudo mv netrounds*.conf /etc/apache2/sites-available/
sudo chown -R root:root /etc/apache2/sites-available/
sudo systemctl reload apache2This overwrites the old configuration with the new one in the updated package.
Again, if you have installed proper SSL certificates (as recommended) instead of the
default snakeoil ones, you will have to modify the file again to point to the correct path
in the SSLCertificateFile and SSLCertificateKeyFile
settings after the Debian package installation has completed. See the Installation Guide,
chapter Service Configuration, section "SSL
Certificate Configuration".
Same Origin Policy Disallows Reading the Remote Resource
This or some similar error may occur if you have set SITE_URL and
SPEEDTEST_ADDRESS to different values in
/etc/netrounds/netrounds.conf. You then need to change
ALLOWED_ORIGINS in /etc/netrounds/restol.conf to allow
both of these values in the restol.conf file. The simplest way to achieve
this is to delete any value previously assigned to ALLOWED_ORIGINS. That
setting will then get a default value which allows SITE_URL and
SPEEDTEST_ADDRESS as found in
/etc/netrounds/netrounds.conf.
Test Agent Appliance Does Not Come Online After Control Center Upgrade
If you upgrade Control Center 3.1 or 3.2 to version 3.3 or later and you are using Test
Agent Appliance 3.3, it may happen that a Test Agent Appliance on which a Test Agent
Application is run (this is supported from version 3.3.1 onward) will not come online but
remain gray in Control Center. This is because traffic on port 6800 is filtered by a
DROP rule.
Resolve this issue by running the following command on the Control Center machine:
sudo iptables -I INPUT -i tun0 -p tcp --dport 6800 -j ACCEPT