Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Junos Node Slicing Upgrade

 

Junos node slicing upgrade involves upgrading Juniper Device Manager (JDM), guest network functions (GNFs), and the base system (BSYS).

Upgrading Junos Node Slicing

Junos node slicing comprises three types of software components:

  • Juniper Device Manager (JDM) package

  • Junos OS image for guest network function (GNFs)

  • Junos OS package for base system (BSYS)

You can upgrade each of these components independently, as long as they are within the allowed range of software versions (see Multiversion Software Interoperability Overview for more details). You can also upgrade all of them together.

Note

Before starting the upgrade process, save the JDM, GNF VM, and BSYS configurations for reference.

Upgrading JDM for External Server Model

  1. Upgrade the JDM by performing the following tasks on both the servers:
    1. Copy the new JDM package (RPM or Ubuntu) to a directory on the host (for example, /var/tmp).

    2. Stop the JDM by using the following command:

      root@Linux server0# jdm stop
    3. Issue the upgrade command to upgrade the JDM package:

      If you are upgrading the JDM RHEL package, use the following command:

      root@Linux server0# rpm -U package_name.rpm --force

      If you are upgrading the JDM Ubuntu package, use the following command:

      root@Linux server0# dpkg -i deb package.deb
      Note
      • A JDM upgrade does not affect any of the running GNFs.

      • Before upgrading JDM, ensure that both JDM deployments are in sync. This means:

        • Junos running on a given GNF should support the same SMBIOS version across both the servers.

        • Before upgrade, ensure that all GNFs exist on both the servers.

      • After upgrading both the JDM servers, you must run commit synchronize before configuring any new GNF. If you do not run commit synchronize and create new GNFs on server1, you will not be able to do commit synchronize later from server0 to server1.

      • You must upgrade both the JDMs.

      See also:

Upgrading JDM for In-Chassis Model

  1. Upgrade the JDM by performing the following tasks on the BSYS instance of both the routing engines:
    1. Copy the new JDM RPM package to a directory (for example, /var/tmp).

    2. Stop the JDM by running the following command:

      root@router> request vmhost jdm stop
    3. Install the JDM RPM package for in-chassis Junos node slicing, by using the command shown in the following example:

      root@router> request vmhost jdm add jns-jdm-vmhost-18.3-20180930.0.x86_64.rpm
      Note

      A JDM upgrade does not affect any of the running GNFs.

Note

In order to upgrade JDM for in-chassis model, you need not uninstall the existing JDM software. Uninstalling the existing JDM might impact the guest network functions (GNFs).

Upgrading GNF and BSYS

The GNF and BSYS packages can be upgraded in the same way as you would upgrade Junos OS on a standalone MX Series router.

Ensure that all GNFs are online when you perform an upgrade. This is because both GNF and BSYS upgrade processes trigger multiversion checks (covered later in this guide), and all GNFs are required to be online during the multiversion check phase, failing which the upgrade will be aborted. In case a GNF remains shut down, you must deactivate its configuration from BSYS CLI, which will result in skipping multiversion checks for that particular GNF.

Note

A force option is also available, through which you can overwrite an existing GNF image with a new one by using the JDM CLI command request virtual-network-functions vnf-name add-image new-image-name force. This can be useful in a rare situation where the GNF image does not boot. You can also use the force option to perform a cleanup if, for example, you abruptly terminated an earlier add-image that was in progress, by pressing Ctrl-C (example: request virtual-network-functions vnf-name delete-image image-name force).

Upgrading JDM to Support WRL 9 based VM Host - In-Chassis Model

If the Routing Engine is to run Junos OS 19.3R1 or later, you must upgrade JDM to 19.3R1 or later.

Note

Use the following steps to upgrade the JDM.

  1. At each of the GNFs, assign mastership to the backup GNFs running on Routing Engine1 (re1).
    root@router> request chassis routing-engine master switch no-confirm
  2. On re0, first stop the GNFs from the JDM, and then stop the JDM itself from BSYS.
    root@jdm> request virtual-network-functions stop gnf-name
    root@router> request vmhost jdm stop
  3. Ensure that re0 VM Host version is Junos OS 19.3R1 or later. To check the VM Host version, use the Junos CLI command show vmhost version.

    You can use the following Junos CLIs to upgrade VM Host software:

    root@router> request vmhost software add package-name
    root@router> request vmhost reboot

    For more information, see Installing, Upgrading, Backing Up, and Recovery of VM Host.

  4. When re0 is back up after the reboot, copy the new JDM RPM package (19.3R1 or later) to a directory (for example, /var/tmp).
  5. Install the new JDM RPM package on re0 and then start the JDM.
    root@router> request vmhost jdm add package-name
    root@router> request vmhost jdm start

    The GNFs on re0 automatically start after this step.

  6. Repeat the steps 1 to 5 on Routing Engine 1 (r1).
  7. Run the request server authenticate-peer-server command at the JDM on both the Routing Engines.
    user@jdm> request server authenticate-peer-server
  8. Configure set system commit synchronize and then apply commit on re0 JDM.
    user@jdm# set system commit synchronize
    user@jdm# commit synchronize
Note

The JDM software version 19.3R1 is capable of running on Junos OS version 19.3R1 as well as on Junos OS versions prior to 19.3R1.

Downgrading JDM for External Server Model

Note

You cannot downgrade Juniper Device Manager (JDM) installed in a single-server based Junos node slicing setup.

Use the following steps to downgrade JDM:

  1. Acquire mastership to the backup GNFs running on server1.
    user@gnf> request chassis routing-engine master acquire no-confirm
    user@gnf# commit synchronize
  2. On server0, stop all the GNFs and delete the commit synchronize configuration.
    user@jdm> request virtual-network-functions test-gnf stop
    user@jdm# delete system commit synchronize
    user@jdm# commit
  3. On server0, stop and uninstall JDM.
    [user@server0 ~]# jdm stop
    [user@server0 ~]# rpm -e jns-jdm
    Note

    If you are using Ubuntu, use the command dpkg --purge jns-jdm to uninstall JDM.

  4. On server0, install the target version of JDM.
    [user@server0]# rpm -ivh jns-jdm-18.3-20181207.0.x86_64.rpm
  5. Configure JDM with root authentication or interfaces, and routing-options.
  6. On server0 JDM, add a GNF image version that is compatible with the JDM version.
    user@jdm> request virtual-network-functions add-image /var/tmp/junos-install-ns-mx-x86-64-18.3-R1.tgz gnf

    In case the GNF version is incompatible with the JDM version, the following error message is shown:

  7. Wait till the GNF comes up on server0 JDM.
  8. Perform a commit synchronize from the Master RE (which is the GNF running on server1).
    user@gnf# commit synchronize
  9. Assign mastership to the GNF which is running on server0 JDM.
  10. On Server 1, repeat the steps 2 through 5.
  11. Run the request server authenticate-peer-server command on both the servers.
    user@jdm> request server authenticate-peer-server
  12. Apply show server connections all-servers and ensure that no issues are seen.
  13. Configure set system commit synchronize and then apply commit on server0 JDM.
    user@jdm# set system commit synchronize
    user@jdm# commit synchronize
  14. Use the command show virtual-network-functions all-servers to see if the GNFs are coming up.

Downgrading JDM for In-Chassis Model

Note

You cannot downgrade Juniper Device Manager (JDM) installed in a single Routing Engine-based Junos node slicing setup.

Use the following steps to downgrade JDM:

  1. Assign mastership to the backup GNFs running on Routing Engine 1 (re1).
    user@gnf> request chassis routing-engine master switch no-confirm
  2. On re0, stop all the GNFs and delete the commit synchronize configuration.
    user@jdm> request virtual-network-functions stop server0 gnf
    user@jdm# delete system commit synchronize
    user@jdm# commit
  3. On re0, uninstall JDM (on BSYS master).
    user@bsys> request vmhost jdm delete
  4. On re0, install the target version (example: 18.3R1) of JDM.
    user@bsys> request vmhost jdm add /var/tmp/jns-jdm-vmhost-18.3-R1.3.x86_64.rpm
  5. On re0, deploy the same version of GNF which is running on server1.
    user@jdm> request virtual-network-functions add-image /var/tmp/junos-install-ns-mx-x86-64-19.1-20181115.1.tgz gnf

    In case the GNF version is incompatible with the JDM version, the following error message is shown:

    You can use the following command to check the GNF version.

    user@gnf1> show version
  6. On re1, repeat the steps 1 through 5.
  7. Run the request server authenticate-peer-server command on both the Routing Engines.
    user@jdm> request server authenticate-peer-server
  8. Perform a commit synchronize from the master Routing Engine (which is the GNF running on server1).
    user@gnf# commit synchronize
  9. Configure set system commit synchronize and then apply commit on re0 JDM.
    user@jdm# set system commit synchronize
    user@jdm# commit synchronize

Now, JDM is up with Junos OS version 18.3R1.

Unified ISSU Support

Junos node slicing also supports unified in-service software upgrade (ISSU), enabling you to upgrade between two different Junos OS versions with no disruption on the control plane and with minimal disruption of traffic. You can perform unified ISSU on BSYS and GNFs separately. Also, you can run unified ISSU on each GNF independently—without affecting other GNFs. See also Understanding the Unified ISSU Process.

Note

The multiversion software support restrictions (such as version deviation limits) are applicable to unified ISSU upgrade as well.

Managing Multiversion Software Interoperability

Junos node slicing supports multiversion software interoperability. However, if there are any incompatibilities between software versions, alert messages appear during the software upgrade process or when a GNF or a FRU comes online. When minor incompatibilities occur, you can choose to accept them and proceed. In case of a major incompatibility, you need to either abort the process or use the force option to accept the incompatibility and proceed.

Note

In case of vmhost software upgrade, the force option is not available. Therefore, if a GNF is offline or is incompatible with the software being installed, and is causing multiversion checks to abort, you need to deactivate that GNF during the software upgrade and then reactivate it once the upgrade is over.

The following are sample messages that appear if incompatibilities are detected during software upgrade:

Sample alert message indicating a minor incompatibility:

user@router> request system software add /var/tmp/junos-install-mx-x86-64-17.4-20170703_dev_common.0.tgz

Sample alert message indicating a major incompatibility:

user@router> request system software add /var/tmp/junos-install-mx-x86-64-17.4I20170713_0718.tgz

Sample output showing how to use the 'force' option to proceed with an upgrade:

user@router> request system software add /var/tmp/junos-install-mx-x86-64-17.4I20170713_0718.tgz force

Viewing Software Incompatibility Alarms

After a software update of a GNF or BSYS, if software incompatibilities between the GNF and the BSYS exist, they will be raised as a chassis alarm. You can view the incompatibility alarm information by using the show chassis alarms command. You can further view the details of the incompatibilities by using the show system anomalies command. For more details, see Viewing Incompatibilities Between Software Versions.

The alarms appear only on GNFs even if the upgrade is performed on the BSYS. The following types of alarm can occur:

  • System Incompatibility with BSYS—This is a major alarm. It appears when any incompatibilities between BSYS and GNF software versions cause the GNF to go offline.

  • Feature Incompatibility with BSYS—This is a minor alarm. It indicates a minor incompatibility between BSYS and GNF software versions. This does not cause the GNF to go offline.

Viewing Incompatibilities Between Software Versions

To view software incompatibilities from the BSYS, use the CLI as shown in the following example:

user@router> show system anomalies gnf-id 4 system

To view software incompatibilities from a GNF, use the CLI as shown in the following example:

user@router> show system anomalies system
Note
  • As shown in the CLI, remember to specify the GNF ID while viewing the incompatibilities from BSYS.

  • The preceding examples show system-level incompatibilities. Use the fru or config options to view FRU or feature-level incompatibilities.

Restarting External Servers

Server maintenance activities such as hardware or host OS upgrade and fault isolation might require you to restart the external servers used in Junos node slicing. Use the following procedure to restart the servers:

  1. Stop all the GNFs.

    If you are restarting both the servers, choose the all-servers option while stopping each GNF as shown in the following example:

    root@server1> request virtual-network-functions gnf_name stop all-servers

    If you are restarting a particular server, stop the GNFs on that server by specifying the server-id as shown in the following example:

    root@server1> request virtual-network-functions gnf_name stop server0
  2. Verify that the GNFs have been stopped.
    root@server1> show virtual-network-functions
    Note

    If you want to view the status of GNFs on both the servers, choose the all-servers option. Example: show virtual-network-functions all-servers).

  3. From the Linux host shell, stop the JDM by using the following command:
    [root@HostLinux ~]# jdm stop
  4. From the Linux host shell, verify that the JDM status shows as stopped.
    [root@HostLinux ~]# jdm status
  5. After rebooting, verify that the JDM status now shows as running.
    [root@HostLinux ~]# jdm status

After a server reboot, the JDM and the configured GNFs will automatically start running.

If you are replacing the servers, ensure that the operating server pair continues to have similar or identical hardware configuration. If the server pair were to become temporarily dissimilar during the replacement (this could be the case when replacing the servers sequentially), it is recommended that you disable GRES and NSR for this period, and re-enable them only when both the servers are similar once again.

Updating Host OS on the External Servers

Before updating the host OS on an external server, you must first stop the GNFs and JDM on that server as described in Restarting External Servers.

Following the host OS update, if you are using Intel X710 NICs, ensure that the version of the X710 NIC driver in use continues to be the latest version as described in Updating Intel X710 NIC Driver for x86 Servers .

Applying Security Updates to Host OS

The host OS requires security updates from time to time. This section highlights the steps involved in applying Security Updates to the host OS using Red Hat (RHEL) OS.

Junos node slicing supports RHEL 7.3.

Before doing any updates to the host OS, ensure that Red Hat Subscription Manager is set to version 7.3 and that Red Hat Subscription Service includes Extended Update Support (EUS).

You can use the command subscription-manager release --show to confirm that the release is set to 7.3. If it is not, you can use the command subscription-manager release --set=7.3 to set the release to 7.3.

Note

You must ensure that the Red Hat Subscription Manager is set to version 7.3. Otherwise, updates to the RHEL will attempt to upgrade to the latest minor release. For example, RHEL 7.3 could become RHEL 7.4 (or a later version) with a general yum update, or a yum security update can pull in a new kernel beyond RHEL 7.3.

Red Hat's extended update support allows for patches and security updates to be applied within the specified release. Allowed use of RHEL's Extended Update support is a function of the RHEL support contract and beyond the scope of this section. You can check to see if your RHEL subscription includes Extended Update Support (EUS), by using the command subscription-manager repos --list | grep rhel-7-server-eus-rpms. EUS support is not enabled by default. EUS can be enabled, by using the command subscription-manager repos --enable rhel-7-server-eus-rpms.

Steps to Apply Host OS Security Updates

Applying security updates to host OS will likely require you to reboot the external x86 servers. See the Updating Host OS on the External Servers topic.

It is also possible that a host OS security update will bring in a new kernel version. Updating the host OS kernel could also overwrite the Intel i40e driver to bring in a version of it that does not meet the i40e driver minimum version requirements. If so, you must update the i40e driver to meet the minimum requirements. For more details, see Updating Intel X710 NIC Driver for x86 Servers.

Before rebooting the external x86 servers, you must stop all GNF VMs and JDM on that server. Since we have two external x86 servers, the host OS Security Updates can be done without disrupting GNF forwarding, by updating one server at a time. A GRES/NSR Master Routing Engine switch-over is required to move the Master Routing Engine away from the affected server.

We start with the default behavior of Routing Engine 1 (re1) as the Backup Routing Engine for each GNF where re1 for each GNF is running on the external x86 server1.

  1. Back up all configurations.
  2. Gather view of host OS kernel and package versions on the external x86 servers before the host OS security update. Also confirm i40e driver and Intel X710 firmware meet minimum requirements (version: 2.4.10 and version: 18.5.17).
    user@server# cat /etc/redhat-release
    user@server# uname -r
    user@server# uname -a
    user@server# rpm -q kernel
    user@server# ethtool -i p3p1
  3. Ensure that RedHat Subscription Manager is set to RHEL 7.3 and the EUS Repository is enabled.
    [user@server ~]# subscription-manager version
    [user@server ~]# subscription-manager repos --list | grep rhel-7-server-eus-rpms
  4. Ensure all GNFs are using Master RE on server0. The backup Routing Engine is re1 on server1. First perform host OS security updates on the server that contains the backup Routing Engines.
    user@router> show chassis routing-engine

    Run this command on all the GNFs to confirm that all the GNFs have their master Routing Engine on server0.

  5. Stop all GNF VMs in JDM cli via request stop on server1 only. server1 contains the backup Routing Engines for all the GNFs. Do not use the all-servers option. Example:
    user@jdm> request virtual-network-functions gnf-a stop server 1
    user@jdm> request virtual-network-functions gnf-b stop server 1
  6. Stop JDM on the affected server from the host OS.
    user@server# jdm status
    user@server# jdm stop
  7. Do the yum security update and reboot the server.
    user@server# yum -y update -security
    root@server# shutdown -r now
  8. Reload or compile the i40e Driver. See the Intel support page for instructions on updating the driver.

    At this point, the host OS security update to server1 is done. Note that the GNF VMs start up on server reboot.

  9. After the security updates are completed, the server rebooted and the GNFs are back up, repeat on the other server.

Applying Security Patches for Ubuntu Container

The Ubuntu container, which Juniper Device Manager (JDM) is based on, needs to have security patches applied from time to time.

Note

JDM must be able to reach the internet and must have name-server configured. Apply the following JDM CLI configuration statement to specify the name-server:

root@jdm# set system name-server address

Use the following steps to apply security updates to the Ubuntu container components of JDM:

  1. If you are using the external server model, from host OS, use the JDM console to enter JDM as root.

    root@server# jdm console

    Or, from the JDM CLI, enter JDM shell by using the command:

    root@jdm> start shell user root

    If you are using the in-chassis Junos node slicing, use the following command on the BSYS VM to enter JDM:

    root@router> request vmhost jdm login

  2. From the JDM shell, use the command apt-get update to download information about new packages or the latest versions of the currently installed packages.

    jdm-srv1:~# sudo apt-get update

  3. From the JDM shell, use the command apt-get upgrade.

    jdm-srv1:~# sudo apt-get upgrade

    You are shown a list of upgrades, and prompted to continue. Answer Y for yes and press Enter.

  4. From the JDM shell, use the command apt-get dist-upgrade to perform the upgrade.

    jdm-srv1:~# sudo apt-get dist-upgrade

    Answer Y when prompted to continue, and wait for the upgrades to finish.

  5. If you are using the external server model, from the host OS, restart the JDM.

    user@server# sudo jdm restart

    If you are using the in-chassis Junos node slicing, use the following commands on the BSYS VM to restart the JDM:

    root@router> request vmhost jdm stop

    root@router> request vmhost jdm start