Example: Performing a Unified ISSU

 

This example shows how to perform a unified in-service software upgrade (ISSU).

Requirements

This example uses the following hardware and software components:

  • MX480 router with dual Routing Engines

  • Junos OS Release 13.3R6 as the starting release

  • Junos OS Release 14.1R4 as the ending release

Before You Begin

Before you perform a unified ISSU, be sure you:

  • Perform a compatibility check to ensure that the software and hardware components and the configuration on the device support unified ISSU by using the request system software validate in-service-upgrade command

  • Read the chapter Unified ISSU System Requirements to anticipate any special circumstances that might affect your upgrade.

    • Verify that your platform supports the unified ISSU feature.

    • Verify that the field-replaceable units (FRUs) installed in your platform support the unified ISSU feature or that you can accept the results of performing the upgrade with some FRUs that do not support unified ISSU.

    • Verify that the protocols and features configured on your platform support the unified ISSU feature or that you can accept the results of performing the upgrade with some protocols and features that do not support unified ISSU.

  • Download the software package from the Juniper Networks Support website at https://www.juniper.net/support/ and place the package on your local server.

    Best Practice

    When you access the Download Software web page for your device, record the md5 checksum. After downloading the software package to your device, confirm that it is not modified in any way by using the file checksum md5 command. For more information about verifying the md5 checksum, see https://kb.juniper.net/InfoCenter/index?page=content&id=KB17665 .

    Note

    Starting with Junos OS Release 16.1R1, while performing a unified ISSU from a FreeBSD 6.1 based Junos OS to an upgraded FreeBSD 10.x based Junos OS, the configuration must be validated on a remote host or on a routing engine. The remote host or the routing engine must be running a Junos OS with an upgraded FreeBSD. In addition, only a few selected directories and files will be preserved while upgrading from FreeBSD 6.1 based Junos OS to FreeBSD 10.x based Junos OS. See Upgrading Junos OS with Upgraded FreeBSD and request system software valdiate on (Junos OS with Upgraded FreeBSD)

Overview

This procedure can be used to upgrade M Series, T Series, MX Series, EX Series, and PTX Series devices that have dual Routing Engines installed and support unified ISSU.

In the example, the hostnames, filenames, and FRUs are representational. When you perform the procedure on your device, the hostnames, filenames, and FRUs are different. The command output is truncated to only show the text of interest in this procedure.

Topology

Figure 1 shows the topology used in this example.

Figure 1: Unified ISSU Example Topology
Unified ISSU Example Topology

Configuration

There are variations of the procedure depending on if you want to install the new software on one or both Routing Engines and if you want to automatically reboot both Routing Engines or manually reboot one of the Routing Engines.

In all cases, you must verify that dual Routing Engines are installed and that graceful Routing Engine switchover (GRES) and nonstop active routing (NSR) are enabled. We recommend that you back up the device software before the upgrade.

To perform a unified ISSU, select the appropriate tasks from the following list:

Verifying Dual Routing Engines and Enabling GRES and NSR

Step-by-Step Procedure

Enabling GRES and NSR is required regardless of which variation of the unified ISSU procedure you use.

To verify that your device has dual Routing Engines and to enable GRES and NSR:

  1. Log in to your device.
  2. Verify that dual Routing Engines are installed in your device by using the show chassis hardware command.
    user@host> show chassis hardware

    The command output contains lines listing Routing Engine 0 and Routing Engine 1.

  3. By default, GRES is disabled; if you have not already done so, enable GRES by including the graceful-switchover statement at the [edit chassis redundancy] hierarchy level on the master Routing Engine.
  4. By default, NSR is disabled; if you have not already done so, enable NSR by including the nonstop-routing statement at the [edit routing-options] hierarchy level.
  5. When you configure NSR, you must also include the commit synchronize statement at the [edit system] hierarchy level so that configuration changes are synchronized on both Routing Engines.
  6. After you have verified your configuration and are satisfied with it, commit the changes by using the commit command.

    When you enable GRES and commit the configuration, the CLI prompt changes to indicate which Routing Engine you are using. For example:

  7. Exit configuration mode by using the exit command.
  8. Verify that NSR is configured on the master Routing Engine (re0) by using the show task replication command.
    {master}
    user@host> show task replication

    In the output, verify that the Synchronization Status field displays Complete.

  9. Verify that GRES is enabled on the backup Routing Engine (re1) by using the show system switchover command.


    user@host> request routing-engine login re1
    {backup}
    user@host> show system switchover

    In the output, verify that the Graceful switchover field state displays On. For more information about the show system switchover command, see show system switchover.

Verifying the Software Versions and Backing Up the Device Software

Step-by-Step Procedure

Unified ISSU requires that both Routing Engines are running the same version of Junos OS before the upgrade. As a preventive measure in case any problems occur during an upgrade, it is a best practice to back up the system software to the device hard disk.

To verify the software versions and back up the device software:

  1. Verify that the same version of Junos OS is installed and running on both Routing Engines by using the show version command.
    {backup}
    user@host> show version invoke-on all-routing-engines
  2. Back up the system software to the device hard disk by using the request system snapshot command on each Routing Engine.Note

    The root file system is backed up to /altroot, and /config is backed up to /altconfig. After you issue the request system snapshot command, the device flash and hard disks are identical. You can return to the previous version of the software only by booting the device from removable media.

    {backup}
    user@host> request system snapshot
    user@host> request routing-engine login re0
    {master}
    user@host> request system snapshot

Adjusting Timers and Changing Feature-Specific Configuration

Step-by-Step Procedure

If you have any of the following feature-specific configuration on your device, perform the appropriate steps.

To adjust timers and change feature-specific configuration:

  1. Bidirectional Forwarding Detection (BFD) sessions temporarily increase their detection and transmission timers during unified ISSU procedures. After the upgrade, these timers revert to the values in use before the unified ISSU started.

    If BFD is enabled on your device and you want to disable the BFD timer negotiation during the unified ISSU, include the no-issu-timer-negotiation statement at the [edit protocols bfd] hierarchy level.

    Note

    If you include this statement, the BFD timers maintain their original values during the unified ISSU, and the BFD sessions might flap during the unified ISSU or Routing Engine switchover, depending on the detection intervals.

  2. If proxy ARP is enabled on your M Series, MX Series, or EX 9200 Series device, remove the unconditional-src-learn statement from the [edit interfaces interface-name unit 0 family inet] hierarchy level.

    By default the statement is not included. This example shows the ge-0/0/1 interface only.

  3. If LACP is enabled on your PTX Series device, remove the lacp statement from the [edit interfaces interface-name aggregated-ether-options] hierarchy level.
  4. If ATM Point-to-Point Protocol (PPP) is enabled on your M Series or T Series device, set the keepalive interval to 10 seconds or greater.

    PPP requires three keepalives to fail before it brings down the session. Thirty seconds (10 seconds x three) provides a safe margin to maintain PPP sessions in case of any traffic loss during the unified ISSU operation.

    This example shows the at-0/0/1 interface only.

  5. If ATM OAM is enabled on your M Series or T Series device, set the OAM F5 loopback cell period to 20 seconds or greater to maintain ATM connectivity across the unified ISSU.

    Include the oam-period statement at the [edit interfaces interface-name unit logical-unit-number] hierarchy level and specify 20 seconds. This example shows the at-0/0/1 interface only.

  6. After you have verified your configuration and are satisfied with it, commit the changes by using the commit command.
  7. Exit configuration mode by using the exit command.

Upgrading and Rebooting Both Routing Engines Automatically

Step-by-Step Procedure

In this procedure, both Routing Engines automatically reboot. Rebooting both Routing Engines automatically is the most common scenario. Variations to this procedure are described in other sections.

Table 1 shows the Routing Engine status prior to starting the unified ISSU.

Table 1: Routing Engine Status Before Upgrading

RE0

RE1

Master

Backup

Old software version installed

Old software version installed

Old software version running

Old software version running

To upgrade and reboot both Routing Engines automatically:

  1. Copy the Junos OS software package to the device by using the file copy ftp://username@hostname.net/filename /var/tmp/filename command.

    We recommend that you copy the package to the /var/tmp directory, which is a large file system on the hard disk.

    {master}
    user@host> file copy ftp://myid@myhost.mydomain.net/jinstall64-14.1R4.10-domestic-signed.tgz /var/tmp/jinstall64-14.1R4.10-domestic-signed.tgz
    Best Practice

    When you access the Download Software web page for your device, record the md5 checksum. After downloading the software package to your device, confirm that it is not modified in any way by using the file checksum md5 command. For more information about verifying the md5 checksum, see https://kb.juniper.net/InfoCenter/index?page=content&id=KB17665 .

  2. On the master Routing Engine, start the upgrade by using the request system software in-service-upgrade package-name reboot command.Note

    Do not try running any additional commands until after the Connection closed message is displayed and your session is disconnected.

    {master}
    user@host> request system software in-service-upgrade /var/tmp/jinstall64-14.1R4.10-domestic-signed.tgz reboot

    When the Routing Engine that was previously the master is rebooted, you are logged out of the device.

  3. Wait a few minutes and then log in to the device again.

    Table 2 shows the Routing Engine status after the unified ISSU.

    Table 2: Routing Engine Status After Upgrading and Rebooting Both Routing Engines

    RE0

    RE1

    Backup

    Master

    New software version installed

    New software version installed

    New software version running

    New software version running

    You are logged in to the new backup Routing Engine (re0).

  4. Verify that both Routing Engines have been upgraded by using the show version command.
    {backup}
    user@host> show version invoke-on all-routing-engines
  5. If you want to, you can optionally display the unified ISSU log messages by using the show log messages command.
  6. If you want to, you can optionally make re0 the master Routing Engine by using the request chassis routing-engine master acquire command.
    {backup}
    user@host> request chassis routing-engine master acquire

    Table 3 shows the Routing Engine status after Step 5 is completed.

    Table 3: Routing Engine Status After Upgrading, Rebooting, and Switching Mastership

    RE0

    RE1

    Master

    Backup

    New software version installed

    New software version installed

    New software version running

    New software version running

  7. Perform the applicable steps in Restoring Feature-Specific Configuration.
  8. If you are satisfied with the results of your testing, you can optionally back up the system software to the device’s hard disk by using the request system snapshot command on each Routing Engine.Note

    The root file system is backed up to /altroot, and /config is backed up to /altconfig. After you issue the request system snapshot command, you cannot easily return to the previous version of the software, because the device flash and hard disks are identical. To return to the previous version of the software, you must boot the device from removable media.

    {master}
    user@host> request system snapshot
    user@host> request routing-engine login re1
    {backup}
    user@host> request system snapshot

Restoring Feature-Specific Configuration

Step-by-Step Procedure

If you have any of the following feature-specific configuration on your device, perform the appropriate steps.

To restore feature-specific configuration:

  1. If BFD is enabled on your device and you previously disabled the BFD timer negotiation, delete the no-issu-timer-negotiation statement at the [edit protocols bfd] hierarchy level.
  2. If proxy ARP is enabled on your M Series, MX Series, or EX9200 device and you previously removed the unconditional-src-learn statement, include the statement again.

    This example shows the ge-0/0/1 interface only.

  3. If LACP is enabled on your PTX Series device and you previously removed the lacp statement, include the statement again.
  4. If ATM PPP is enabled on your M Series or T Series device and you previously set the keepalive interval to 10 seconds or greater, restore the original value.

    This example shows the at-0/0/1 interface only and shows the interval being set to the default 3 seconds.

  5. If ATM OAM is enabled on your M Series or T Series device and you previously set the OAM F5 loopback cell period to 20 seconds or greater, change the configuration back to the original value.

    This example shows the at-0/0/1 interface only and shows the period being set to 10 seconds.

  6. After you have verified your configuration and are satisfied with it, commit the changes by using the commit command.
  7. Exit configuration mode by using the exit command.

Upgrading Both Routing Engines and Rebooting the New Backup Routing Engine Manually

Step-by-Step Procedure

In certain circumstances, you might want to install the new software on only one Routing Engine and reboot only the master until after you can test the new software. A Routing Engine does not start running the new software until after it is rebooted.

The advantage is if the results of your testing requires you to downgrade the software, you can switch Routing Engines to run the old software on one Routing Engine and then install the old software on the other Routing Engine. This is not the typical scenario.

To upgrade both Routing Engines and to reboot the new backup Routing Engine manually:

  1. Perform the steps in Verifying Dual Routing Engines and Enabling GRES and NSR.
  2. Perform the steps in Verifying the Software Versions and Backing Up the Device Software.
  3. Perform the steps in Adjusting Timers and Changing Feature-Specific Configuration.
  4. Copy the Junos OS software package to the device using the file copy ftp://username@hostname.net/filename /var/tmp/filename command.

    We recommend that you copy the package to the /var/tmp directory, which is a large file system on the hard disk.

    {master}
    user@host> file copy ftp://myid@myhost.mydomain.net/jinstall64-14.1R4.10-domestic-signed.tgz /var/tmp/jinstall64-14.1R4.10-domestic-signed.tgz
    Best Practice

    When you access the Download Software web page for your device, record the md5 checksum. After downloading the software package to your device, confirm that it is not modified in any way by using the file checksum md5 command. For more information about verifying the md5 checksum, see https://kb.juniper.net/InfoCenter/index?page=content&id=KB17665 .

    Table 4 shows the Routing Engine status prior to starting the unified ISSU.

    Table 4: Routing Engine Status Before Upgrading and Manually Rebooting the Backup Routing Engine

    RE0

    RE1

    Master

    Backup

    Old software version installed

    Old software version installed

    Old software version running

    Old software version running

  5. On the master Routing Engine, start the upgrade by using the request system software in-service-upgrade package-name command without the reboot option.
    {master}
    user@host> request system software in-service-upgrade /var/tmp/jinstall64-14.1R4.10-domestic-signed.tgz

    Table 5 shows the Routing Engine status after the unified ISSU and before manually rebooting the backup Routing Engine.

    Table 5: Routing Engine Status After Upgrading and Before Manually Rebooting the Backup Routing Engine

    RE0

    RE1

    Backup

    Master

    New software version installed

    New software version installed

    Old software version running

    New software version running

  6. Verify that the new backup, (old master) Routing Engine (re0), is still running the previous software image and that the new master Routing Engine (re1) is running the new software image, by using the show version command.
    {backup}
    user@host> show version invoke-on all-routing-engines
  7. At this point, if you do not want to install the newer software version on the new backup Routing Engine (re0), issue the request system software delete package-name command on it.

    Otherwise, to complete the upgrade, go to the next step.

  8. Reboot the new backup Routing Engine (re0) by issuing the request system reboot command.
    {backup}
    user@host> request system reboot

    If you are not on the console port, you are disconnected from the device session.

    Table 6 shows the Routing Engine status after the unified ISSU, after rebooting the backup Routing Engine, but before switching mastership.

    Table 6: Routing Engine Status After Upgrading, Manually Rebooting, and Before Switching Mastership

    RE0

    RE1

    Backup

    Master

    New software version installed

    New software version installed

    New software version running

    New software version running

  9. Wait a few minutes, then log in to the device again.

    You are logged in to the new backup Routing Engine (re0).

  10. Verify that both Routing Engines have been upgraded by using the show version command.
    {backup}
    user@host> show version invoke-on all-routing-engines
  11. If you want to, you can optionally display the unified ISSU log messages by using the show log messages command.
  12. If you want to, you can optionally make re0 the master Routing Engine by using the request chassis routing-engine master acquire command:
    {backup}
    user@host> request chassis routing-engine master acquire

    Table 7 shows the Routing Engine status after the unified ISSU, after rebooting the backup Routing Engine, and after switching mastership.

    Table 7: Routing Engine Status After Upgrading, Manually Rebooting, and Switching Mastership

    RE0

    RE1

    Master

    Backup

    New software version installed

    New software version installed

    New software version running

    New software version running

  13. Perform the applicable steps in Restoring Feature-Specific Configuration.
  14. If you are satisfied with the results of your testing, you can optionally back up the system software to the device’s hard disk by using the request system snapshot command on each Routing Engine.Note

    The root file system is backed up to /altroot, and /config is backed up to /altconfig. After you issue the request system snapshot command, you cannot easily return to the previous version of the software, because the device flash and hard disks are identical. To return to the previous version of the software, you must boot the device from removable media.

    {master}
    user@host> request system snapshot
    user@host> request routing-engine login re1
    {backup}
    user@host> request system snapshot

Upgrading and Rebooting Only One Routing Engine

Step-by-Step Procedure

In certain circumstances you might want to install the new software on only one Routing Engine.

The advantage is if the results of your testing requires you to downgrade the software, you can switch Routing Engines to run the old software on one Routing Engine and then install the old software on the other Routing Engine. This is not the typical scenario.

Table 8 shows the Routing Engine status prior to starting the unified ISSU.

Table 8: Routing Engine Status Before Upgrading and Rebooting One Routing Engine

RE0

RE1

Master

Backup

Old software version installed

Old software version installed

Old software version running

Old software version running

To upgrade and rebooting only one Routing Engine:

  1. Perform the steps in Verifying Dual Routing Engines and Enabling GRES and NSR.
  2. Perform the steps in Verifying the Software Versions and Backing Up the Device Software.
  3. Perform the applicable steps in Adjusting Timers and Changing Feature-Specific Configuration.
  4. Copy the Junos OS software package to the device by using the file copy ftp://username@hostname.net/filename /var/tmp/filename command.

    We recommend that you copy the package to the /var/tmp directory, which is a large file system on the hard disk.

    {master}
    user@host> file copy ftp://myid@myhost.mydomain.net/jinstall64-14.1R4.10-domestic-signed.tgz /var/tmp/jinstall64-14.1R4.10-domestic-signed.tgz
    Best Practice

    When you access the Download Software web page for your device, record the md5 checksum. After downloading the software package to your device, confirm that it is not modified in any way by using the file checksum md5 command. For more information about verifying the md5 checksum, see https://kb.juniper.net/InfoCenter/index?page=content&id=KB17665 .

  5. On the master Routing Engine, start the upgrade by using the request system software in-service-upgrade package-name no-old-master-upgrade command.
    {master}
    user@host> request system software in-service-upgrade /var/tmp/jinstall64-14.1R4.10-domestic-signed.tgz no-old-master-upgrade

    Table 9 shows the Routing Engine status after the unified ISSU upgrades the master Routing Engine but before the backup Routing Engine is upgraded.

    Table 9: Routing Engine Status After Upgrading One Routing Engine and Before Upgrading the Other Routing Engine

    RE0

    RE1

    Backup

    Master

    Old software version installed

    New software version installed

    Old software version running

    New software version running

  6. Verify that the new backup, (old master) Routing Engine (re0), is still running the previous software image and that the new master Routing Engine (re1) is running the new software image, by using the show version command.
    {backup}
    user@host> show version invoke-on all-routing-engines
  7. If your testing is complete and you want to install the new software on the backup Routing Engine, you must first disable GRES and NSR on both Routing Engines and commit the configuration.
  8. Install the new software on the backup Routing Engine (re0) by using the request system software add /var/tmp/jinstall64-14.1R4.10-domestic-signed.tgz command.
    user@host> request system software add /var/tmp/jinstall64-14.1R4.10-domestic-signed.tgz
  9. Reboot re0 by using the request system reboot command.
    user@host> request system reboot

    If you are not on the console port, you are disconnected from the router session.

  10. After waiting a few minutes, log in to the device again.

    You are logged in to the backup Routing Engine (re0).

  11. Verify that both Routing Engines are running the new software image by using the show version command.
    {backup}
    user@host> show version invoke-on all-routing-engines
  12. If you want to, you can optionally display the unified ISSU log messages by using the show log messages command.
  13. If you want to, make re0 the master Routing Engine by using the request chassis routing-engine master acquire command.
    {backup}
    user@host> request chassis routing-engine master acquire

    Table 10 shows the Routing Engine status after the unified ISSU, after rebooting the backup Routing Engine, and after switching mastership.

    Table 10: Routing Engine Status After Upgrading, Manually Rebooting, and Switching Mastership

    RE0

    RE1

    Master

    Backup

    New software version installed

    New software version installed

    New software version running

    New software version running

  14. Enable GRES and NSR again by performing the steps in Verifying Dual Routing Engines and Enabling GRES and NSR.
  15. Perform the applicable steps in Restoring Feature-Specific Configuration.
  16. If you are satisfied with the results of your testing, you can optionally back up the system software to the device’s hard disk by using the request system snapshot command on each Routing Engine.Note

    The root file system is backed up to /altroot, and /config is backed up to /altconfig. After you issue the request system snapshot command, you cannot easily return to the previous version of the software, because the device flash and hard disks are identical. To return to the previous version of the software, you must boot the device from removable media.

    {master}
    user@host> request system snapshot
    user@host> request routing-engine login re1
    {backup}
    user@host> request system snapshot