Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
ON THIS PAGE
 

Verifying Nonstop Software Upgrade for QFabric Systems

Note:

Before you perform a nonstop software upgrade, contact JTAC to perform a pre-upgrade health check on the QFabric system.

This topic discusses how you can monitor the progress of each of the three steps in a nonstop software upgrade. By identifying the key actions and events that define this process, you can track the status of the upgrade with confidence.

Tip:

When performing a nonstop software upgrade, open two SSH sessions to the QFabric CLI. Use one session to monitor the upgrade itself and use a second session to verify that the QFabric system components respond to operational mode commands as expected.

Verifying a Director Group Nonstop Software Upgrade

Purpose

During the Director group portion of a nonstop software upgrade, you should expect to see the Director device that hosts the CLI session selected as the primary device. When primary role of all processes moves to the primary, the QFabric system upgrades the backup Director device and this Director device reboots. After the backup Director device comes back online, the primary Director device suspends CLI operations for 15 minutes, upgrades itself, and reboots. At this point, the backup becomes the new primary Director device and you can issue CLI operational commands. Finally, the former primary comes back online as a backup and both devices are operational once again. In addition to the steps below, you can issue the show system software upgrade status command to view the progress of the upgrade.

Action

  1. In one SSH session to the QFabric CLI, verify the current status of the QFabric system by issuing the show fabric administration inventory, show fabric administration inventory director-group status, and show fabric session-host commands. In this case, Director device DG0 is the primary device but DG1 hosts the CLI session.

  2. In a second SSH session to the QFabric CLI, issue the request for the Director group nonstop software upgrade.

  3. If the CLI session is being hosted by the primary Director device, skip to step 4. However, if the CLI session is hosted by the backup Director device, the Director group primary role switches to the backup device after you issue the nonstop software upgrade command. In this example, primary role switches to Director device DG1.

  4. The Director group nonstop software upgrade process continues by downloading and installing software for the fabric manager Routing Engines and the Director devices.

  5. When the system upgrades and reboots the backup Director device DG0, notice how this device is not displayed in the output of the show fabric administration inventory director-group status command. Because Director device DG1 appears, this means that the DG1 is operational and acts as the primary device.

    Note:

    If your second SSH session is being hosted by the rebooting Director device, your session terminates and you need to log back in to establish a new session running on the active Director device.

  6. The upgrade continues with primary Director device DG1 suspending CLI services for 15 minutes, transferring primary role to Director device DG0, and then rebooting Director device DG1 (which terminates the CLI session).

  7. Upon reopening the SSH session, notice that Director device DG0 is now the primary device hosting the session and Director device DG1 does not appear in the QFabric system inventory while it is rebooting.

  8. When Director device DG1 comes back online, it returns to the QFabric system inventory as a backup Director device and hosts some of the Routing Engine processes (which should appear load balanced between the primary and backup Director devices).

Meaning

Verifying a Fabric Nonstop Software Upgrade

Purpose

During the fabric portion of a nonstop software upgrade, you should expect to see both fabric control Routing Engines upgrade first, followed by the upgrade of each Interconnect device one at a time. In addition to the steps below, you can issue the show system software upgrade status command to view the progress of the upgrade.

Action

  1. In an SSH session to the QFabric CLI, issue the request for the fabric nonstop software upgrade.

  2. When the fabric components reboot, they appear as Disconnected in the output of the show fabric administration inventory infrastructure fabric-controls and show fabric administration inventory interconnect-devices commands.

  3. When the fabric components return to full service, they appear as Connected in the output of the show fabric administration inventory command.

Meaning

Verifying a Redundant Server Node Group Nonstop Software Upgrade

Purpose

During the redundant server Node group portion of a nonstop software upgrade, you should expect to see the backup Node device upgrade first, followed by the upgrade of the primary Node device. Server Node groups with a single device upgrade the device in the same way as a standalone switch. In addition to the steps below, you can issue the show system software upgrade status command to view the progress of the upgrade.

Action

  1. In an SSH session to the QFabric CLI, issue the request for the redundant server Node group nonstop software upgrade.

  2. Issue the show system software upgrade status command to view the status of the upgrade.

  3. During the redundant server Node group upgrade, the backup Node device (in this case, P1571-C) is upgraded first and appears in the Disconnected state in the output of the show fabric administration inventory command.

  4. After the backup Node device comes back online, the primary Node device (in this case, P1550-C) appears in the Disconnected state in the output of the show fabric administration inventory command while the primary Node device upgrades its software.

  5. After both Node devices in the redundant server Node group come back online, both Node devices appear as Connected to indicate the successful completion of the Node group nonstop software upgrade step.

Verifying a Network Node Group Nonstop Software Upgrade

Purpose

During the network Node group portion of a nonstop software upgrade, you should expect to see the backup network Node group Routing Engine upgrade first, followed by the Node devices within the network Node group upgrading one at a time, and ending with the upgrade of the primary network Node group Routing Engine. In addition to the steps below, you can issue the show system software upgrade status command to view the progress of the upgrade.

Note:

If you configure an upgrade group for Node groups containing 2 or more Node devices, all Node devices within the upgrade group reboot at the same time.

Action

  1. In an SSH session to the QFabric CLI, issue the request for the network Node group nonstop software upgrade.

  2. Issue the show system software upgrade status command to view the status of the upgrade.

  3. Verify the progress of the upgrade by issuing the show chassis nonstop-upgrade node-group, show fabric administration inventory, show fabric administration inventory infrastructure, and show fabric administration inventory node-groups NW-NG-0 commands. You should see the backup network Node group Routing Engine reboot first, followed by each Node device within the network Node group, and ending with the reboot of primary network Node group Routing Engine. Restarting devices appear as Disconnected in the output of the show fabric administration inventory command and restarting Routing Engines do not appear in output of the show fabric administration inventory infrastructure command until they return to service.

Related Documentation