Nonstop Software Upgrade Checklist for QFabric Systems

Preparing for an NSSU

Note: Before you perform a nonstop software upgrade, contact JTAC to perform a pre-upgrade health check on the QFabric system.

In preparation for an NSSU, ensure the following:

• The NSSU is being done during a maintenance window.
• Console connectivity is available for both Director groups.
• The existing configuration has been backed up.

Next, complete the following steps:

1. Collect the output of the lsvm and dns.dump commands. lsvm command provides the universal unique identifiers (UUIDs) of all the virtual machines, and the dns.dump provides the IP addresses of all the components that can be used in case of failure.
2. Run the following commands to perform sanity checks of the entire QFabric system:

   script /opt/dcf/scripts/director_upgrade_sanity_check.pl
   script /opt/dcf/scripts/fabric_upgrade_sanity_check.pl
   script /opt/dcf/scripts/nodegroup_upgrade_sanity_check.pl
   

   Note: In case of any errors, contact Juniper Networks Technical Assistance Center (JTAC).

3. Enter the request component login DRE-0 command to connect to DRE-0. Check the available storage using the show system storage command. There should be about 500 MB of space available in the /var partition. If not, delete the files under /var/log. Also, check the /var/log/kdm and /var/tmp directories for any large files that can be deleted. Issue the clear log filename-with-path command to clear the large files in the /var/log/ and in /var/tmp/ directories.
4. Modify the multicast discovery advertisements (MDA) timer on all Fibre Channel (FC) and Fibre Channel over Ethernet (FCoE) gateways connected to QFabric, if applicable. We recommend this step to prevent FC session drops during NSSU. Enter the following command on FC and FCoE gateways: set fc-fabrics fabric-name protocols fip fka-adv-period 45000.
5. Enter the following commands to update the system log configuration. This will reduce the amount of system log messages added to the database during upgrade. Logging to an external server does not have any impact during upgrade.
   • set system syslog host syslog server address
   • For remote servers:

     set system syslog file cli_commands interactive-commands any

     set system syslog file for all files configured any critical

     set system syslog file messages any critical

Upgrading Director Groups

The NSSU process individually upgrades members of a Director group, so that one device in the group is always operational. It switches the mastership of the Routing Engine processes to the backup Director device, before upgrading the master Director device.

• Run the sanity check scripts to ensure that the Director device is ready to be upgraded.

  root@dg0 ~]# /opt/dcf/scripts/fabric_upgrade_sanity_check.pl
  

• Verify whether both the Director devices are online.

  root@Qfabric_device> show fabric administration inventory director-group status 

Enter the following commands to clean the storage and system log.

request system storage cleanup director-group
request system storage cleanup infrastructure FM-0
request system storage cleanup qfabric component all
clear log messages

After these processes are complete, you can run the NSSU on the master Director group. After the upgrade is complete, verify the process using the following commands:

show version component director-group
show version component DRE-0
show fabric admin inventory
show fabric admin inventory infrastructure
show fabric admin inventory director-group status

Restore the system log configuration that you updated in step 5.

Upgrading QFabric

• Run the sanity check scripts to ensure that the Director device is ready to be upgraded.

  root@dg0 ~]# /opt/dcf/scripts/fabric_upgrade_sanity_check.pl
  

Enter the following commands to clean the storage.

request system storage cleanup infrastructure FC-0
 request system storage cleanup infrastructure FC-1
request system storage cleanup interconnect-device ic-name
 

After these processes are complete, you can run the NSSU on QFabric. After the upgrade is complete, verify the process using the following commands:

show version component fabric
show fabric admin inventory
show fabric admin inventory infrastructure
show chassis fabric connectivity

Upgrading Network Node Groups

• Run the sanity check scripts to ensure that the Node groups are ready to be upgraded.

  root@dg0 ~]# /opt/dcf/scripts/nodegroup_upgrade_sanity_check.pl
  

Enter the following commands to clean the storage.

 request system storage cleanup node-group NW-NG-0 

After these processes are complete, you can run the NSSU on the network Node groups. After the upgrade is complete, verify the process using the following commands:

show version component NW-NG-0
show fabric admin inventory NW-NG-0
show fabric admin inventory infrastructure
show chassis fabric connectivity

Note: Follow the same procedure for redundant server Node groups (RSNGs) and server Node groups (SNGs).