[Contents] [Prev] [Next] [Index] [Report an Error] [No Frames]

JUNOSe 9.1.x System Basics Configuration Guide > Configuring a Unified In-Service Software Upgrade > Unexpected Application-Specific Behavior During Unified ISSU

Unexpected Application-Specific Behavior During Unified ISSU

This section describes the behavior of applications that vary from the expected behavior during a unified in-service software upgrade.

AAA Authentication and Authorization Disabled

Authentication and command authorization are temporarily disabled on the serial console connection during the upgrade phase. You can connect to the serial console and issue the reload, show issu, and show version commands without the action of authentication and authorization servers, such as RADIUS or TACACS+.

ATM Affected Behaviors

The following aspects of ATM behavior during unified ISSU are different than the behavior during normal router operations.

ILMI Sessions Not Maintained

The router does not maintain ILMI sessions during a unified in-service software upgrade. The router terminates all ILMI sessions and restarts them during the upgrade. If the ILMI protocol is enabled on any port, you are warned during the initialization phase when unified ISSU is verifying the prerequisites for the upgrade. You can choose to continue the upgrade—and bring down the sessions—or to halt the in-service software upgrade.

OAM CC Effects on VCC

When an ATM VC is configured as an OAM CC source, periodic OAM cells are generated for about 15 seconds. The device configured as the OAM CC sink is likely to declare the VCC down during this time. Unified ISSU generates a warning when it detects an OAM CC source configuration during the initialization phase while unified ISSU is verifying the prerequisites for the upgrade. You can choose to continue or halt the in-service software upgrade

When an ATM VC is configured as OAM CC sink, it cannot receive OAM CC cells generated by the source for about 15 seconds. The OAM CC does not time out and the VCC is not declared down. OAM CC cell generation resumes when the unified ISSU operation is completed.

OAM VC Integrity Verification Cessation

During the unified ISSU operation, verification of OAM VC integrity stops. This verification resumes when the unified ISSU operation is completed.

ATM does not respond to incoming OAM loopback cells during the upgrade for a period of less than 30 seconds. The lack of response might cause ATM peers to declare VCC (VPC) down.

Port Data Rate Monitoring Cessation

The monitoring of ATM port data rates is halted during a unified in-service software upgrade. Monitoring resumes immediately after the unified ISSU operation is completed. The data rates reported by the show atm interface command are inaccurate for the period of one configured load interval after unified ISSU is completed.

VC and VP Statistics Monitoring Halts Unified ISSU Progress

A unified in-service software upgrade cannot proceed if VC or VP statistics monitoring is in progress.

DHCP Affected Behaviors

DHCP Common Component Information Suspended

The common DHCP component supports unified ISSU. This component configures option 60 vendor-option strings when you issue the set dhcp vendor-option command. The DHCP common component ceases all CLI and SNMP operations when you issue issu start command. You can therefore obtain no information about the common DHCP infrastructure until the in-service software upgrade is completed.

DHCP External Server Prevents Unified ISSU Operation

The DHCP external server application does not support unified ISSU. You must completely unconfigure this application from all virtual routers to perform a unified in-service software upgrade.

DHCP Relay and DHCP Relay Proxy Prevent Unified ISSU

The DHCP relay and DHCP relay proxy applications do not support unified ISSU. You must completely unconfigure these applications from all virtual routers to perform a unified in-service software upgrade.

DHCP Packet Capture Halted on Line Modules

The DHCP packet capture application supports unified ISSU in that its configuration does not halt a unified in-service software upgrade. However, packet capture on line modules is halted during the upgrade phase. Packets are not captured and buffered for later forwarding to the SRP module during this phase. Capture resumes automatically during the service restoration phase.

DoS Protection State Freeze

The denial-of-service (DoS) protection application freezes its state when the in-service software upgrade is initiated. Any suspicious control flow, protocol, or priority remains suspicious until unified ISSU completes.

Freezing the DoS protection state prevents any active control flows from interfering with the system while the unified ISSU is in progress. However, no new control flows, protocols, or priorities are monitored for suspicious activity, and no suspicious activity can be detected until the upgrade is completed.

NOTE: Because of this limitation on DoS functionality, we recommend that you do not initiate unified ISSU until all suspicious control flows, protocols, and priorities have been resolved.

When the in-service software upgrade is completed, the DoS protection application resumes attending to all dynamic state that was frozen at the beginning of the unified ISSU process.

Some suspicious control flows might remain in a suspicious list based on your configuration. If the upgrade software version has DoS protection classification algorithms that are better or different than in the previous version, you might want to clear these suspicious control flows to enable the classification algorithms to determine whether the flow is still considered to be suspicious.

Ethernet Affected Behaviors

The following aspects of Ethernet behavior during a unified in-service software upgrade are different than during normal router operations.

ARP Packets Briefly Not Sent or Received

There is a brief period at the end of the upgrade phase when ARP packets are not sent or received. This event can affect ARP entries on attached devices that were in the process of being aged out.

Link Aggregation interruption

During the in-service software upgrade, LACP PDUs are not generated or received for about 15 seconds on Ethernet ports configured with LACP,

This interruption has no effect on the local side of the link because JUNOSe software generates LAC PDU packets every 30 seconds. The link is not declared down unless packets are missed three times. LACP packet generation continues when the unified ISSU operation is completed.

If a device on the other end of the link is configured with the short timeout of one second, then the device is likely to declare the link to be down and remove the link from the LAG bundle.

Port Data Rate Monitoring Halted

The monitoring of Ethernet port data rate is halted during a unified in-service software upgrade. Monitoring resumes immediately after the unified ISSU operation is completed. The data rates reported by the show interface command are inaccurate for the period of one configured load interval after unified ISSU is completed.

VLAN Statistics Monitoring Halts Unified ISSU Progress

A unified in-service software upgrade cannot proceed if VLAN statistics monitoring is in progress.

FTP Server File Transfers Halted

During a unified in-service software upgrade, file transfers that involve the FTP server are halted because of the SRP module switchover during the upgrade.

For this reason, the FTP server does not support unified ISSU. You can continue with the in-service software upgrade only when the FTP server is not enabled. You must disable the FTP server to perform the upgrade.

IS-IS Effects on Graceful Restart and Network Stability

IS-IS has the following issues to consider before you begin a unified in-service software upgrade:

Configuring Graceful Restart Before Unified ISSU Begins

You must configure IS-IS graceful restart on the router and on all IS-IS neighbors before you begin the in-service software upgrade. When the unified ISSU process verifies the upgrade requirements during the initialization phase, it detects whether graceful restart is configured. If it is not configured, the CLI displays a warning message and prompts you to proceed or halt. You can stop at this point to configure graceful restart.

If instead you proceed, the in-service software upgrade can complete successfully, but the IS-IS neighbors are likely to break the adjacencies with the upgrading router and consider that routes formerly reached through this router are now unreachable. When the in-service software upgrade completes and the routing protocols restart, the IS-IS neighbors can relearn the routes through the router.

When you issue the issu start command, IS-IS lengthens its hello timer values and sends LSPs with the new values. The upgrade proceeds when the IS-IS neighbors have acknowledged the new values.

Configuring Graceful Restart When BGP And LDP are Configured

When BGP, IS-IS, and LDP are all configured on a router on which you will perform a unified in-service software upgrade, ensure that the IS-IS graceful restart timeout is longer than the LDP graceful restart timeout. The IS-IS graceful restart does not complete when the LDP graceful restart timeout is longer than the IS-IS graceful restart timeout. Configure IS-IS graceful timeout with the nsf t3 command. Configure LDP graceful restart timeout with the mpls ldp graceful-restart timers max-recovery command.

Routing Around the Restarting Router to Minimize Network Instability

NOTE: The situation described in this section is very uncommon. This rare circumstance arises when you have redundant uplinks to the core and network topology changes cause routes to go through the upgrading router. In a typical network design, this is not an issue and you do not need to route peers around the upgrading router,

During the unified ISSU upgrade phase, network instability can result if the restarting router goes into an unstable state after the unified ISSU process fails. Some IS-IS traffic loss occurs during the resulting line module resets. For those reasons, you might want IS-IS peers to route around the router that is being upgraded.

You can use the overload advertise-high-metric issu command to cause the router to advertise a high metric to its neighbors so that they route around the upgrading router. When you issue the issu start command, the router raises the metric to the maximum link cost on all interfaces running IS-IS. The maximum value depends on the metric type. IS-IS neighbors then choose a path with lower metrics to reach any destination that was previously reached through the upgrading router. When unified ISSU is completed, IS-IS reverts the metrics back to the values that were configured before the in-service software upgrade.

When traffic engineering has been configured, the traffic engineering metrics are also increased. New tunnels are not established through the upgrading router and any tunnels undergoing re-optimization in other routers go around the upgrading router.

IS-IS support for unified ISSU does not depend on this configuration. If you do not issue the overload advertise-high-metric issu command, the in-service software upgrade can still proceed to successful completion without disrupting IS-IS functionality.

overload advertise-high-metric issu

L2TP Failover of Established Tunnels

L2TP never declares itself as unified ISSU unsafe. However, unified ISSU forces an L2TP failover for all established tunnels. Successful recovery of a tunnel and its sessions following the in-service software upgrade requires a successful L2TP failover resynchronization, either by the L2TP silent failover method or the L2TP failover protocol.

When the unified ISSU operation attempts to verify the upgrade prerequisites, a warning message is generated if any tunnels are present for which failover resynchronization is disabled.

You can use the show l2tp tunnel failover-resync disable command to identify the tunnels referred to by the warning message. The command enables filtering based upon the effective failover resynchronization mechanism:

host1#show l2tp tunnel failover-resync disable

L2TP tunnel 2/1 is Up with 1 active session

1 L2TP tunnel found

If a successful failover resynchronization cannot be performed for a tunnel following the upgrade, then the tunnel and all of its sessions are subject to disconnection.

L2TP automatically detects a peer L2TP disconnect after the in-service software upgrade is completed by detecting a control channel failure.

When peer LNSs are not configured with PPP keepalives or inactivity timeouts, you must configure an inactivity timeout for L2TP on the LAC. This timeout enables the router to detect a PPP disconnect when signaling has been dropped during the unified ISSU forwarding interruption. In the absence of this configuration, the connection at the LAC and LNS is left as logged in for an extended period of time following the upgrade.

OSPF Effects on Graceful Restart, Timeouts, and Network Stability

OSPF has the following issues to consider before you begin a unified in-service software upgrade:

Configuring Graceful Restart Before Unified ISSU Begins

You must configure OSPF graceful restart before you begin the in-service software upgrade. When the unified ISSU process verifies the upgrade requirements during the initialization phase, it detects whether graceful restart is configured. If it is not configured, the CLI displays a warning message and prompts you to proceed or halt. You can stop at this point to configure graceful restart.

If instead you proceed, the in-service software upgrade can complete successfully, but the OSPF neighbors are likely to break the adjacencies with the upgrading router and consider that routes formerly reached through this router are now unreachable. When the in-service software upgrade completes and the routing protocols restart, the IS-IS neighbors can relearn the routes through the router.

You must also ensure that the OSPF neighbors have been configured as graceful restart helper routers. During the unified ISSU initialization phase, OSPF graceful restart on the upgrading router cannot verify whether its neighbors are helper routers, and reports that fact by means of the CLI.

Configuring Graceful Restart When BGP And LDP are Configured

When BGP, LDP, and OSPF are all configured on a router on which you will perform a unified in-service software upgrade, ensure that the OSPF graceful restart timeout is longer than the LDP graceful restart timeout. The OSPF graceful restart does not complete when the LDP graceful restart timeout is longer than the OSPF graceful restart timeout. Configure OSPF graceful restart timeout with the graceful-restart restart-time command. Configure LDP graceful restart timeout with the mpls ldp graceful-restart timers max-recovery command.

Configuring a Longer Dead Interval Than Normal

To prevent OSPF from timing out to the OSPF neighbors, you must configure a dead interval that is longer than the period required for the in-service software upgrade to complete. You must use the value provided by unified ISSU in a warning message displayed during the initialization phase.

Routing Around the Restarting Router to Minimize Network Instability

NOTE: The situation described in this section is very uncommon. This rare circumstance arises when you have redundant uplinks to the core and network topology changes cause routes to go through the upgrading router. In a typical network design, this is not an issue and you do not need to route peers around the upgrading router,

During the unified ISSU upgrade phase, network instability can result if the restarting router goes into an unstable state after the unified ISSU process fails. Some OSPF traffic loss occurs during the resulting line module resets. For those reasons, you might want OSPF peers to route around the router that is being upgraded.

You can use the overload advertise-high-metric issu command to cause the router to advertise a high link cost to its neighbors so that they route around the upgrading router. When you issue the issu start command, the router raises the link cost to the maximum link cost on all interfaces running OSPF. The higher cost is advertised in the OSPF LSAs. OSPF neighbors then choose a path with lower metrics to reach any destination that was previously reached through the upgrading router. When unified ISSU is completed, OSPF reverts the link costs back to the values that were configured before the in-service software upgrade.

When traffic engineering has been configured, the traffic engineering metrics are also increased. New tunnels are not established through the upgrading router and any tunnels undergoing re-optimization in other routers go around the upgrading router.

OSPF support for unified ISSU does not depend on this configuration. If you do not issue the overload advertise-high-metric issu command, the in-service software upgrade can still proceed to successful completion without disrupting OSPF functionality.

overload advertise-high-metric issu

PIM Suspended During Unified ISSU

You can minimize PIM traffic loss during the in-service software upgrade by issuing the ip pim dr-priority command to set a priority so that PIM neighbors do not forward traffic through the upgrading router. By default, a PIM interface has a priority of one. If you set the priority to one, the lowest possible priority, then the upgrading router is not selected to be a designated router in the PIM network if an interface on another router in that network has a higher priority.

ip pim dr-priority

Subscriber Logins and Logouts Suspended During Unified ISSU

All new subscriber logins are ignored during the upgrade phase. New subscriber logouts are cached and processed after the unified ISSU operation is completed.

Subscriber Statistics Accumulation or Deletion

All subscriber statistics present in the line modules are cleared when the line module forwarding planes are upgraded. For this reason, the router has to read the statistics from the forwarding plane before it is upgraded.

However, forwarding through the line modules continues after that point, until the line module forwarding plane is upgraded. Some statistics can therefore accumulate in the forwarding plane in the interval between these two events. These statistics are not preserved across the upgrade.

Statistics for subscribers that log out during the forwarding plane upgrade are collected and reported before the forwarding plane is reloaded. Statistics are not collected for any subscribers who are connected before you issue the issu start command but who log out before the forwarding plane upgrade is completed.

The following subscriber statistics are preserved across the upgrade:

All other statistics are set to zero, including all statistics belonging to the SNMP generic interface MIB for every interface layer.

SONET/SDH Behavior During Unified ISSU

During a unified in-service software upgrade, several aspects of SONET behavior differ from normal operation.

During a unified in-service software upgrade, the LOS does not last more than 2.5 seconds. The remote device detect a momentary LOS but does not perceive this short LOS as a link failure and does not bring the link down,

TACACS+ Services Not Available

During the upgrade phase of unified ISSU, TACACS+ services are unavailable. If you have configured AAA authentication for Telnet (with the aaa new-model command) this lack of availability affects CLI authentication, authorization, and accounting activities.

Because there is no alternate method of accounting other than TACACS, CLI exec and command accounting does not work during this phase.

Interruption in Traffic Forwarding for Layer 3 Routing and Signaling Protocols

The routing protocols are affected by two interruptions in traffic forwarding caused by the in-service software upgrade during the upgrade phase.

The protocols must gracefully restart to come back online, recover their routing state on the newly active SRP module, and respond to their peers. Therefore, you must enable graceful restart for all protocols before you begin the in-service software upgrade. All neighbors of the routing protocols must also be configured to support graceful restart.

A data plane outage of about one second also takes place during the switchover of the fabric from the active primary SRP module to the standby SRP module.

If capable, routing protocols temporarily lengthen their timers to survive the outages. During the initialization phase, unified ISSU checks for timers that are set too short and whether the protocol enables timer renegotiation. If these checks fail, unified ISSU generates a warning and enables you to reconfigure the protocols before you issue the issu start command.

We recommend that you configure timers to be longer than usual for the routing protocols that cannot renegotiate timers. You can use bidirectional forwarding detection (BFD) to quickly detect forwarding interruptions.

Table 49 describes how individual routing protocols behave during a unified in-service software upgrade.

Table 49: Behavior of Routing Protocols During a Unified In-Service Software Upgrade 
Protocol
Behavior

BFD

BFD renegotiates its timers as needed. Typically, the timers are lengthened until the SRP module switchover takes place, then shortened for the forwarding plane upgrade, and finally shortened to the original configured values.

BGP

The configured BGP timers are typically long enough to survive the forwarding outages. If, not, unified ISSU generates a warning message.

BGP sends out keepalive messages immediately before and immediately after both the SRP module switchover and the forwarding plane restart, independent of the interval since it last sent them.

IS-IS

If necessary, temporarily lengthens the hello timers.

LDP

Unified ISSU warns you if the hello timers or the keepalive timers are not long enough to survive the forwarding plane upgrade.

LDP sends out hello messages and keepalive messages immediately before and immediately after both the SRP module switchover and the forwarding plane restart, independent of the interval since it last sent them.

OSPF

OSPF timers are not negotiable between peers. Unified ISSU generates a warning if the hello timers or the keepalive timers are not long enough to survive the forwarding plane upgrade.

OSPF begins a graceful restart before the SRP module switchover. When you configure graceful restart before the in-service software upgrade, you must ensure that the graceful restart times are long enough to allow recovery.

OSPF sends out hello messages and keepalive messages immediately before and immediately after forwarding plane restart, independent of the interval since it last sent them.

PIM

If necessary, temporarily lengthens the hold times in hello messages. PIM guarantees that at least one hello message with a lengthened hold time is sent to each neighbor.

If necessary, increases the join-prune hold time. PIM guarantees that at least one join-prune message with a lengthened hold time is sent to each neighbor.

RIP

RIP timers do not affect unified ISSU.

RSVP-TE

If necessary, temporarily lengthens the graceful restart timers to survive the SRP module switchover.

If necessary, lengthens the hello timers to survive the forwarding plane upgrade.

You might want some or all traffic to be routed around the upgrading router rather than accept a forwarding loss during the forwarding interruption. To do so, you must configure your routing protocols appropriately. For example, you might raise the link cost in IS-IS and OSPF, causing their neighbors to seek alternate routes that have lower link costs. In PIM, you can set the priority for the router interface to zero to ensure that the upgrading router is not selected as a designated router.

Recommended Routing Protocol Timer Settings

You can use the default values for many of the routing protocol timers with no adverse effect on a unified in-service software upgrade. For other timers, we recommend particular values, as described in Table 50.

Table 50: Recommended Routing Protocol Timer Settings 
Protocol
Timers

BFD

Use the default timers.

BGP

Use the default timers, including graceful restart default timers.

DVMRP

Use the default timers.

IGMP

Use the default timers.

IS-IS

Use the default timers, including graceful restart default timers.

LDP

Use the default timers, including graceful restart default timers, except for the following:

  • Set the hello hold time to at least 901 seconds for a helper or a restarter configuration for a link-level adjacency or for LDP targeted sessions.

OSPF

Use the default timers, including graceful restart default timers, except for the dead interval. Set the OSPF dead interval to at least 301 seconds.

PIM

Set the query interval to at least 210 seconds.

ISSU generates a warning for any of the following conditions, but you can ignore the warning without causing a higher FC outage:

  • The current router is a DR.
  • The current router is configured as an Auto RP mapping agent and is chosen as the RP for any group.
  • The current router is an elected or candidate BSR, or if BSR candidate RPs are configured.
  • The graceful restart timer is less than the default value, 30 seconds.

RIP

Use the default timers; graceful restart is not supported. For scaled configurations, such as for 2000 RIP interfaces, use the following values:

  • Flush interval: 600 seconds
  • Holddown time: 260 seconds
  • Invalid interval: 260 seconds
  • Update interval: 60 seconds

RSVP-TE

Use the default timers, including graceful restart default timers, except for the following:

  • For graceful restart, the hello timeout interval is the product of hello misses multiplied by the hello refresh interval. Determine which period is longer, the IC upgrade time or the forwarding upgrade time. Configure the hello refresh and hello miss values so that the hello timeout is greater than the longer of those two periods.
  • For node hellos, the product of the refresh misses multiplied by the hello refresh interval must be great than the FC outage time. For an outage time of less than 30 seconds, for example, configure the following values:
  • Set the node hello refresh interval to 8000.
  • Set the node hello refresh misses to 4.

[Contents] [Prev] [Next] [Index] [Report an Error] [No Frames]