Rule 5: RSTP Deployment Rules for TCX1000 Devices
When deploying TCX1000 devices, there are rules that you must consider regarding rapid spanning tree (RSTP), which is used resolve loops in your DCN HA management network RSTP domain. This section describes these rules in more detail with some examples, as well as describes the concepts of RSTP.
Before We Begin, What is RSTP?
Before for look at the deployment rules for Rapid Spanning Tree Protocol (RSTP), we need to briefly look at what RSTP is all about. This section looks at RSTP concepts like tree, hops and RSTP maximum diameter.
The requirement for redundant management connections to each TCX1000 device in your DCN HA management network creates loops in the network and makes the network susceptible to broadcast storms. However, the DCN HA network needs to include loops because they provide redundant paths for management communications in case of a link failure in the network. RSTP addresses both of these issues because it provides link redundancy while simultaneously preventing undesirable loops.
RSTP resolves loops in a Layer 2 network by controlling the port states on the device and enabling or disabling the port from forwarding traffic. RSTP ensures that each Layer 2 device in the uses only a single path at a time for management communications.
Layer 2 devices include switches, the TCX1000-RDM20 and TCX1000-ILA.
TCX1000 devices support RSTP IEEE 802.1w.
On the TCX1000-RDM20, the following ports participate in RSTP:
On the TCX1000-ILA, the following ports participate in RSTP:
RSTP enables or disables the ports on Layer 2 devices in the network and creates a tree to each device in the network over which management communications occur. If a break occurs in the network, RSTP automatically enables or disables ports on the Layer 2 devices in the tree and builds a new tree to reestablish management communications.
The tree includes a root bridge, which is at the top of the tree. Every other Layer 2 device in the tree knows the path the root bridge and uses it.
Going from one Layer 2 device in the network to another represents one hop.
The RSTP diameter is the number of Layer 2 devices from root bridge to the furthest Layer 2 device in the network. So, to determine the RSTP diameter of the network, you count the number of hops, then + 1. RSTP has limitations on maximum diameter, which is dictated by the Root Bridge Max-Age parameter.
This guide does not go into detail on RSTP. Instead it focuses on RSTP in the context of connecting TCX1000 devices to your DCN HA. For more details on the concepts of RSTP you can reference the following documents, which are for Juniper Networks switches:
This document provides a good general description of RSTP concepts: How Spanning Tree Protocols Work
This document provides an example of RTSP on Juniper Networks EX Series Layer 2 Switches:Example: Faster Convergence and Improved Network Stability with RSTP on EX Series Switches
RSTP Network Deployment Rules
Only TCX1000-RDM20 devices running software release 3.1 or higher support IEEE 802.1w RTSP protocol. TCX1000-RDM20 devices running software releases lower than release 3.1 do not support RSTP and do not support management communications over their Line port.
When deploying TCX1000 devices in your optical network, you must adhere to the following rules for your DCN HA RSTP domain:
RSTP Rule 1: RSTP maximum diameter of 20 or less — This means that any Layer 2 device (TCX1000 or switch) in the DCN HA can be no more than 19 hops away from the proNX Optical Director because:
RSTP diameter = number of hops to device +1.
It is important to remember that if you exceed the RSTP maximum diameter, RSTP could resolve a loop that would result in loss of communication with a TCX1000 device after only a single fault. This is why you must be sure to count hops and check the maximum diameter when designing your DCN HA. For an example, see Example: Single Network Failure Isolates Device in Network.
RSTP Rule 2: RSTP maximum domain size of 100 Layer 2 devices (TCX1000 or switch) or less in a single RSTP domain — If this rule cannot be respected, you must break up your DCN HA RSTP domain by segregating L2/L3 RSTP domains and ensuring OSC forwarding is off between the RSTP domains, which is the default on the TCX1000-RDM20. This is likely only an issue for remote management communications in large DCN HA where the RSTP maximum diameter rule is exceeded.
RSTP Rule 3: We recommend that you provision your RSTP network so that the one of your DCN Layer 2 switches is the RTSP root bridge of the network. The typical RSTP Bridge Priority is 32768 or below on DCN switches.
Counting RSTP Hops and Diameter
Figure 1 shows an example of two DCN HA Ethernet networks.
The TCX1000 network on the left is homed to the DCN by a single Layer 2 switch at each end the network. The TCX1000 devices at each end of the network have a single connection to the Layer 2 switch.
The TCX1000 network on the right is homed to the DCN by the same single Layer 2 at each end the network. However the TCX1000 devices at each end of the network have redundant connections to the DCN.
Let us take a look at how this one change impacts the hop count
and the RSTP maximum diameter. But first, RSTP Rule 1 says RSTP maximum diameter of 20 or less. — This means that
any Layer 2 device in the DCN HA can be no more than 19 hops away
from the DCN HA and hence the proNX Optical Director because:
RSTP diameter = number of hops to device +1.
Looking at the network on the left of Figure 1 we can see that there is a total of 6 hops from the first Layer 2 switch in the DCN HA to TCX1000-RDM20-b, making the RSTP maximum diameter: 6 +1=7.
Hop 1-from the first switch to the second
Hop 2-from the second switch to TCX1000-RDM20-a
Hop 3-from TCX1000-RDM20-a to TCX1000-ILA-a
Hop 4-from TCX1000-ILA-a to TCX1000-ILA-b
Hop 5-from TCX1000-ILA-b to TCX1000-RDM20-b
Hop 6-from TCX1000-RDM20-b to the third switch on the right
The second network example on the right of Figure 1 shows the same network as on the left. However, in this network example, we have redundant paths for management communications by connecting both DCN ports on TCX1000-RDM20-c and TCX1000-RDM20-d to the DCN HA. This effectively decreases the number of hops required for management communications to 4 hops. The RSTP maximum diameter is decreased to: 4 hops +1=5. The redundant paths from the DCN to TCX1000-RDM20-c and TCX1000-RDM20-d enable management communications to go over either path in only 4 hops from either TCX1000-RDM20-c or TCX1000-RDM20-d:
Hop 1-from the first switch the second switch
Hop 2-from the second switch to TCX1000-RDM20-c
Hop 3-from TCX1000-RDM20-c to TCX1000-ILA-c
Hop 4-from TCX1000-ILA-c to TCX1000-ILA-d
Hop 1-from the first switch to the third switch (on the right in the DCN)
Hop 2-from the third switch to TCX1000-RDM20-d
Hop 3-from TCX1000-RDM20-d to TCX1000-ILA-d
Hop 4-from TCX1000-ILA-d to TCX1000-ILA-c
Example: Single Network Failure Isolates Device in Network
Figure 2 shows an example network where we exceed the RSTP maximum diameter and a single link failure isolates a TCX1000 devices. This example network shows a 3 TCX1000-RDM20 x 4 TCX1000-ILA deployment, which is a linear multi-span network that consists of three point-to-point TCX1000-RDM20 networks that each have 4 TCX1000-ILAs in their amplifier chains. This example illustrates a worse case scenario, where there is a single failure at the worst location (x) between two switches in your DCN. This single failure, causes the switch between the DCN HA and TCX1000-RDM20-f to become unmanageable because the RSTP maximum diameter of the network exceeds 20. Due to the break at (x) it would take 20 hops to reach the switch, which exceeds RSTP guidelines.