Connectivity

Reachability

Essential connectivity in SR is achieved by advertising prefix and adjacency segments. These are denoted by segment identifiers (SID). Many other types of SIDs exist, each serving a different purpose, but these two suffice to establish basic reachability. The prefix, and its specialized node SID form, uniquely identifies a router in a given topology, reachable through a particular path-finding algorithm; the adjacency SID identifies a routed link to an IGP neighbor.

Some segments are directly advertised as labels, as in the case of adjacency SIDs; prefix and node SIDs are indirectly advertised as indexes into an SR Global Block (SRGB). The SRGB represents the label space that nodes use to refer to global segments. It is one reason SR does not need to advertise a label for each IGP-learned prefix. The SRGB is configurable, and ideally consistent across the SR domain.

There is at least one segment associated with an IPv4 prefix and possibly another for IPv6. Node segments are associated with the router’s loopback address. Indeed, the IPv4 and IPv6 node indexes are expected to be treated just as loopback address are – managed by the operator and unique per-node in an SR domain.

Adjacency segments default to being dynamically assigned labels with a pop, and forwarded via the corresponding IGP adjacency action. These are used for strict steering, including traffic protection. Unlike node SIDs, adjacency SIDs have the option to be locally or globally significant (albeit, globally significant adjacency SIDs are uncommon).

Brownfields

Our example network is depicted in Figure 1. This is a common design for many carriers. Both Internet and VPN overlay service are delivered by the combination of LDP/RSVP-TE as label distribution protocols, and multi-area IS-IS as the IGP of choice.

Figure 1: Network Topology

Since our focus is the underlay, let Internet and L3VPN connectivity suffice as proof of our work. Of course, any MPLS service – 6PE, 6VPE, L2VPN, EVPN, et. al., – could be delivered over this same network.

To orient ourselves, let’s run a traceroute from the CE in New York (ce1.nyc) to Washington (ce1.iad).

Connectivity verification

Note that the labels in use will change once SR starts to be used.

For now, let’s enable SR on pe1.nyc and carefully trace the changes to the control and forwarding planes.

Enabling SR on pe1.nyc

The SRGB is configured to range from 1000 to 1127, inclusive. Within that range, this router represents its IPv4 loopback address as 1001 (1000 + 1, its IPv4-index). Let’s confirm this to be the case.

Control plane: node SID verification

Apart from the manually-configured node SID, let’s verify that adjacency SIDs have also been dynamically allocated.

Control plane: adjacency SID verification

There is one adjacency SID to each of pe1.nyc’s IS-IS neighbors. Additionally, the node SID is highlighted as a sub-TLV of the IP-extended prefix TLV. The flags also clarify and contrast between the two SID types. ‘V’ indicates whether the subTLV carries a value (as in the case of an adjacency SID) or an index (as with a node SID). ‘L’ represents local vs. global significance. Finally, the ‘N’ flag states that a given prefix SID is more specifically a node SID as it is attached to the router’s loopback address.

Forwarding plane: PE1’s FIB

L-ISIS stands for labeled IS-IS. The adjacency SIDs are installed in the FIB but the node SID seems to be missing. This is normal. Since the ‘P’ flag (PHP-off) was not set, this router does not expect to receive packets with the top label 1001. Instead, 1001 would be popped by its neighbors (once they are enabled for SR). This is analogous to implicit-null behavior used by traditional label distribution protocols.

Inter-area

Since none of the other routers are currently enabled for SR, pe1.nyc’s SIDs aren’t yet learned by area-external routers. Intra-area routers, such as pe2.nyc, learn but ignore them.

Nothing has changed, as one would expect. Labeled IS-IS has a lower protocol preference than LDP or RSVP-TE, and requires operator intervention to start being used. No SR forwarding is taking place as our SR domain is an island of one, as shown in Figure 2. In the next section, we’ll configure node SIDs on all routers in New York (which also share a common IS-IS area), starting with the L1-L2 attached p1.nyc.

Figure 2: Connectivity Verification

Enabling SR on p1.nyc

Control plane

Let’s verify what has changed more quickly and briefly this time. Abridged output will be shown to highlight points of interest indicated by ellipses (…).

Two differences are apparent compared to pe1.nyc. As an area border router (ABR), p1.nyc not only advertises its own node SID into both the L1 and L2 areas, it also re-originates pe1.nyc’s node SID advertisement across areas. This allows global segments to be learned across IGP areas. The ‘R’ flag indicates this node SID is being re-advertised across areas/levels.

Crucially, the ‘P’ flag is set on this re-advertised node SID, in stark contrast to p1.nyc’s own node SID. This signals to SR-capable neighbors that p1.nyc expects them to use the continue (swap) action and retain the top label which it interprets as a forwarding instruction to pe1.nyc.

Forwarding plane: P1’s FIB

We can confirm that p1.nyc has installed an entry for label 1001, which represents pe1.nyc’s node SID:

Not only can you see a pop (the MPLS forwarding plane’s version of continue) action for incoming label 1001, you can see pop actions for adjacency SID labels 28-33.

Forwarding plane: PE1’s FIB

On pe1.nyc there is now a forwarding entry for label 1003, representing p1.nyc’s node SID.

Inter-area

Other than pe1.nyc, none of p1.nyc’s other neighbors are SR-enabled. All L2 neighbors learn about, but ignore, the segments.

Connectivity verification

As unexciting as the lack of change in forwarding behavior may be, this is reassuring. Enabling SR has not inadvertently changed, let alone harmed, existing services. After all, a controlled deployment is preferred.

Enabling SR on the remaining New York routers

Control plane

At this point you should know what to expect. Two new node SIDs would be advertised into the L1 area; pe2.nyc’s node SID would be re-advertised by both p1.nyc and p2.nyc into the L2 subdomain. Like pe1. nyc’s before it, it would be ignored by the non-SR enabled L2 routers for now. Let’s see:

This should now be familiar, there are adjacency SIDs for each IS-IS neighbor (SR-capable or not), as well as node SIDs (re-advertised in all cases but one) for all SR-capable routers.

Connectivity verification

Finally you can confirm no change to traffic forwarding.

All this work to no end? Hardly. It has been proven that enabling the SR control plane does not force an operator to immediately utilize its forwarding plane. Indeed, this is likely to be the first step in any migration to SR – control plane verification coupled with continued service assurance.

Switching to SR

SR is now enabled throughout New York. All it will take to make the adjustment is a protocol preference on an ingress router. So far we have been testing forwarding between CEs in New York and Washington. Since our SR domain is constrained to the former, let’s quickly verify connectivity between CEs within the region.

Connectivity verification

None of the SR labels are in use. This makes sense since pe2.nyc, the ingress router, continues to prefer LDP-learned next hops when doing BGP service route resolution.

Control plane

Let’s verify the service route and its next hop on ingress PE:

The BGP route we’re tracing is learned with a protocol next hop of 128.49.106.1 – this is pe1.nyc’s loop-back address. A label is associated with this forwarding equivalence class (FEC) from both LDP and SR (L-ISIS stands for labeled IS-IS) neighbors. The former has a superior protocol preference, leading to the pe2.nyc imposing the LDP-learned label 257 from its chosen neighbor (p2.nyc).

To make the change, let’s improve the protocol preference for L-ISIS on pe2.nyc.

Prefer SR for intra-region traffic originating at pe2.nyc

Control plane: confirm pe2.nyc performs route resolution using SR instead of LDP

The protocol next hop now prefers the SR divined label (1001, pe1.nyc’s node SID). This should have (finally!) affected the forwarding plane.

Connectivity verification: intra-NY traffic is SR-switched

Note the change in the label pushed by pe2.nyc from 257 to 1001. This confirms traffic in the SR domain is now exercising the LSP woven by segments. Just to make sure all extra-regional traffic remains unaffected by the protocol preference change on pe2.nyc, a trace to the cross-country destination in Washington, remains traditionally label-switched.

Connectivity verification: extra-NY traffic still uses LDP

Now that we are convinced there is no discontinuity of service as a result of our actions, let’s mirror pe2. nyc’s SR preference on all New York routers. The configuration is identical:

At this point, the New York routers prefer SR to LDP for next-hop resolution to other New York routers. Routers outside the region remain without SR and traffic continues to be forwarded without disruption.

Class of Service Preservation

Now that our island is functional, let’s take a brief look at how SR supports networks that offer strict SLAs around uptime and class of service (CoS). A widely deployed technique is to disable penultimate hop popping (PHP) in favor of ultimate hop pop-like (UHP) behavior. Since SR does not directly advertise node SIDs as labels, it uses flags to request UHP behavior from upstream routers. Let’s enable UHP on pe1.nyc and review the change to its neighbors’ FIB.

Enable explicit-null UHP behavior on egress router pe1.nyc

Control plane: verify the change in pe1.nyc’s IS-IS LSP

In contrast with earlier output, pe1.nyc now advertises its node SID with both the ‘P’ (PHP-Off, or disable PHP) and ‘E’ (enable explicit-null, or use reserved label 0) flags set.

This leads its neighbors, p1.nyc and p2.nyc, to use the continue (swap) instead of the next (pop) action for label 1001 (pe1.nyc’s node index is 1, and the network uses a common SRGB of 1K as discussed earlier). There is no change to pe2.nyc’s FIB. It is not directly connected to pe1.nyc. PHP and UHP behaviors are only relevant for the immediately upstream routers.

Forwarding plane: verify pe1.nyc neighbors’ FIB

Both p1 and p2 are now swapping label 1001 with label 0, which is the well-known IPv4 explicit null label. This label ensures the MPLS traffic class (previously also known as experimental bits) is consistently copied to the last hop router. At that last hop, label 0 is popped and an IPv4 lookup occurs. The existence of label 0 has enabled CoS transparency and preservation.

Before proceeding, note that UHP is also enabled on pe2.nyc.

Traffic Protection and Restoration

Packet networks have long been able to deliver protection within 50ms of failure, crossing the bar set by SONET/SDH networks. IP-based fast reroute (FRR) is available for LDP as loop-free alternate (LFA) and remote LFA (rLFA). In turn, RSVP-TE has rich mechanisms in place to protect against loss of link, node, or fate-sharing resources identified as shared-risk link groups (SRLG).

Neither approach is without its caveats. LFA is topology-dependent in its ability to provide protection; rLFA improves on this at the expense of added targeted LDP session state and complexity. RSVP-TE has the most thorough feature set, which depends on additional signaling and state maintenance of bypass and detour LSPs.

SR’s ability to source route, explicitly steering, and lack of transit control plane state lends itself well to establishing costless local protection paths. As long as there is physical redundancy in a topology, Topology-Independent Loop-free Alternate (TI-LFA) will find alternate routes. It can reroute around protected links, nodes, or locally-defined fate-sharing groups. Another improvement over erstwhile protection technologies is that TI-LFA protection paths are also ECMP-capable.

TI-LFA protection is local to the point of local repair (PLR). The backup paths are pre-computed but not pre-signaled. There is no singular merge point (MP) as each release point can be per-prefix optimized. The pre-computed paths prune or heavily cost out the protected resource from the topology in order to calculate post-IGP-convergence path(s). A failure of the protected resource results in a single update of the data plane, combining both local and global repair into one step.

Let’s enable link protection on p1.nyc and observe the resulting change on its SR neighbors.

Enable TI-LFA link protection for all level 1 links on p1.nyc

Control plane: verify p1.nyc is announcing its adjacency SIDs as protection-eligible

The ‘B’ flag signals that p1.nyc considers these adjacency SIDs candidates for protection. Let’s look at one of these in detail – SID 47, which represents the link to pe2.nyc.

Forwarding plane: verify p1.nyc has installed backup paths for protection-eligible SIDs

There are two unequal next hops, the latter newly added as a backup. Note that p1.nyc’s computed backup to reach pe2.nyc, avoiding the protected ge-0/0/3 link, is via p2.nyc. The primary action for incoming label 47 would be to pop and send directly to pe2.nyc. The backup action is to swap it for pe2.nyc’s node SID (label 1000) and send to p2.nyc, as shown in Figure 3. At that router, the label is swapped for 0, as a result of pe2.nyc advertising its reachability via explicit-null.

Figure 3: TI-LFA Avoiding Failed Link Between p1.nyc and pe2.nyc

Forwarding plane: verify p2.nyc acts as a transit router on this stateless protection path

This concludes the first leg: SR is enabled and in active use, albeit in a contained part of the network. There is reachability, and the ability to preserve CoS markings, as well as offer stateless traffic protection and restoration.

The next step is to broach SR into the rest of the network.