Example: Configuring Link Protection with Host Fast Reroute
Understanding Host Fast Reroute
Host fast reroute (HFRR) adds a precomputed protection path into the Packet Forwarding Engine (PFE), such that if a link between a provider edge device and a server farm becomes unusable for forwarding, the PFE can use another path without having to wait for the router or the protocols to provide updated forwarding information. This precomputed protection path is often called a repair or a backup path.
HFRR is a technology that protects IP endpoints on multipoint interfaces, such as Ethernet. This technology is important in datacenters where fast service restoration for server endpoints is critical. After an interface or a link goes down, HFRR enables the local repair time to be approximately 50 milliseconds.
Consider the network topology shown in Figure 1.
Routing devices create host route forwarding entries triggered by the Address Resolution Protocol (ARP) and IPv6 Neighbor Discovery Protocol (NDP). HFRR augments the host routes with backup next hops supplied by routing protocols. These backup next hops enable arriving traffic to keep flowing while the network reconverges.
Traffic flows from networks connected to the provider edge devices, PE1 and PE2, to host A and host B. This traffic is protected with HFRR. If the link goes down between device PE2 and the host servers, traffic is rerouted through device PE1 to the host servers. In the topology, host A and host B represent LAN PCs, collectively known as a server farm. The PE devices are routers with a Layer 3 VPN configured between them. Device PE1 learns about the directly connected hosts by way of ARP or the IPv6 NDP.
Device PE2 also has information about the server farm network and advertises this information to Device PE1. This advertisement is transmitted through the Layer 3 VPN using internal BGP (IBGP). On Devices PE1 and PE2, this route is considered a direct route to the server farm subnet.
Device PE1 uses the host routes learned through ARP and NDP to send traffic to the host machines in the server farm. If the link between Device PE1 and the server farm is disrupted and if HFRR is not configured, the routing device finds the next best route, which is the IBGP route. This implementation results in traffic loss for an interval until the update occurs and the network reconverges. HFRR configured on Device PE1 resolves this issue by augmenting the ARP and NDP routes with a backup path so that traffic can continue to be forwarded without interruption.
The backup path in this particular topology is the IBGP Layer 3 VPN route. In an actual deployment, Device PE2 can also configure link protection for its directly connected server farm network, and Device PE1 can advertise reachability to the server farm through itself using the Layer 3 VPN routes to Device PE2. Therefore, HFRR should be enabled on both Device PE1 and Device PE2. Also, Device PE1 and Device PE2 should both advertise reachability to the server farm through BGP.
A temporary routing loop can develop between the PE devices if, for example, the link between Device PE1 to the server farm and the link between Device PE2 to the server farm both go down at same time. The loop can continue until BGP on both ends learns that the server farm subnet is down and withdraws the BGP routes.
ARP Prefix Limit and Blackout Supplementary Timeout
When you configure HFRR profiles, an optional ARP prefix limit sets a maximum for the number of ARP routes and, therefore, FRR routes created for each HFRR profile in the routing table. This limit prevents ARP attacks from exhausting the virtual memory on the routing devices. The ARP prefix limit does not limit ARP routes in the forwarding table. It does, however, limit the number of ARP routes that Junos OS reads for a profile and therefore limits the number of HFRR routes that the routing process (rpd) creates in the routing table and the forwarding table.
The ARP prefix limit is applied to each HFRR profile. It does not limit the total count of all ARP/HFRR routes in the routing table. It only limits the number of ARP/HFRR routes for each HFRR profile.
There are two configuration statements (global-arp-prefix-limit and arp-prefix-limit) that set the ARP prefix limit, one at the global [edit routing-options host-fast-reroute] hierarchy level and the other at the [edit routing-instances instance-name routing-options interface interface-name] hierarchy level, respectively. The global global-arp-prefix-limit statement sets a default ARP prefix limit for all HFRR profiles configured on the routing device. The arp-prefix-limit statement overrides the global-arp-prefix-limit for that HFRR profile for that protected interface.
When the number of ARP routes in an HFRR profile reaches 80% of the configured ARP prefix limit, a warning message is sent to the system log. The warning message is displayed for any subsequent ARP route added to that HFRR profile if the ARP prefixes remain at greater than 80% of the configured value.
When the number of ARP routes in an HFRR profile reaches 100% of the configured ARP prefix limit for an HFRR profile, another warning message is sent to the system log. When the number crosses the 100% threshold, the HFRR profile is deactivated. When this happens, all ARP/FRR routes are deleted from the routing table. FRR routes are deleted from forwarding table as well.
After the HFRR profile is deactivated, a blackout timer is started. The timeout value of this timer is the ARP cache timeout (kernel timeout) + the supplementary blackout timer.
There are global and per-HFRR CLI statements (global-supplementary-blackout-timer and supplementary-blackout-timer). The global value is at the [edit routing-options host-fast-reroute] hierarchy level and applies to all HFRR profiles on the routing device. The supplementary blackout timer configured for the routing-instance interface at the [edit routing-instances instance-name routing-options interface interface-name] hierarchy level overrides the global value for that HFRR profile only.
When the blackout timer expires, the HFRR profile is reactivated, and Junos OS relearns the ARP routes and re-creates the HFRR routes. If the ARP prefix limit is not exceeded again, the HFRR routes will be up.
If an HFRR profile is blacklisted and is in the deactivated state, a reevaluation of the ARP state is performed during every commit operation or whenever the routing process (rpd) is restarted with the restart routing command.
Primary Route and Backup Route Candidates
The primary route for the HFRR next hop is provided by the ARP and IPv6 NDP routes. These are /32 or /128 routes. The backup route is an exact prefix match of the address configured on the local interface. For example, if the local address configured is 10.0.0.5/24, the routing device looks for an exact match of prefix 10.0.0.0 with a prefix length of 24 for selection of backup route.
Constraints for backup route selection are as follows:
Must be a prefix matching the same subnet address configured on the routing device’s HFRR-enabled interface.
The remote end must not have route aggregation (also known as summarization) configured. For example, if the remote end combines two or more /24 subnets to advertise a subnet with a prefix length smaller than /24, the Junos OS does not select this summarized route to be a backup route.
If there is another route in the routing table learned by another protocol with a longest-prefix match for the /32 or /128 (ARP or NDP) route, that route is not selected to be a backup candidate. For example, suppose that the local interface address is 10.0.0.5/24. Also suppose that the routing table contains an IBGP route with a prefix of 10.0.0.0/24 and an OSPF route with a prefix of 10.0.0.0/28. Even though the /28 route is a better route for certain prefixes in the subnet, the Junos OS does not consider 10.0.0.0/28 to be a backup candidate. The IBGP route becomes the backup candidate for all host routes. However, after the global repair, the OSPF route is used for forwarding.
In short, the backup candidate must be a route with the same prefix as the subnet local interface that you are protecting with HFRR.
Backup Path Selection Policy
Only Layer 3 VPN routes are considered for backup selection. HFRR uses the usual BGP path selection algorithm to select one best backup route. Only one backup path is selected. In case there are multiple backup path candidates, the selection algorithm selects the best backup path. HFRR provides only two paths, one primary and one backup at any point in time. If the selected backup path itself has two paths in it, then the first path in that backup next hop is used as the backup next hop for the HFRR route.
The primary path is installed with a weight of one. The backup path is installed with a weight of 0x4000. The backup path obviously must be a path through an interface that is not the same as the primary interface.
The backup route is looked up only in the routing table to which interface belongs. For IPv4, the Junos OS uses routing-instance-name.inet.0. For IPv6, the Junos OS looks in routing-instance-name.inet6.0.
Characteristics of HFRR Routes
The HFRR route is a forwarding-only route and is not used for route resolution. HFRR routes have host addresses, meaning that they have /32 or /128 as the prefix length. In the case of platforms with dual Routing Engines, the backup routing process (rpd) also creates HFRR routes. However, the backup outing process (rpd) does not install HFRR routes to a routing table until the backup becomes the master after a Routing Engine switchover.
Also note that if an HFRR route is present in the routing table, the HFRR route is used for the unicast reverse-path-forwarding (uRPF) computation.
Removal of HFRR Routes
HFRR routes are deleted if the protected interface is deleted or deactivated in the configuration, if HFRR is configured on a routing instance and the routing instance is deactivated or deleted, or when the statement that enables HFRR (link-protection (Host Fast Reroute)) is deleted or deactivated. HFRR routes are deleted and readded when there is a catastrophic operation on routing the instance, such as when the routing process is restarted. HFRR routes are also be removed if all backup routes are deleted. such as when BGP withdraws routes or when BGP is deactivated or deleted.
After a protected interface goes down and if HFRR is deleted or deactivated, a timer starts with a timeout of 20 seconds. The HFRR route deletion occurs after the timer expires. This is to ensure that if the interface is flapping (quickly going up and down), the Junos OS does not unnecessarily perform route deletions and additions that cause traffic loss. This timer is used only when the interface is down or when the HFRR route is deleted or deactivated.
HFRR routes are purged immediately in the following cases:
A backup route goes down and there are no other potential backup paths.
An ARP delete message is received.
The routing process (rpd) terminates.
Interfaces That Support HFRR
HFRR is allowed only on Ethernet interfaces. The commit operation fails if you configure HFRR on point-to-point interfaces.
Only interfaces configured under routing instance of type VPN routing and forwarding (VRF) are accepted. The commit operation fails if you configure HFRR on other types of routing instances.
When the following requirements are not met, the commit operation does not fail. However, the interface is not protected by HFRR, and the interface is marked inactive in the show hfrr profiles command output:
HFRR is allowed only on numbered interfaces, meaning that an address must be assigned to the interface. You cannot, for example, configure IPv4on the interface with an address and IPv6 without an address.
Interfaces that are configured for HFRR protection must be configured at the [edit interfaces] hierarchy level and also must be attached to the routing instance.
The routing instance must have a virtual tunnel (VT) interface or the vrf-table-label statement included.
Another reason the interface might be marked inactive in the show hfrr profiles command output is when the interface is migrating from one instance to another, and the HFRR configuration is in the previous routing instance.
HFRR is not supported on overlapping logical units if they belong to the same routing instance, as shown here:
If you configure overlapping subnets as shown here, and if you enable HFRR on both of the overlapping subnets, the routing protocol process (rpd) generates an RPD_ASSERT error.
Example: Configuring Link Protection with Host Fast Reroute
This example shows you how to configure host fast reroute (HFRR). HFRR protects IP endpoints on multipoint interfaces, such as Ethernet.
This example uses the following hardware and software components:
Two provider edge (PE) devices and four provider (P) devices.
The example assumes that hosts are present, behind the PE devices.
The example assumes that at least one Layer 3 switch, such as an EX Series switch, is attached to the hosts.
Junos OS 11.4R2 or later.
In this example, traffic flows to server hosts from networks connected to PE devices. This traffic is protected with HFRR. If the link goes down between one PE device and the server farm, traffic is rerouted to the server farm through the other PE device.
You can configure HFRR by adding the link-protection statement to the interface configuration in the routing instance.
We recommend that you include this statement on all PE devices that are connected to server farms through multipoint interfaces.
In this example, the PE devices advertise reachability to their server farms through Layer 3 VPN routes and BGP.
As optional settings, the PE devices have the high availability features Nonstop Active Routing and Virtual Router Redundancy Protocol (VRRP) configured. Nonstop active routing (NSR) enables a routing platform with redundant Routing Engines to switch from a primary Routing Engine to a backup Routing Engine without alerting peer nodes that a change has occurred and without losing routing and protocol information. VRRP provides for automatic assignment of available routers to participating hosts, thus increasing the availability and reliability of routing paths.
Figure 2 shows the topology used in this example.
This example shows the configuration on all of the routing devices and shows the step-by-step procedure on Device PE1.
CLI Quick Configuration
To quickly configure this example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, and then copy and paste the commands into the CLI at the  hierarchy level.
The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode in the CLI User Guide.
To configure HFRR:
- Configure the interfaces.[edit interfaces]user@PE1# set ge-4/1/0 unit 0 family inet address 192.0.2.2/24user@PE1# set ge-4/1/0 unit 0 description toPE2user@PE1# set ge-0/2/0 unit 0 family inet address 10.10.10.1/30user@PE1# set ge-0/2/0 unit 0 description toP1user@PE1# set ge-0/2/0 unit 0 family mplsuser@PE1# set ge-0/2/4 unit 0 family inet address 10.10.15.2/30user@PE1# set ge-0/2/4 unit 0 description toP5user@PE1# set ge-0/2/4 unit 0 family mplsuser@PE1# set lo0 unit 0 family inet address 10.255.8.207/32
- (Optional) Configure VRRP on the interface to Device PE2.
- Configure MPLS on the interfaces.[edit protocols mpls]user@PE1# set interface ge-0/2/0.0user@PE1# set interface ge-0/2/4.0
- Configure BGP.[edit protocols bgp group pe-ce]user@PE1# set type internaluser@PE1# set local-address 10.255.8.207user@PE1# set family inet-vpn unicastuser@PE1# set neighbor 10.255.8.86user@PE1# set export send-routes
- Configure a policy that advertises direct and local interface
routes.[edit policy-options policy-statement send-routes term 1]user@PE1# set from protocol directuser@PE1# set from protocol localuser@PE1# set then accept
- Configure an interior gateway protocol, such as IS-IS
or OSPF.[edit protocols ospf area 0.0.0.0]user@PE1# set interface ge-0/2/0.0user@PE1# set interface ge-0/2/4.0user@PE1# set interface lo0.0 passive
- Configure a signaling protocol, such as RSVP or LDP.[edit protocols ldp]user@PE1# set interface ge-0/2/0.0user@PE1# set interface ge-0/2/4.0
- (Optional) Configure nonstop active routing.[edit routing-options]user@PE1# set nonstop-routing
- Configure the autonomous system (AS).[edit routing-options]user@PE1# set routing-options autonomous-system 100
- Configure the Layer 3 VPN routing instance.[edit routing-instances cust1]user@PE1# set instance-type vrfuser@PE1# set interface ge-4/1/0.0user@PE1# set route-distinguisher 100:100user@PE1# set vrf-target target:100:100user@PE1# set vrf-table-label
- Configure HFRR link protection.[edit routing-instances cust1 routing-options]user@PE1# set interface ge-4/1/0.0 link-protection (Host Fast Reroute)
If you are done configuring the device, commit the configuration.user@PE1# commit
Confirm your configuration by issuing the show interfaces, show protocols, show policy-options, show routing-options, and show routing-instances commands.
Confirm that the configuration is working properly.
Make sure that HFRR is enabled.
user@PE1> show hfrr profiles
HFRR pointer: 0x9250000 HFRR Current State: HFRR_ACTIVE HFRR Protected IFL Name: ge-4/1/0.0 HFRR Protected IFL Handle: 0x921086c HFRR Routing Instance Name: cust1 HFRR Routing Instance Handle: 0x9129740 HFRR Sync BG Sceduled: NO HFRR RTS Filter On: YES HFRR Delete BG Scheduled: NO HFRR Num ARP Routes learnt: 100 HFRR Num FRR Routes Created: 100
The output shows that the HFRR is enabled on interface ge-4/1/0.0.
Verifying ARP Routes
Make sure that the expected ARP routes are learned.
user@PE1> show route protocol arp
inet.0: 43 destinations, 43 routes (42 active, 0 holddown, 1 hidden) inet.3: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden) cust1.inet.0: 1033 destinations, 2043 routes (1033 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 192.0.2.3/24 @[ARP/4294967293] 00:04:35, from 192.0.2.1 Unusable 192.0.2.4/24 @[ARP/4294967293] 00:04:35, from 192.0.2.1 Unusable 192.0.2.5/24 @[ARP/4294967293] 00:04:32, from 192.0.2.1 Unusable 192.0.2.6/24 @[ARP/4294967293] 00:04:34, from 192.0.2.1 Unusable 192.0.2.7/24 @[ARP/4294967293] 00:04:35, from 192.0.2.1 Unusable 192.0.2.8/24 @[ARP/4294967293] 00:04:35, from 192.0.2.1 Unusable 192.0.2.9/24 @[ARP/4294967293] 00:04:35, from 192.0.2.1 Unusable 192.0.2.10/24 @[ARP/4294967293] 00:04:35, from 192.0.2.1 Unusable 192.0.2.11/24 @[ARP/4294967293] 00:04:33, from 192.0.2.1 Unusable 192.0.2.12/24 @[ARP/4294967293] 00:04:33, from 192.0.2.1 Unusable 192.0.2.13/24 @[ARP/4294967293] 00:04:33, from 192.0.2.1 Unusable ...
Verifying Fast Reroute Routes
Make sure that the expected fast reroute (FRR) routes are learned.
user@PE1> show route protocol frr
inet.0: 43 destinations, 43 routes (42 active, 0 holddown, 1 hidden) inet.3: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden) cust1.inet.0: 1033 destinations, 2043 routes (1033 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 192.0.2.3/24 #[FRR/200] 00:05:38, from 192.0.2.1 > to 192.0.2.3 via ge-4/1/0.0 to 10.10.15.1 via ge-0/2/4.0, Push 16, Push 299792(top) 192.0.2.4/24 #[FRR/200] 00:05:38, from 192.0.2.1 > to 192.0.2.4 via ge-4/1/0.0 to 10.10.15.1 via ge-0/2/4.0, Push 16, Push 299792(top) 192.0.2.5/24 #[FRR/200] 00:05:35, from 192.0.2.1 > to 192.0.2.5 via ge-4/1/0.0 to 10.10.15.1 via ge-0/2/4.0, Push 16, Push 299792(top) 192.0.2.6/24 #[FRR/200] 00:05:37, from 192.0.2.1 > to 192.0.2.6 via ge-4/1/0.0 to 10.10.15.1 via ge-0/2/4.0, Push 16, Push 299792(top) 192.0.2.7/24 #[FRR/200] 00:05:38, from 192.0.2.1 > to 192.0.2.7 via ge-4/1/0.0 to 10.10.15.1 via ge-0/2/4.0, Push 16, Push 299792(top) 192.0.2.8/24 #[FRR/200] 00:05:38, from 192.0.2.1 > to 192.0.2.8 via ge-4/1/0.0 to 10.10.15.1 via ge-0/2/4.0, Push 16, Push 299792(top) 192.0.2.9/24 #[FRR/200] 00:05:38, from 192.0.2.1 > to 192.0.2.9 via ge-4/1/0.0 to 10.10.15.1 via ge-0/2/4.0, Push 16, Push 299792(top) 192.0.2.10/24 #[FRR/200] 00:05:38, from 192.0.2.1 ...
Make sure that the expected routes appear in the forwarding table.
user@PE1> show route forwarding-table destination 192.0.2.3
Routing table: default.inet Internet: Destination Type RtRef Next hop Type Index NhRef Netif default perm 0 rjct 36 1 Routing table: default-switch.inet Internet: Destination Type RtRef Next hop Type Index NhRef Netif default perm 0 rjct 554 1 Routing table: __master.anon__.inet Internet: Destination Type RtRef Next hop Type Index NhRef Netif default perm 0 rjct 532 1 Routing table: cust1.inet Internet: Destination Type RtRef Next hop Type Index NhRef Netif 192.0.2.3/24 user 0 ulst 1048575 2 0:0:14:14:1:3 ucst 767 3 ge-4/1/0.0 indr 1048574 1001 10.10.15.1 Push 16, Push 299792(top) 1262 2 ge-0/2/4.0 192.0.2.3/24 dest 0 0:0:14:14:1:3 ucst 767 3 ge-4/1/0.0 ...