There are several different high availability models possible with the 128T routing software. This document covers dual router high availability, which is when two instances of the 128T software are each configured as separate routers (rather than as two nodes within a single router). This is characterized by:
- An iBGP interface shared between the two devices in lieu of a "fabric" interface in the dual node deployment.
shared-phys-address(and hence no shared interfaces) between the two devices. Interface protection in a dual router HA deployment is accomplished using traditional routing protocols (Layer 3) rather than IP/MAC takeover (Layer 2).
- No state synchronization between the two devices (and hence no "HA link"). While this improves overall performance for the routers (since there is no overhead incurred due to state synchronization), the implication is that there are some capabilities not supported in this design. See Unsupported Features, below.
Dual router high availability is recommended for data center designs where there will be a large volume of traffic; the elimination of shared state synchronization yields a simpler design, benefitting critical infrastructure.
When deploying two nodes in a dual router high availability deployment, several features that rely on synchronized state between nodes are no longer available.
- Source NAT. When source NAT is enabled, a system will allocate ephemeral ports on its egress interface as sessions leave the 128T router. Because there is no state synchronization between the two nodes in this deployment model, these ephemeral ports are not shared. Thus, if the active node fails and traffic starts transiting its counterpart, the source NAT allocation on the newly active system will be different. This will impact application flows.
- Shared interfaces. A router node cannot perform gratuitous ARP takeover of an interface from another, distinct router node.
Due to the way that dual router high availability operates without state synchronization, in the event of a failure to one router in a pair, all traffic will be sent to the counterpart router (once routing converges). The now-active router does not have a shared database to reconstruct session state; it will receive mid-session packets (from the client's and server's perspectives) and need to construct its own state from these packets.
For this reason, services that leverage a dual router HA pair must reference a
service-policy that has
transport-state-enforcement allow configured. Otherwise, mid-session TCP packets cause the 128T device to send a TCP RST to the sender.
Unlike a "traditional" dual node high availability design, in which two nodes compose a single router and use an out-of-band interface (the "sync interface") to synchronize state and a fabric interface to forward packets from one node to the other, the dual router trades these in for an inter-router iBGP connection for forwarding packets.
It is possible to configure multiple inter-router connections for added resiliency if there are spare physical connections available between the two routers.
The sample high level topology we will discuss in this document is as follows:
Notes about the Topology
In this sample exercise, each of the two routers (
routerB) have two WAN interfaces and one LAN interface. Between them is an interconnect cable; for collocated routers, this can be a simple "crossover" cable between the two systems.
Unlike the dual node redundancy model, where the two devices collectively harbor a single instance of the
routingManager process, routing within the dual router redundancy model must be accomplished "by hand;" i.e., discretely on each individual system. This consists of two components:
- Each router uses BGPoSVR to exchange routes with the other.
- For services that use
peer-type service-routes to reach another 128T instance, these service-routes will need to include the complementary router as an additional
next-peer. I.e., each router in the HA pair will point to the
next-peer128T as well as a
next-peerfor one another.
Each of the two routers that comprise the highly available pair will be configured similarly. Here is our
Unlike the dual node high availability design, the dual router high availability design does not synchronize state between routers. Instead, the two devices exchange reachability information using iBGP; this is implemented on the 128T using BGP over SVR (BGPoSVR), as seen in the sample configuration.
In our sample configuration we use
device-interface eno1 as our iBGP link. The sample here uses link-local IP addresses, presuming that the two nodes are situated next to one another in the same data center. The
neighborhood dc1-interrouter configuration, also provisioned on
routerB, indicates to conductor that the two devices are mutually reachable. This hint (combined with the loopback interfaces) is what creates the peering relationship, the services, and the service-routes in support of BGPoSVR.
This iBGP will also interact with other routing protocols to exchange reachability information with one another. For example, you may
redistribute ospf into this iBGP, so that each device is aware of the other's reachability, as shown here:
Service Policy Configuration
As mentioned in the design constraints section, due to the fact that there is no shared state between the routers in a dual router HA deployment, when a router fails its counterpart will pick up all traffic for active sessions as soon as routing converges. This will result in mid-flow TCP packets arriving at the new router, and the default behavior for the 128T router is to reject all mid-flow TCP packets by sending a TCP RST back to the sender.
This behavior is governed by the
transport-state-enforcement field in the
service-policy. Any service traffic that is conveyed to or from a system that is configured as a dual router HA deployment must have
transport-state-enforcement set to
allow, or else all TCP-based traffic will need to be restarted post-switchover. Below is an example
service-policy showing the
transport-state-enforcement set to
For more information on this setting, refer to the section on
transport-state-enforcement in our online documentation.
Service Route Configuration
There are two considerations when constructing
service-route configuration in a dual router HA environment:
- Routing traffic from a router to a remote set of routers configured for dual router HA
- Routing traffic from the dual router HA to other peers
In both of these cases, the
service-route will use the
Routing to a Dual Router HA
Below is an example
service-route that would typify a branch router's path to a data center dual router HA pair:
In this example, the branch router's traffic matching
service-1 will have two possible peers it can leverage,
While it is possible to manually configure these yourself, the conductor will typically autogenerate these types of
service-route elements on your behalf.
Routing from a Dual Router HA
Below is an example
service-route that typifies a dual router HA system sending traffic to a branch site (using a summary service):
This example would be found on
routerA; note that while we still use
next-peer, the hops are different. First is
branch1, the target of our summary service. Second, however, is
routerB. This is effectively a manually-created dual router HA version of a dual node HA's "fabric" path:
routerA has a path to reach
There will also be a corresponding
routerB that looks slightly different:
Here we see that
routerB will "hop through"
routerA to get to
branch1, if its direct path to
branch1 is down. (I.e., all peer paths are unavailable.)
service-route configuration elements will not be built for you by conductor, and must be manually created. (They are typically part of provisioning templates, such that as new routers are deployed the configuration templates will include these
service-route elements for the new peers.)