Data Center Interconnect Design and Implementation Using Type 5 Routes

The backbone devices in this architecture are part of the WAN cloud and must provide connectivity both to the spine devices in each data center as well as to the other backbone device. This connectivity must be established before EVPN Type 5 routes can be exchanged between spine devices in different data centers.

Figure 2 provides an overview of the IP addresses that are configured in these steps.

Figure 2: IP Address Summary for Backbone and Spine Devices

To configure the spine device and backbone device interfaces:

EBGP is used as the routing protocol of the underlay network in this reference design. The backbone devices must participate in EBGP with the spine devices to support underlay connectivity.

The process for enabling EBGP on the spine and leaf devices is covered in the IP Fabric Underlay Network Design and Implementation section of this guide. This procedure assumes EBGP has already been enabled on the spine and leaf devices, although some EBGP configuration on the spine devices needs to be updated to support backbone devices and is therefore included in these steps.

EBGP works in this reference design by assigning each leaf, spine, and backbone device into it’s own unique 32-bit autonomous system (AS) number.

Figure 3 shows an overview of the EBGP topology for the spine and backbone devices when backbone devices are included in the reference design.

Figure 3: EBGP Topology with Backbone Devices

Figure 4 illustrates the EBGP protocol parameters that are configured in this procedure. Repeat this process for the other devices in the topology to enable EBGP on the remaining devices.

Figure 4: EBGP Configuration in a Backbone Topology

To enable EBGP to support the underlay network in this reference design:

The backbone devices must run IBGP to have overlay network connectivity and be able to support DCI using EVPN Type 5 routes.

Figure 5 shows the IBGP configuration of the validated reference design when backbone devices are included in the topology. In the validated reference design, all spine and leaf devices in the same data center are assigned into the same autonomous system. The backbone devices are assigned into the same autonomous system as the spine and leaf devices of the data center that is using the backbone device as the entry point into the WAN cloud.

Figure 5: IBGP Overview with Backbone Devices

Figure 6 illustrates the route reflector configuration in the validated reference design. One route reflector cluster—cluster ID 192.168.2.10—includes backbone device 1 as the route reflector and all spine devices in data center 1 as route reflector clients. Another route reflector cluster—cluster ID 192.168.2.11—includes backbone device 2 as the route reflector and all spine devices in data center 2 as route reflector clients.

Figure 6: IBGP Route Reflector Topology

The validated reference design supports multiple hierarchical route reflectors, where one cluster includes backbone devices acting as route reflectors for the spine device clients and another cluster includes spine devices acting as route reflectors for leaf device clients. To see the configuration steps for configuring the other route reflector, see Configure IBGP for the Overlay.

Figure 7 shows the full hierarchical route reflector topology when two data centers are connected:

Figure 7: Hierarchical IBGP Route Reflector Topology

For more information on BGP route reflectors, see Understanding BGP Route Reflectors.

This procedure assumes IBGP has been enabled for the spine and leaf devices as detailed in Configure IBGP for the Overlay. The spine device configurations are included in this procedure to illustrate their relationships to the backbone devices.

To setup IBGP connectivity for the backbone devices:

EBGP is also used as the routing protocol between the backbone devices in this reference design. The backbone devices are connected using IP and the backbone devices must be configured as EBGP peers.

A second EBGP group—BACKBONE-BGP—is created in these steps to enable EBGP between the backbone devices. Each backbone device is assigned into a unique 32-bit AS number within the new EBGP group in these steps. The backbone devices, therefore, are part of two EBGP groups—UNDERLAY-BGP and BACKBONE-BGP—and have a unique AS number within each group. EVPN signaling, which has to run to support EVPN between the backbone devices, is also configured within the EBGP group during this procedure.

Figure 8 illustrates the attributes needed to enable EBGP between the backbone devices.

Figure 8: EBGP Topology for Backbone Device Connection

To enable EBGP as the routing protocol between the backbone devices:

EVPN Type 5 messages are exchanged between IRB interfaces on spine devices in different data centers when EVPN Type 5 routes are used for DCI. These IRB interfaces are configured in a routing instance.

Each data center has a unique virtual network identifier (VNI 102001 and 202001) in this configuration, but both VNIs are mapped to the same VLAN (VLAN 2001) in the same routing instance (VRF 501).

See Figure 9 for an illustration of the routing instance.

Figure 9: DCI Using EVPN Type 5 Routes

To enable DCI using EVPN Type 5 routes:

Enter the following commands to verify that traffic can be sent between data centers using EVPN Type 5 routes:

Table 1 provides a history of all of the features in this section and their support within this reference design.