Verification

BGP Underlay

Purpose

Verifying the state of eBGP between adjacent layers is essential for EVPN VXLAN to operate as expected. This network of point-to-point links between each layer supports:

• Load balancing using ECMP for greater resiliency and bandwidth efficiencies.
• Bi-directional forwarding (BFD), to decrease convergence times during failures.
• BGP peering.
• Loopback to VXLAN reachability.

Without requiring verification at each layer, the focus can be on Dist1 and Dist2 and their eBGP relationships with Access1 and Access2, and Core1 and Core2. If both distribution switches have “established” eBGP peering sessions with each adjacent layer, you can move to the next phase of verification.

Due to the automated assignment of loopback IP addresses, for this fabric, we have the following configuration to remember:

Switch Type	Switch Name	Auto assigned Loopback IP
Core	Core1	172.16.254.2
Core	Core2	172.16.254.1
Distribution	Dist1	172.16.254.3
Distribution	Dist2	172.16.254.4
Access	Access1	172.16.254.6
Access	Access2	172.16.254.5

Action

Verify that BGP sessions are established from Dist1 and Dist2 with access and core devices to ensure loopback reachability, BFD session status, and load-balancing using ECMP.

Note:

Operational data can be gathered through the Campus Fabric section of the Mist GUI or using an external application such as SecureCRT or Putty.

Verification of BGP Peering

Dist1:

From Switch > Utilities, access Remote Shell via the bottom right of the campus fabric, from the switch view or via Secure Shell (SSH).

From the BGP summary, we can see that the underlay (10.255.240.X) peer relationships are established to indicate that the underlay links are attached to the correct devices and the links are up.

It also shows the overlay (172.16.254.x) relationships are established and that it is peering at the correct underlay loopback addresses. This demonstrates underlay loopback reachability.

We can also see the routes received. Since the time established is roughly equal, this looks good so far.

The campus fabric build illustrates per device real-time BGP peering status shown in Figure 2 from Dist1.

If BGP is not established, go back and validate the underlay links and addressing. Also, ensure that the loopback addresses are correct. Loopback addresses must be pingable from other loopback addresses. For example, Dist1 can reach Core1 and Access1’s loopback addresses once the underlay eBGP peering sessions are established.

Note:

eBGP sessions are established between adjacent layers in the Campus Fabric IP Clos.

Let’s verify the routes are established to the core and other devices across multiple paths. For example, Access1 and Access2 should leverage both paths through Dist1 and Dist2 to access Core1 and Core2’s loopback addresses as well as each other’s loopback addresses.

Access1: Loopback reachability to Core1 through Dist1 and Dist2

Access1: Loopback reachability with Core2 through Dist1 and Dist2

Access1: Loopback reachability with Access2 through Dist1 and Dist2

This can be repeated for Access 2 and so forth to verify ECMP load balancing.

Meaning: At this point, BGP underlay and overlay are operational through the verification of eBGP between corresponding layers of the campus fabric, and that routes are established to access, core, and distribution layers.

EVPN VXLAN Verification Between Access and Core Switches

Since the desktop can ping its default gateway, we can assume the Ethernet switching tables are correctly populated, and that VLANs and interface mode are correct. If pinging the default gateway failed, then troubleshoot underlay connectivity.

Verification of the EVPN Database on Both Access Switches

You can view the entire database or search by MAC address.

Both access switches have identical EVPN databases, which is expected. Notice the entries for Desktop1 (10.99.99.99) and Desktop2 (10.88.88.88) present in each access switch. These entries are learned locally or through the campus fabric as shown in the Active Source column of the output.

10.99.99.99 is associated with irb.1099 and we see VNI of 11099. Let's just double-check VLAN-VNI mapping on the access and core switches.

Access

Core

Verification of VXLAN Tunnelling Between Access and Core Devices

Access1:

Access 2:

We need to configure the attachment of the WAN router to complete the entire design. Without the WAN router configuration, the fabric only allows the following communications.

Note:

Neither access switch displays Core1 or Core2 as remote tunnel destinations. The reason is that we have not yet configured the uplinks to the WAN router on top of the fabric. Hence no core switch knows about VLAN1099.

External Campus Fabric Connectivity Through the Border Gateway Core EX9204 Switches

There are several options available for attaching a WAN router to a campus fabric:

• Using a L2 forwarding:
• With this method, the fabric uplinks are configured as ESI-LAG and contain one or more tagged VLANs (one for each VRF) to communicate with the WAN router.
• You must configure the IP address of the WAN router interface manually as the next-hop IP address for default forwarding on each fabric VRF as already done above.
• The WAN router must understand standard IEEE 802.3ad LAG with active LACP.
• If you have more than one WAN router attached for redundancy, then we recommend that you configure VRRP between them for the interface IP addresses towards the Fabric.
• Using a L3 forwarding:
• Configure the fabric uplinks as L3 peer-to-peer IP links.
• You must establish a peer-to-peer link, per fabric VRF, with the WAN router.
• Usually, there are multiple peer-to-peer links on a single physical uplink. The peer-to-peer links are further segmented using tagged VLANs to provide isolation on the uplinks.
• There is no need to manually configure the next hops for each VRF inside the fabric since it is assumed that the propagation of the default gateways will be obtained for the WAN router via a routing protocol.
• Between the Fabric and the WAN router, a routing protocol must be established to exchange routes. Campus fabric supports external BGP and OSPF as routing protocols towards the WAN router.

For simplicity, we have chosen to utilize the L2 exit via ESI-LAG in this JVD. However, this is not the recommended method for large fabrics since IP Clos is intended to grow. For IP Clos fabrics in production environments, you should consider using BGP as the route exchange protocol as well as a redundant pair of WAN routers.

Remember that you chose to leverage the BGP capability of the EX9204 switches during the Campus Fabric IP Clos deployment, as shown in Figure 3 .

Mist enables the EX9204 Switch to translate between VXLAN traffic within the campus fabric and standard Ethernet switching for external connectivity, in this case, an MX router. Let’s verify the Ethernet Segment Identifier (ESI) status on the core switches.

We must configure the ESI-LAG since Mist does not configure this automatically. Add a port profile on core switches interfaces facing the WAN router.

The following represents an existing port profile applied to each MX router facing an EX9204 switch port. (On the Core2 switch the Port is xe-1/0/1).

Save the configuration and then verify the changes on the core switch.

Note that LACP is up, and this infers there is an existing configuration on the MX router.

Verify if Desktop1’s MAC address is advertised via BGP:

Verify if the route is received on the core.

Let's check to see if the core has Desktop1 MAC address.

Verify the MAC address mapped to the correct VTEP interface. This is on the core. You can also verify on an access switch.

Finally, the VTEP interface is up and passing traffic.

Then, confirm the EVPN database now has the ESI entry. Back to Desktop1 to see if it can cross the fabric.

Last step is to verify Desktop1 can ping Desktop2.

Meaning: Connectivity within the campus fabric and externally is verified. Desktops communicate with each other through the campus fabric, each in an isolated VRF, then forwarded to the MX router through the dual homing ESI-LAG on both Core1 and Core2 for routing between VRFs or routing instances. Internet connectivity was also verified from each desktop.