APPENDIX: IP Clos Fabric Verification (Optional)

Wired Client Verification

Verification of the campus fabric IP Clos deployment. See Figure 1. Currently, there are five desktop VMs to validate the fabric with static IP address configuration. Let’s take a quick look to see if Desktop1 VM can connect internally and externally.

Validation steps:

• Confirmed local IP address, VLAN, and default gateway were configured on Desktop1 VM.
• Can ping default gateway—indicates that we can reach the access switch.
• Can ping Desktop5 VM in the same VLAN at the same switch.
• Can ping Desktop4 VM in the same VLAN at the other switch (access2 Virtual Chassis).
• Can not ping Desktop2 + 3 VM in different VRF. This is expected as there is isolation amongst VRFs in the fabric and we have not integrated the WAN router yet to hairpin that traffic.
• Can not ping the Internet. This is expected as we have not integrated the WAN router yet.
• ARP shows MAC addresses in Desktop1 VLAN reached to.
• No DHCP lease handouts. This is expected as our DHCP server is beyond the WAN router attached.

To further validate the campus fabric in the portal, select the Campus Fabric option under the Organization tab on the left side of the portal.

Remote shell access into each device within the campus fabric is supported here as well as a visual representation of the following capabilities:

• Automatically assigned router ID (lo0.0)
• BGP peering establishment.
• Transmit and receive traffic on a link-by-link basis.
• Telemetry, such as LLDP, from each device that verifies the physical build.

BGP Underlay Verification

Purpose

Verifying the state of eBGP between the core and distribution layers is essential for EVPN-VXLAN to operate as expected. This network of point-to-point links between each layer supports:

• Load balancing using ECMP for greater resiliency and bandwidth efficiencies.
• BFD to decrease convergence times during failures.
• Loopback reachability to support VXLAN tunnelling.

In the above figure, you see a (simulated) bad link between two fabric nodes that you should fix.

Due to the automated assignment of router IDs and loopback IP addresses in the fabric, we have the following configuration to remember:

Switch Type	Switch Name	Auto assigned Loopback IP
Service Block	service1	172.16.254.1
Service Block	service2	172.16.254.2
Core	core1	172.16.254.3
Core	core2	172.16.254.4
Distribution	dist1	172.16.254.6
Distribution	dist2	172.16.254.5
Access Standalone	access1	172.16.254.8
Access Virtual Chassis	access2	172.16.254.7

Another way to find the assigned router ID and loopback IP address is the switch configuration panel as in the below figure:

Action

Verify that BGP sessions are established between core devices and distribution devices to ensure loopback reachability, BFD session status, and load-balancing using ECMP.

Note:

Operational data can be gathered through the campus fabric section of the portal as Remote Shell or using an external application such as SecureCRT or Putty.

Verification of BGP Peering

Access the Remote Shell via the lower-right corner of the campus fabric, from the switch view, or via Secure Shell (SSH).

Core1 Switch:

From the BGP summary, we can see that the underlay (10.255.240.X) peer relationships are established, which indicates that the underlay links are attached to the correct devices and the links are up.

It also shows the overlay (172.16.254.x) relationships are established and that it is peering at the correct loopback addresses. This demonstrates underlay loopback reachability.

We can also see routes received; time established are roughly equal which looks good so far.

The campus fabric build illustrates per device real-time BGP peering status shown below from Core1:

If BGP is not established then go back and validate the underlay links and addressing, and that the loopback addresses are correct. Loopback addresses should be pingable from other loopback addresses.

Verification of BGP connections can be performed on any of the other switches (not shown).

The primary goal of eBGP in the underlay is to provide loopback reachability between core and distribution devices in the campus fabric. This loopback is used to terminate VXLAN tunnels between devices. The following shows loopback reachability from Core1 to all devices in the campus fabric:

Note:

eBGP sessions are established between core-distribution and core-service layers in the campus fabric. Loopback reachability has also been verified between the core and all other devices.

Let’s verify that the routes are established to the access layer and other devices across multiple paths. For example, Core1 should leverage both paths through dist1 and dist2 to reach access1 and vice versa.

This is the reverse check from access1 to be able to reach core1 via dist1 and dist2 switches.

Finally, we validate BFD for fast convergence in the case of a link or device failure:

Conclusion: At this point, the BGP underlay and overlay are operational through the verification of eBGP between corresponding layers of the campus fabric and loopback routes are established between service, core, distribution and access layers.

We have one link in the fabric that did not come up so we have not met the full requirement before we can go into production. However, we can further use the fabric as all nodes in the fabric can still reach each other.

EVPN VXLAN Verification Between Access and Service Block Switches

Since the desktop can ping its default gateway, we can assume the Ethernet switching tables are correctly populated, and VLAN and interface modes are correct. If pinging the default gateway failed, then troubleshoot client to access connectivity.

Note:

In this fabric, the core and distribution switches do not have any VXLAN VTEPs assigned since no VLAN interface is configured there. Hence, we will not find any VXLAN or EVPN information on those devices. Those devices just forward our control and data packets.

Verification of the MAC address table, the EVPN database and remote VXLAN tunnel summary on both access switches is shown below:

Access1 Switch:

Access2 Switch:

Both access switches have all three VLANs assigned locally hence they will show information about local MAC-Addresses as well as once learned remotely via EVPN. The EVPN Database also reflects this learning. Also, the individual remote VXLAN VTEP’s are learned using the router ID / lo0.0 assigned to remote switches.

Verification of the MAC-Address table, the EVPN Database and remote VXLAN-Tunnel summary on both Service Block Switches.

Service1 Switch:

Service2 Switch:

At this point, you won’t see any data in the Ethernet switching table on the service block switches, since no VLAN is defined there yet. The EVPN database is in sync between the service block switches, which is expected. Also, the individual remote VXLAN VTEPs are learned using the router ID / lo0.0 assigned to the remote switches.

In the next step, we verify the configuration of VLAN1099 and what VNI was assigned to it on the two access and service block switches:

Note:

Juniper Mist cloud management does not enable a user to configure the value for the VNI used for a particular VLAN so as to maintain consistency among different fabrics. Today the VNI is calculated by the VLAN-ID + 10.000 (but may change in the future).

We need to configure the attachment of the WAN router to complete the entire design. Without the WAN router configuration, the fabric only enables the following communications:

• The same VLAN/VNI on the same access switch but different ports.
• The same VLAN/VNI on different access switches.
• Different VLAN/VNI attached to the same VRF on the same access switch, but different ports.
• Different VLAN/VNI attached to the same VRF on different access switches.

All traffic between VRFs is always isolated inside the fabric. For security reasons, there is no possible configuration to perform route leaking between VRFs. This means that traffic between them is handled directly inside the fabric without the need to traverse through the WAN router as a possible enforcement point.