Using a RIOT Loopback Port to Route Traffic in an EVPN-VXLAN Network

RIOT Loopback Solution Overview

You can configure a Layer 3 VXLAN gateway using a RIOT loopback port in an EVPN-VXLAN ERB fabric with:

• MAC VRF routing instances—either VLAN-based or VLAN-aware bundle service type.

  See MAC-VRF Routing Instance Type Overview for more on MAC-VRF routing instances.

• Enterprise style interface configuration.

  See Flexible Ethernet Services Encapsulation and Understanding Flexible Ethernet Services Support With EVPN-VXLAN for details on enterprise style and flexible Ethernet services configuration.

• EVPN Type 2 routing (with both the asymmetric and symmetric integrated routing and bridging [IRB] models) and EVPN Type 5 routing.

  Starting in Junos OS Release 21.3R1-S1 and 21.4R1, you can enable symmetric EVPN Type 2 routing on QFX5210 switches that act as Layer 3 gateways. See Symmetric Integrated Routing and Bridging with EVPN Type 2 Routes in EVPN-VXLAN Fabrics for more details on the symmetric IRB model.

  Note:

  We support EVPN Type 5 routing using only the symmetric IRB model. This is the default behavior when you configure a routing instance to use Type 5 routes with the ip-prefix-routes statement. See Understanding EVPN Pure Type 5 Routes for an overview of EVPN Type 5 routes and other EVPN route types.

The following figure shows an EVPN-VXLAN ERB fabric. The leaf devices route traffic through VXLAN tunnels to the other leaf devices in the fabric. Leaf 1 is a QFX5210 switch that doesn't support native VXLAN routing in and out of the VXLAN tunnel.

Figure 1: EVPN-VXLAN ERB Overlay Fabric with a RIOT Loopback Layer 3 Gateway Leaf Device

For Leaf 1 to serve as Layer 3 VXLAN gateway, you need to configure the RIOT loopback port solution on that device. RIOT loopback routing is transparent to the other leaf devices that connect through VXLAN tunnels in the fabric. You can include Layer 3 gateway leaf devices in the same fabric that use RIOT loopback port routing with devices that use native VXLAN routing.

How RIOT Loopback Processing Works

The following figure shows how the RIOT loopback process routes traffic in or out of a VXLAN tunnel that connects to another leaf device.

Figure 2: RIOT Loopback Two-Pass Processing

You configure the RIOT loopback port as an Ethernet link aggregation group (LAG). Adjust the number of member links depending on the bandwidth of VXLAN traffic that uses the loopback path. You can use any network ports for the LAG that aren't already used for another purpose. The device automatically turns off MAC learning on the RIOT loopback LAG so the port doesn't learn its own MAC address when traffic passes through it.

The traffic flows through the RIOT loopback LAG in the first pass, then loops back into the RIOT loopback LAG for the second pass. What happens during the RIOT loopback process depends on the direction of traffic flow and the type of routes.

The device uses the RIOT loopback process for:

• Access port to network port routing and VXLAN tunnel initiation with asymmetric or symmetric Type 2 routing and Type 5 routing.

• Network port to access port VXLAN tunnel termination and routing with symmetric Type 2 routing and Type 5 routing only.

The device doesn't need to use the RIOT loopback LAG in the following cases:

• Access port to access port with asymmetric Type 2 routing.

  The device routes the traffic locally through the IRB interfaces on the device as usual (no VXLAN bridging needed).

• Network port to access port with asymmetric Type 2 routing.

  In this case, the ingress VTEP already routed the traffic exiting the VXLAN tunnel to the destination VLAN. The device uses normal Layer 2 VXLAN traffic processing to bridge the traffic on the destination VLAN.

We describe more about the RIOT loopback process with different EVPN route types next. Also, see Configure a RIOT Loopback Port on a Layer 3 VXLAN Gateway Leaf Device for details on what you need to configure for each EVPN route type.

Asymmetric Type 2 Routes with RIOT Loopback Processing

With asymmetric Type 2 routing, all VLANs extend over the EVPN network on all devices. The integrated bridging and forwarding (IRB) actions on the two VXLAN tunnel endpoints (VTEP) differ as follows:

• The ingress VTEP IRB interface routes the traffic from the source VLAN to the destination VLAN. Then the device bridges the traffic on the destination VLAN across the VXLAN tunnel.

  Note:

  With asymmetric routing, you must configure all destination VLANs on the ingress VTEP even if the device doesn't serve hosts on all VLANs.

• The egress VTEP receives the traffic on the destination VLAN, and then forwards the traffic on the destination VLAN. The egress VTEP doesn't need to route the traffic.

This means that devices don't need the RIOT loopback process with asymmetric Type 2 routes on VXLAN tunnel traffic coming into the device from the EVPN network.

Symmetric Type 2 and Type 5 Routes with RIOT Loopback Processing

For symmetric Type 2 routes and Type 5 routes to work with the RIOT loopback process, the device uses an extra VLAN for each tenant Layer 3 (L3) virtual routing and forwarding (VRF) instance. Each extra VLAN maps to the VXLAN network identifier (VNI) for the corresponding L3 VRF instance.

The extra VLAN enables symmetric Type 2 or Type 5 routing in both directions (to and from the VXLAN tunnel) as follows:

• Access port to network port traffic:

  In the first pass, the RIOT loopback process routes the traffic out of the RIOT loopback port. In the second pass for tunnel initiation, the RIOT loopback process needs the VNI for the corresponding L3 VRF instance. The RIOT loopback process uses the extra VLAN's VLAN-to-VNI mapping for that purpose.

• Network port to access port traffic

  When terminating the VXLAN tunnel, the device needs a VLAN tag with which to send the traffic out of the RIOT loopback port. The RIOT loopback process adds the extra VLAN ID as the VLAN tag in the first pass. In the second pass, the RIOT loopback process uses the VLAN tag to find the corresponding L3 VRF instance to do the route lookup.

IRB Interface Status Dependency on RIOT Loopback Port State

Layer 3 VXLAN gateway devices route traffic between VLANs using IRB interfaces. On devices with RIOT loopback processing, all IRB interfaces you configure for VXLAN routing depend on the RIOT loopback LAG. As a result, the RIOT loopback LAG must be available to process VXLAN traffic before the IRB interfaces are able to route traffic. For this feature to work, the device must consider the RIOT loopback LAG state when determining IRB interface status.

With RIOT loopback configuration, you configure the device to include the state of another local interface when evaluating the status of an IRB interface. In this case, the local interface is the RIOT loopback LAG.

Use the local-interface name statement at the [edit interfaces irb unit unit-number interface-state] hierarchy. Specify name as the logical interface name of the RIOT loopback LAG for the unit.

Also, to configure the delay to ensure that the RIOT loopback LAG is up before the device evaluates the IRB interface as up, you configure the hold-time up seconds option at the [edit interfaces irb unit unit-number interface-state] hierarchy. This value is the time the device waits after the RIOT loopback interface is up before it includes that state when evaluating the IRB interface status.

The RIOT loopback LAG usually remains up unless you change its configuration. As a result, we recommend to set hold-time up to a higher value based on the scale of routes in your network. A higher value helps prevent the IRB logical interfaces from flapping. We recommend that you try approximately 120 seconds for medium-to-large scale deployments.