Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
 

Apstra EVPN Support Addendum

When deploying EVPN on Apstra-supported devices and NOSs, you must be aware of several caveats and limitations. Even though EVPN is a standard, vendors implement protocols in very different manners. Also, different ASICs support varying feature sets that impact EVPN BGP VXLAN implementations (Routing In and Out of Tunnels (RIOT) for example). The following sections describe supported EVPN deployment implementations.

Qualified Vendor and NOS

Apstra software supports EVPN on the following hardware. For recommended NOS versions, see Qualified Device and NOS.

Hardware ASIC Support

Apstra supports EVPN on the following hardware ASICs:

  • Cisco Cloudscale
  • Mellanox Spectrum A1
  • Trident Trident2 (see below)
  • Trident Trident2+ (see below)
  • Trident Trident3 (see below)
  • Trident Tomahawk (see below)
  • Juniper Q5
Table 1: Apstra EVPN ASIC Support
ASIC Example Switches Notes
Arista Trident2 Arista DCS-7050 Can be used as Spine, Leaf, or Border Leaf. Must set up EOS Recirculation interface(s) to be used as a Layer3 Leaf (see Arista VXLAN documentation for more information).
Arista Trident3 DCS-7050CX3 Can be used as Spine, Leaf, or Border Leaf.
Arista XP80 Arista DCS-7160 Can be used as Spine, Leaf, or Border Leaf.
Arista Jericho DCS-7280R Can be used as Spine, Leaf, or Border Leaf.
Cisco Cloudscale Cisco 93180YC-EX Can be used as Spine, Leaf, or Border Leaf
Cisco Trident2 with ALE Cisco 9396PX, 9372PX, 9332PQ, 9504 Can be used as Spine, Leaf, or Border Leaf (see TCAM Carving in NXOS section).
Cisco Trident2+ Cisco 3132Q-V Cannot be used as Border Leaf
Cumulus Trident2+ Dell S4048T-ON, S6010-ON, EdgeCore AS5812-54X, AS6812-32X Can be used as Spine, Leaf, or Border Leaf (see Cumulus RN-766 Support) in Cumulus.
Cumulus Maverick Dell S4148F-ON, S4148T-ON Can be used as Spine, Leaf, or Border Leaf (see Cumulus RN-766 Support) in Cumulus.
Cumulus Mellanox A1 Mellanox MSN2010, MSN2100, MSN2410, MSN2700 (Spectrum A1) Can be used as Spine, Leaf, or Border Leaf. Spectrum A0 not supported.
Cumulus Tomahawk Dell Z9100-ON, EdgeCore AS7712-32X Can be used as Spine, Leaf, or Border Leaf (see Cumulus RN-766 Support). Must set up Hyperloop interface(s) to be used as a Layer3 Leaf in Cumulus.
Juniper Q5 Juniper QFX10002 Can be used as Spine, Leaf, or Border Leaf
Juniper Trident2 Juniper QFX5100 Can be used as Spine or Layer2 Leaf
Juniper Trident2+ Juniper QFX5110 Can be used as Spine, Leaf, or Border Leaf
Juniper Trident3 Juniper QFX5120 Can be used as Spine, Leaf, or Border Leaf

For recommended NOS versions, refer to Device and NOS Support <device_support>.

Limitations

EVPN Layer2 Limitations

  • VLAN (Rack-local) Virtual networks must be in the default routing zone.
  • VxLAN (Inter-rack) Virtual networks cannot be part of the default routing zone.

EVPN Layer3 Limitations

  • Generic systems with BGP peering to non-default routing zones must connect to leaf devices.
  • Generic systems with BGP peering only to the default routing zone can connect to leafs, spines or superspines.
  • Multi-zone security segmentations only support up to 16 routing zones (VRFs) on Arista (HW Limitation)
  • Inter routing zone (VRF) routing must be handled on a generic system (EVPN type 5 route leaking)
  • All BGP sessions and loopback addresses are part of the default routing zone.

Cumulus RN-766 Support

In all current versions of Cumulus Linux; when using Broadcom Trident II+, Trident3, and Maverick platforms in an external VXLAN routing environment; the switch does not rewrite MAC addresses and TTL, so packets are dropped by the next hop. See Cumulus Linux Release Notes for RN-766 for more information.

Work-arounds are automatically implemented for Cumulus Linux RN-766 for the following criteria:

  • Device profiles with "ASIC" field set to 'T2+', 'T3', or 'maverick' assigned to border-leaf device(s) in the blueprint
  • Overlay Control Protocol set to MP-EBGP EVPN
  • External Connectivity Point (ECP) configured for default or new routing zone
  • External Connectivity Point (ECP) set to L2 mode, with Layer 2 VLAN based BGP peering with the generic system
  • External Connectivity Point (ECP) configured with VLAN ID, SVI Subnet, and SVI IPs for border-leaf(s) and router(s).
  • User assigns VNI from a VNI pool to the blueprint

Additional VNI and configuration is automatically allocated for the Cumulus Linux RN-766 work-around.

CAUTION:

The RN-766 workaround is not supported on blueprints with MLAG L3 peer links deployed.

CAUTION:

RN-766 workaround is not supported for blueprints with Overlay Control Protocol set to Static VXLAN and any L3 External Connectivity Points (ECP).

TCAM Carving in NX-OS

To successfully deploy EVPN on Cisco Nexus devices other then Cisco Cloudscale, you must first configure Cisco NXOS TCAM carving. These other devices may include Cisco NXOSv, or Cisco Nexus "Trident2" devices such as 9396PX, 9372PX, 9332PQ, or 9504. On Cisco NXOS the ARP Suppression feature is used in order to minimize ARP flooding.

For details, see Juniper Support Knowledge Base article KB36733

Before installing the device agent, we recommend that you apply TCAM Carving during device management setup or during Cisco Power-on Auto Provisioning (POAP). TCAM Carving requires a device reboot.

Alternatively, you can apply TCAM Carving with configlets when you deploy the blueprint. You must manually reboot devices.

Use show hardware access-list tcam region to show and verify TCAM allocation on Cisco NX-OS.

Cisco NXOSv TCAM Carving

Cisco Trident2 TCAM Carving

Arista EOS VxLAN Routing

Recirculation Interface for Arista Trident2 Devices

VxLAN Routing for Trident2 devices (for example, 7050QX-32) is supported but requires assigning EOS recirculation interfaces to unused physical interfaces on the device. You can use configlets to deploy this to all devices that require this configuration.

VxLAN Routing System Profile for Arista Jericho Devices

We recommend when using VxLAN Routing for Jericho devices (for example, 7280SR-48C6) that you assign EOS VxLAN Routing System Profile on the device.

Before installing the device agent, we recommend that you apply the Arista TCAM system profile during the device management setup or during Arista Zero-Touch Provisioning (ZTP). TCAM system profile requires a device reboot.

Alternatively, you can use configlets to deploy this to all devices requiring this configuration and manually reboot the devices.

Graph Node VTEP Types

Unicast VTEPs

Unicast VTEPs do not apply to Cumulus and Arista.

Cisco Unicast VTEPs - Vendor Definition: Anycast VTEP

Apstra IP Allocation

Unique per leaf in MLAG pair

Not allocated to singleton switches

MLAG Configuration

Single Switch Configuration

Logical VTEPs

Arista Logical VTEPs

Apstra IP Allocation

Logical VTEP configured as primary IP on loopback1 interface for both MLAG and singleton switches

All top of rack nodes share same logical VTEP IP:

  • MLAG leafs share same logical VTEP IP
  • Singleton leaf gets its own VTEP IP

MLAG Configuration

Single Switch Configuration

Cumulus Logical VTEPs

Apstra IP Allocation

For MLAG (clagd) leafs, shared as clagd-vxlan-anycast-ip on lo interface, shared on leaf1 and leaf2

For singleton leafs, configured as additional IP alongside loopback ip on iface lo

MLAG Configuration

Single Switch Configuration

Anycast VTEP

Anycast VTEPs do not apply to Cisco and Cumulus.

Arista Anycast VTEPs

Apstra IP Allocation

One anycast VTEP for entire blueprint, shared between all Arista leafs

Configured as secondary IP on loopback1 interface

MLAG Configuration

Single Switch Configuration