Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
ON THIS PAGE
 

EVPN-VXLAN for AI-ML Data Centers

Overview of EVPN-VXLAN for AI-ML Data Centers

This document covers the steps necessary to configure Ethernet VPN-Virtual Extensible LAN (EVPN-VXLAN) in an artificial intelligence (AI) and machine learning (ML) data center fabric.

Features and Benefits of an AI-ML Data Center

  • Improve scalability: You can enable multitenancy within the same data center using an IP fabric overlay.

  • Improve productivity: You can run different AI workloads (multiple large language models (LLMs) for different tenants) in the same data center.

  • Improve security: You can isolate L2 at the local top-of-rack (ToR) level with multiple MAC-VRF instances, or L3 at the ToR level with multiple EVPN Type 5 routing instances (IP-VRF-to-IP-VRF model). See the configuration section for examples of these use cases.

  • Reduce configuration efforts: You can extend the tenants' logical context between different ToR switches in different points of delivery (PODs) without changing the configuration of the intermediate spine or superspine devices.

Use Feature Explorer to confirm platform and release support for specific features.

Configuration

Configuration Overview

We'll look at two use cases relevant to this topic. The first use case is running two MAC-VRF instances on the same device in a data center. The second use case is running two EVPN Type 5 VRF instances on the same device in a data center.

Use Case #1: Two MAC-VRF instances on the same device:

  • Separate MAC-VRF instances help to isolate the AI data center tenants at the L2 level, and extend this isolation using the EVPN-VXLAN overlay.

  • The intermediate AI data center spine and superspine devices don't require provisioning each new AI data center tenant.

  • The L2 connectivity is closer to the actual service connection.

  • AI data center tenants can be in the same MAC-VRF L2 EVPN instance (EVI) when you configure the tenants with the vlan-aware EVPN service type.

Use Case #2: Two EVPN Type 5 IP-VRF instances on the same device:

  • Multiple EVPN Type 5 routing instances can isolate the AI data center tenants at the L3 routing level. Pure Type 5 routing can also extend the context within a POD or between PODs.

  • EVPN signaling exchanges between the ToR switches of the AI data center automatically establish VXLAN tunnels for Type 5 routes.

Topology

The topology for these examples uses QFX5240-64QD switches for both the spine and leaf layers. The network is an edge-routed bridging (ERB) architecture.

Figure 1: Topology Network topology diagram showing gateway devices GW1 and GW2, leaf switches Leaf1 and Leaf2, and host devices H1 and H2. GW1 and GW2 interconnect and link to Leaf1 and Leaf2. Leaf1 connects to H1, Leaf2 to H2, illustrating a data center setup with redundancy.

How to Configure Two MAC-VRFs

Use the following steps as a guide to configure two MAC-VRF instances on the same leaf node. We use actual values for example purposes. You should customize these steps with relevant values for your implementation.
  1. Configure a MAC-VRF routing instance.
  2. Configure the EVPN protocol with VXLAN encapsulation and supporting statements.
  3. Configure a virtual tunnel endpoint (VTEP) interface.
  4. Configure a service type. We use vlan-aware for this example. vlan-aware allows configuring more than one VLAN.
  5. Configure an interface for the routing instance.
  6. Configure a route distinguisher (RD) and a VRF target.
  7. Configure one or more VLANs.
  8. Configure a second MAC-VRF routing instance. The full configuration is displayed. Note the differences in VLANs, RD, VRF-target, and interfaces. This routing instance uses the vlan-based service type, limiting the configuration to a single VLAN. Either vlan-based or vlan-aware are valid choices.

Verification

Verify that routing is working as expected. Note that verification requires other network devices to be configured, and your outputs will vary.

How to Configure Two Type 5 IP-VRFs

Use the following steps as a guide to configuring two Type 5 IP-VRFs on the same leaf node. We use actual values for example purposes. You should customize these steps with relevant values for your implementation.

  1. Configure a VRF routing instance.
  2. Configure the EVPN protocol with Type 5 support.
  3. Configure routing options.
  4. Configure interfaces, RD, and VRF target.
  5. Configure a second Type 5 IP-VRF on the same leaf node. The full configuration is displayed. Note the differences in VLANs, RD, VRF-target, and interfaces.
  6. The routing policy supporting each VRF is shown here.

Verification

Verify that routing is working as expected. Note that verification requires other network devices to be configured, and your outputs will vary.

Related Documentation