Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
ON THIS PAGE
 

Example: Configure an IPv6 Underlay for Layer 2 VXLAN Gateway Leaf Devices

Overview

Ethernet VPNs (EVPNs) enable you to connect customer sites using Layer 2 virtual bridges. Virtual Extensible LANs (VXLANs) establish overlay tunnels that stretch the Layer 2 connection over an intervening Layer 3 network. Like VLANs, VXLANs help provide network segmentation, but without the scaling limitation of traditional VLANs. EVPN with VXLAN encapsulation enables Layer 2 connectivity at scale.

The physical underlay network in EVPN-VXLAN installations is often a two-layer IP fabric that includes spine and leaf devices. In this design, the spine devices provide connectivity between the leaf devices and the leaf devices provide connectivity to attached hosts. In different overlay network configurations, the leaf or spine devices might function as either or both of the following:

  • Layer 2 gateways that handle traffic within a VXLAN.

  • Layer 3 gateways that handle traffic between VXLANs using integrated routing and bridging (IRB) interfaces.

On supported platforms, in either case, the underlay network for the VXLAN overlay can use the IPv6 protocol to take advantage of the extended addressing and other capabilities of IPv6.

This example shows a use case to configure an IPv6 underlay for the Layer 2 VXLAN gateway leaf devices in a simple EVPN-VXLAN fabric. In this use case, the EVPN-VXLAN fabric supports a bridged overlay with VXLAN tunnels between the leaf devices. The leaf devices connect to end systems that might be single homed or include EVPN multihoming for redundancy. The following figure shows a high-level view of the topology in this example:

Figure 1: EVPN-VXLAN Fabric with an IPv6 Underlay for Layer 2 VXLAN Gateway DevicesEVPN-VXLAN Fabric with an IPv6 Underlay for Layer 2 VXLAN Gateway Devices

The following list describes the main differences in how you set up an IPv6 underlay compared to setting up an IPv4 underlay:

  • You assign an IPv6 address to the loopback interface on the devices that serve as the Layer 2 or Layer 3 VXLAN gateway VTEPs.

  • ACX Series and QFX Series devices support an IPv6 VXLAN underlay only with MAC-VRF routing instances. (See MAC-VRF Routing Instance Type Overview for more information about using MAC-VRF routing instances.) As a result, you configure the EVPN instance as a MAC-VRF instance.

  • On ACX Series and QFX Series devices, you set the VTEP source interface as an IPv6 address. You also must assign IPv6 addresses to the EVPN core-facing interfaces for IP reachability with IPv6.

  • IPv6 protocols need a 32-bit router ID to function properly. The router ID you assign must be:

    • A 32-bit value as a 4-octet, unsigned, non-zero integer.

    • Unique within the routing domain.

  • You can’t mix IPv4 and IPv6 underlays in the same fabric, so you must configure an IPv6 underlay across all EVPN instances in the fabric.

Requirements

This example consists of a full mesh two-layer spine-and-leaf EVPN-VXLAN fabric with two spine devices and three leaf devices. You can configure the IPv6 underlay in this example using:

  • QFX Series switches that support this feature.

  • Junos OS Release 21.4R1 or later for QFX5120 switches and switches in the QFX10000 line, or Junos OS Evolved 22.3R1 or later for QFX5130-32CD and QFX5700 switches.

    Note:

    We also support this feature in Junos OS Release 21.2R2 on QFX5120 switches and switches in the QFX10000 line and in Junos OS Evolved Release 23.4R1 on ACX7100-32C and ACX7100-48L devices .

The leaf devices can host multihomed or single homed end devices on the access side. This example illustrates configuring an Ethernet segment for EVPN multihoming on one leaf and a single homed end system interface on another leaf. However, the elements you configure for the IPv6 underlay are independent of the access-side configuration.

Topology

This example shows how to configure an IPv6 underlay on Leaf 1 and Leaf 3 for VXLAN overlay tunnels like those in Figure 1. The configuration uses OSPFv3 for IPv6 connectivity and iBGP with IPv6 neighbor addressing in a single autonomous system in the following topology:

Figure 2: Example TopologyExample Topology

Leaf 1 serves a customer edge switch that is multihomed to Leaf 1 and Leaf 2, so you would use a similar configuration on Leaf 2 to reach devices on that Ethernet segment.

In the example topology, Leaf 1 includes an aggregated Ethernet interface bundle for the connection to Spine 1. You configure the remaining spine and leaf connections on Leaf 1 and Leaf 3 as single interfaces. Leaf 3 includes an access-side interface configuration to a single-homed end system.

This example includes show commands you can run to verify IPv6 underlay operation. For simplicity, we show these verification commands and output only for Leaf 3. You see similar results from the same commands on the other leaf devices.

Configure Leaf 1

CLI Quick Configuration on Leaf 1

To quickly configure Leaf 1 with an IPv6 underlay according to Figure 2, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, and then copy and paste the commands into the CLI at the [edit] hierarchy level.

Step-by-Step Procedure on Leaf 1

  1. Configure the interfaces for the EVPN fabric device connections. For illustrative purposes, in this example Leaf 1 connects to Spine 1 with an aggregated Ethernet (AE) interface bundle and to Spine 2 with a single interface.

  2. Configure an interface for network traffic and the associated VLANs. This example uses a service provider style interface configuration.

  3. Assign an IPv6 address to the loopback interface on this device.

  4. (QFX5120 switches only) A device might have problems with VTEP scaling when the configuration uses multiple MAC-VRF instances. As a result, to avoid this problem, we require that you enable the shared tunnels feature on QFX5120 switches when setting up an IPv6 underlay. When you configure the shared-tunnels option, the device minimizes the number of next-hop entries to reach remote VTEPs.

    Include the following statement to globally enable shared VXLAN tunnels on the device:

    Note:

    After you configure the shared-tunnels option, you must reboot the device for the setting to take effect.

    The shared tunnels feature is enabled by default on QFX5130-32CD and QFX5700 switches, so you don't need to explicitly set this option on those switches.

    This statement is optional on the QFX10000 line of switches, which can handle higher VTEP scaling.

  5. (ACX7100-32C, AX7100-48L, and ACX7024 devices only) You can use enable an IPv4 or IPv6 underlay. If you are currently using an IPv4 underlay, and you want to use an IPv6 underlay instead, you will need to remove any existing VXLAN configurations first.

    To enable an IPv6 underlay, enable the vxlan-extended statement.

    After you enable the vxlan-extended statement, the packet forwarding engine (PFE) reboots. After the PFE comes back up, you can configure an underlay and overlay for the IPv6 VXLAN underlay. Once you have configured the IPv6 VXLAN underlay and overlay, your VXLAN deployment will be using an IPv6 underlay.

  6. (ACX7100-32C, AX7100-48L, and ACX7024 devices only) To go back to using an IPv4 underlay, enable the default system profile.

    Apply your underlay and overlay configuration for your IPv4 VXLAN underlay configuration. After the PFE reboots, your VXLAN deployment will be using an IPv4 underlay.

  7. Create an EVPN-VXLAN MAC-VRF instance. To use an IPv6 underlay, on ACX Series and QFX Series devices, you configure the device loopback interface as an IPv6 VTEP source interface. You configure the IPv6 underlay in a later step.

    In this step you also configure the following elements in the MAC-VRF instance:

    • Set the VLAN-aware Ethernet service type so you can associate multiple VLANs with the instance.

    • Assign a route distinguisher for the instance.

    • Assign the route target.

      We also set the auto route target option here, which uses one target for both import and export and helps to simplify the configuration.

  8. Configure the VLANs associated with the MAC-VRF instance and VLAN to VNI mappings—in this example, VLAN 100 (VNI 1100) and VLAN 110 (VNI 1110). This step includes the access-side ESI interface in the instance as well (ae10, which you configure in the last step).

  9. Set up the IPv6 underlay. This example uses OSPFv3 for the IPv6 underlay connectivity.

    Note:

    You might alternatively use BGP (for example, external BGP [eBGP]) as the IPv6 underlay routing protocol.

  10. Set up the IPv6 overlay. This example uses internal BGP (iBGP) as the overlay routing protocol for EVPN with VXLAN tunneling. Our example is based on a pure IPv6 overlay. This means we must explicitly configure a router ID because there are no IPv4 addresses assigned to any interface. In this example, we configure the router ID as an arbitrary 32-bit integer in dotted quad notation.

    Note:

    When running IPv6 routing protocols, you must configure a router ID for proper operation. The router ID must be a 4-octet unsigned non-zero integer that is unique in the routing domain.

    For simplicity, the router ID is often set to match an IPv4 address on the router, typically a loopback address. While the router ID looks like an IPv4 address, there is no need for it to be routable, or that it be assigned to any interfaces on the device. In an IPv6-based overlay, we use the IPv6 loopback address for the VTEP local address.

  11. Set up an Ethernet segment (ESI) from Leaf 1 to CE 1, which is multihomed to Leaf 1 and Leaf 2. You would configure the ESI on Leaf 2 similarly. For simplicity, this example doesn't show the Leaf 2 configuration.

Configure Leaf 3

CLI Quick Configuration on Leaf 3

To quickly configure Leaf 3 with an IPv6 underlay according to Figure 2, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, and then copy and paste the commands into the CLI at the [edit] hierarchy level.

Step-by-Step Procedure on Leaf 3

  1. Configure the interfaces for the EVPN fabric device connections from Leaf 3 to Spine 1 and Spine 2.

  2. Configure an interface for network traffic and the associated VLANs. This example uses a service provider style interface configuration.

  3. Assign an IPv6 address to the loopback interface on this device ..

  4. (QFX5120 switches only) A device might have problems with VTEP scaling when the configuration uses multiple MAC-VRF instances. As a result, to avoid this problem, we require that you enable the shared tunnels feature on QFX5120 switches when setting up an IPv6 underlay. When you configure the shared-tunnels option, the device minimizes the number of next-hop entries to reach remote VTEPs.

    Include the following statement to globally enable shared VXLAN tunnels on the device:

    Note:

    The shared tunnels feature is enabled by default on QFX5130-32CD and QFX5700 switches, so you don't need to explicitly set this option on those switches.

    This statement is optional on the QFX10000 line of switches, which can handle higher VTEP scaling.

  5. (ACX7100-32C, AX7100-48L, and ACX7024 devices only) You can use enable an IPv4 or IPv6 underlay. If you are currently using an IPv4 underlay, and you want to use an IPv6 underlay instead, you will need to remove any existing VXLAN configurations first.

    To enable an IPv6 underlay, enable the vxlan-extended statement.

    After you enable the vxlan-extended statement, the packet forwarding engine (PFE) reboots. After the PFE comes back up, you can configure an underlay and overlay for the IPv6 VXLAN underlay. Once you have configured the IPv6 VXLAN underlay and overlay, your VXLAN deployment will be using an IPv6 underlay.

  6. (ACX7100-32C, AX7100-48L, and ACX7024 devices only) To go back to using an IPv4 underlay, enable the default system profile.

    Apply your underlay and overlay configuration for your IPv4 VXLAN underlay configuration. After the PFE reboots, your VXLAN deployment will be using an IPv4 underlay.

  7. Create an EVPN-VXLAN MAC-VRF instance. To use an IPv6 underlay, you configure the device loopback interface as an IPv6 VTEP source interface. You configure the IPv6 underlay in a later step.

    In this step you also configure the following elements in the MAC-VRF instance:

    • Set the VLAN-aware Ethernet service type so you can associate multiple VLANs with the instance.

    • Assign a route distinguisher for the instance.

    • Assign the route target.

      We also set the auto route target option here, which uses one target for both import and export and helps to simplify the configuration.

  8. Configure the VLANs associated with the MAC-VRF instance and VLAN to VNI mappings—in this example, VLAN 100 (VNI 1100) and VLAN 110 (VNI 1110).

  9. Set up the IPv6 underlay. This example uses OSPFv3 for the IPv6 underlay connectivity.

    Note:

    You might alternatively use BGP (for example, external BGP [eBGP]) as the IPv6 underlay routing protocol.

  10. Set up the IPv6 overlay. This example uses using internal BGP (iBGP) as the overlay routing protocol for EVPN with VXLAN tunneling.

    Note:

    Even though we use the IPv6 address family, you must configure a router ID for proper operation. The router ID must be a 4-octet unsigned non zero integer that is unique in the routing domain. You configure the router ID using dotted quad notation.

    In an IPv6 based overlay, we use the IPv6 loopback address for the VTEP local address.

Verify the IPv6 Underlay on Leaf 3

Use the CLI commands in this section to verify the IPV6 underlay configuration is operational on the leaf devices in this example. This section shows the results from running these commands on Leaf 3.

This example includes show mac-vrf forwarding command-name commands that display information for MAC-VRF instance configurations. Most show mac-vrf forwarding commands are aliases for the same command in the following command hierarchies that you might use for the default switching instance or other instance types:

  • QFX Series switches—show ethernet-switching command-name

  • MX Series routers and EX9200 line of switches—show l2-learning command-name or show bridge command-name

See MAC-VRF Routing Instance Type Overview for a full list of the MAC-VRF instance show commands and their mappings to the commands that display equivalent results for other instances.

On devices with multiple MAC-VRF EVPN instances, to avoid VTEP scaling issues, we might require or recommend that you enable the shared tunnels feature. On some platforms, shared tunnels are enabled by default. In this example, we enable shared tunnels on the leaf devices using the set forwarding-options evpn-vxlan shared-tunnels configuration statement. MAC-VRF show commands display shared tunnel VTEP interfaces as vtep-indexshared-tunnel-unit, where:

  • index is the index associated with the MAC-VRF routing instance.

  • shared-tunnel-unit is the unit number associated with the shared tunnel remote VTEP logical interface.

For example:

Note:

After you configure the shared-tunnels option, you must reboot the device for the setting to take effect.

Verify Peer Device Connectivity

Purpose

Check that the leaf device established BGP IPv6 connectivity to its peer spine and leaf devices in the fabric.

Action

Run the show bgp summary command on the leaf device:

Meaning

Leaf 3 (IPv6 address 2001:db8::3:1 in Figure 2) sees its eBGP peer devices Spine 1 (2001:db8::1), Spine 2(2001:db8::1:1), Leaf 1(2001:db8::2:1) and Leaf 2 (2001:db8::3:1).

Verify VTEP Source Parameters

Purpose

View the configured IPv6 VTEP source interface(s).

Action

Run the show mac-vrf forwarding vxlan-tunnel-end-point source command:

Meaning

The output shows you configured Leaf 3 with IPv6 VTEP source address 2001:db8::4:1 on the loopback port in MAC-VRF instance USER-MVS1 for VLAN V110, which you mapped to VNI 1110.

Verify Remote VTEPs

Purpose

Verify the device has forwarding information for the remote VTEPs.

Action

Run the show mac-vrf forwarding vxlan-tunnel-end-point remote command:

Meaning

The output shows that Leaf 3 has forwarding information for remote IPv6 VTEPs on Leaf 1 (2001:db8::2:1) and Leaf 2 (2001:db8::3:1).

Verify MAC-VRF EVPN Instance Forwarding

Purpose

View the forwarding table for the configured MAC-VRF instance to see the interfaces for the remote VTEPs associated with the instance.

Action

Run the show mac-vrf forwarding mac-table instance name command for the MAC-VRF instance in this example, USER_MVS1:

Meaning

The output for this command shows the MAC addresses that were populated in the MAC table.