Navigation
Guide That Contains This Content
[+] Expand All
[-] Collapse All

    Example: Configuring EVPN-VXLAN In a Collapsed IP Fabric Topology Within a Data Center

    Ethernet VPN (EVPN) is a control plane technology that enables hosts (physical servers and virtual machines [VMs]) to be placed anywhere in a network and remain connected to the same logical Layer 2 overlay network. Virtual Extensible LAN (VXLAN) is a tunneling protocol that creates the data plane for the Layer 2 overlay network.

    The physical underlay network over which EVPN-VXLAN is commonly deployed is a two-layer IP fabric, which includes spine and leaf devices as shown in Figure 1. The spine devices—for example, QFX10000 switches—provide connectivity between the leaf devices, and the leaf devices—for example, QFX5100 switches—provide connectivity to attached hosts. In the overlay network, the leaf devices function as Layer 2 gateways that handle traffic within a VXLAN, and the spine devices function as Layer 3 gateways that handle traffic between VXLANs through the use of integrated routing and bridging (IRB) interfaces. For more information about configuring EVPN-VXLAN in a two-layer IP fabric, see Example: Configuring IRB Interfaces in an EVPN-VXLAN Environment to Provide Layer 3 Connectivity for Hosts in a Data Center.

    Figure 1: Two-Layer IP Fabric

    Two-Layer IP Fabric

    You can also deploy EVPN-VXLAN over a physical underlay network in which the IP fabric is collapsed into a single layer of QFX10000 switches that function as leaf devices. In this collapsed fabric, which is shown in Figure 2, the leaf devices serve as both Layer 2 and Layer 3 gateways. In this topology, transit spine devices provide Layer 3 routing functionality only.

    Figure 2: Collapsed IP Fabric

    Collapsed IP Fabric

    This example describes how to configure EVPN-VXLAN, in particular, the Layer 3 gateway, on a leaf device in a collapsed IP fabric topology.

    Requirements

    This example uses the following hardware and software components:

    • Two routers that function as transit spine devices.
    • Three QFX10000 switches running Junos OS Release 15.1X53-D60 or later software. These switches are leaf devices that provide both Layer 2 and Layer 3 gateway functionality.

      Note: This example focuses on the configuration of the Layer 2 overlay network on a leaf device. The transit spine devices used in this example provide Layer 3 functionality only. As a result, this example does not include the configuration of these devices.

      Further, this example provides the configuration for leaf 1 only. The configuration for leaf 1 essentially serves as a template for the configuration of the other leaf devices. For the configuration of the other leaf devices, where appropriate, you can replace leaf 1-specific information with the information specific to the device you are configuring, add additional commands, and so on.

    • Two physical (bare-metal) servers and one server with VMs that are supported by a hypervisor.

    Overview and Topology

    The collapsed IP fabric topology shown in Figure 3 includes two transit spine devices, a collapsed IP fabric, which includes three leaf devices that function as both Layer 2 and Layer 3 gateways, two physical servers, and one virtualized server on which VMs and a hypervisor are installed. Physical server 1 is connected to leaf 1 and leaf 2 through a link aggregation group (LAG) interface. On both leaf devices, the interface is assigned the same Ethernet segment identifier (ESI) and set to multihoming active-active mode.

    All leaf devices are in the same autonomous system (65200).

    Figure 3: Collapsed IP Fabric Topology Within a Data Center

    Collapsed IP Fabric Topology
Within a Data Center

    In this topology, an application on physical server 1 needs to communicate with VM 1 on the virtualized server. Physical servers 1 and 2 are included in VLAN 1, and the virtualized server is included in VLAN 2. For communication between VLANs 1 and 2 to occur, two IRB interfaces—irb.1, which is associated with VLAN 1, and irb.2, which is associated with VLAN 2—must be configured on each leaf device.

    The most significant difference between the configuration of an EVPN-VXLAN overlay network deployed over a collapsed IP fabric and the configuration of an overlay network deployed over a two-layer IP fabric is the configuration of the Layer 3 gateway. Therefore, this example focuses on the EVPN-VXLAN configuration, in particular, the Layer 3 gateway configuration on the leaf devices.

    For the collapsed IP fabric topology, you can configure the IRB interfaces within an EVPN instance using one of the following methods:

    • Method 1—For each IRB interface on a particular leaf device, for example, leaf 1, the following is specified:
      • A unique IP address.
      • The same MAC address.

      For example:

      irb.1

      IP address: 10.1.1.1/24

      MAC address: 00:00:5e:00:53:01

      irb.2

      IP address: 10.1.2.1/24

      MAC address: 00:00:5e:00:53:01

    • Method 2—For each IRB interface on leaf 1, the following is specified:
      • A unique IP address.
      • A unique MAC address.

      For example:

      irb.1

      IP address: 10.1.1.1/24

      MAC address: 00:00:5e:00:53:aa

      irb.2

      IP address: 10.1.2.1/24

      MAC address: 00:00:5e:00:53:bb

    Regardless of the method that you use to configure the IRB interfaces on leaf 1, if irb.1 and irb.2 are also configured on other leafs in the collapsed IP fabric, for example, leafs 2 and 3, you must specify the same configurations that you specified on leaf 1 for those IRB interfaces on leafs 2 and 3. For example, Figure 4 shows the configurations for irb.1 and irb.2 on leafs 1, 2, and 3 for both methods.

    Figure 4: Method 1 and 2 IRB Interface Configurations on Multiple Leaf Devices

    Method 1 and
2 IRB Interface Configurations on Multiple Leaf Devices

    Note: In this example, method 1 is used to configure the IRB interfaces.

    As shown in this example, with the same MAC address configured for each IRB interface on each leaf device, each host uses the same MAC address when sending inter-VLAN traffic regardless of where the host is located or which leaf device receives the traffic. For example, in the topology shown in Figure 3, multi-homed physical server 1 in VLAN 1 sends a packet to VM 1 in VLAN 2. If leaf 1 is down, leaf 2 continues to forward the inter-VLAN traffic even without the configuration of a redundant default gateway MAC address.

    Note that the configuration of IRB interfaces used in this example does not include a virtual gateway address (VGA) and a corresponding MAC address that establishes redundant default gateway functionality, which is mentioned above. By configuring the same MAC address for each IRB interface on each leaf device, hosts use the local leaf device configured with the common MAC address as the default Layer 3 gateway. Therefore, you eliminate the need to advertise a redundant default gateway and dynamically synchronize the MAC addresses of the redundant default gateway throughout the EVPN control plane. As a result, when configuring each leaf device, you must disable the advertisement of the redundant default gateway by including the default-gateway do-not-advertise configuration statement in the [edit protocols evpn] hierarchy level in your configuration.

    Note: Although the IRB interface configuration used in this example does not include a VGA, you can configure it as needed to make EVPN-VXLAN work properly in your collapsed IP fabric topology. If you configure a VGA for each IRB interface, you specify the same IP address for each VGA on each leaf device instead of configuring the same MAC address for each IRB interface on each leaf device as is shown in this example.

    When it comes to handling the replication of broadcast, unknown unicast, and multicast (BUM) traffic, note that the configuration on leaf 1:

    • includes the set protocols evpn multicast-mode ingress-replication command. This command causes leaf 1, which is a hardware VTEP, to handle the replication and sending of BUM traffic instead of a multicast client in the EVPN-VXLAN topology.
    • Retains the QFX10000 switch’s default setting of disabled for ingress node replication for EVPN-VXLAN. With this feature disabled, if a QFX10000 switch that functions as a VTEP receives a BUM packet intended, for example, for a physical server in a VLAN with the VNI of 1001, the VTEP replicates and sends the packet only to VTEPs on which the VNI of 1001 is configured. If this feature is enabled, the VTEP replicates and sends this packet to all VTEPs in its database, including those that do not have VNI 1001 configured. To prevent a VTEP from needlessly flooding BUM traffic throughout an EVPN-VXLAN overlay network, we strongly recommend that if not already disabled, you disable ingress node replication on each of the leaf devices by specifying the delete vlans vlan-name vxlan ingress-node-replication command.

    Configuration

    CLI Quick Configuration

    To quickly configure this example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, and then copy and paste the commands into the CLI at the [edit] hierarchy level.

    Leaf 1

    set interfaces et-0/0/53 ether-options 802.3ad ae202
    set interfaces ae202 esi 00:11:22:33:44:55:66:77:88:99
    set interfaces ae202 esi all-active
    set interfaces ae202 aggregated-ether-options lacp active
    set interfaces ae202 aggregated-ether-options lacp system-id 00:00:00:04:04:04
    set interfaces ae202 unit 0 family ethernet-switching interface-mode trunk
    set interfaces ae202 unit 0 family ethernet-switching vlan members 1-2
    set interfaces irb unit 1 family inet address 10.1.1.1/24
    set interfaces irb unit 1 mac 00:00:5e:00:53:01
    set interfaces irb unit 2 family inet address 10.1.2.1/24
    set interfaces irb unit 2 mac 00:00:5e:00:53:01
    set interfaces lo0 unit 0 family inet address 192.168.0.11/32
    set interfaces lo0 unit 1 family inet address 192.168.10.11/32
    set protocols bgp group overlay-evpn type internal
    set protocols bgp group overlay-evpn local-address 192.168.0.11
    set protocols bgp group overlay-evpn family evpn signaling
    set protocols bgp group overlay-evpn local-as 65200
    set protocols bgp group overlay-evpn multipath
    set protocols bgp group overlay-evpn neighbor 192.168.0.22
    set protocols bgp group overlay-evpn neighbor 192.168.0.33
    set protocols evpn encapsulation vxlan
    set protocols evpn extended-vni-list 1001
    set protocols evpn extended- vni-list 1002
    set protocols evpn multicast-mode ingress-replication
    set protocols evpn default-gateway do-not-advertise
    set protocols evpn vni-options vni 1001 vrf-target export target:1:1001
    set protocols evpn vni-options vni 1002 vrf-target export target:1:1002
    set policy-options community comm-leaf_esi members target 9999:9999
    set policy-options community com1001 members target:1:1001
    set policy-options community com1002 members target:1:1002
    set policy-options policy-statement LEAF-IN term import_leaf_esi from community comm-leaf_esi
    set policy-options policy-statement LEAF-IN term import_leaf_esi then accept
    set policy-options policy-statement vrf-1-to-200 term import_vni1001 from community com1001
    set policy-options policy-statement vrf-1-to-200 term import_vni1001 then accept
    set policy-options policy-statement vrf-1-to-200 term import_vni1002 from community com1002
    set policy-options policy-statement vrf-1-to-200 term import_vni1002 then accept
    set routing-instances VRF_1 instance-type vrf
    set routing-instances VRF_1 interface irb.1
    set routing-instances VRF_1 interface irb.2
    set routing-instances VRF_1 interface lo0.1
    set routing-instances VRF_1 route-distinguisher 192.168.0.11:1
    set routing-instances VRF_1 vrf-target target:1:1
    set switch-options vtep-source-interface lo0.0
    set switch-options route-distinguisher 192.168.0.11:5000
    set switch-options vrf-import LEAF-IN
    set switch-options vrf-import vrf-1-to-200
    set switch-options vrf-target target:9999:9999
    set vlans bd1 vlan-id 1
    set vlans bd1 l3-interface irb.1
    set vlans bd1 vxlan vni 1001
    delete vlans bd1 vxlan ingress-node-replication
    set vlans bd2 vlan-id 2
    set vlans bd2 l3-interface irb.2
    set vlans bd2 vxlan vni 1002
    delete vlans bd2 vxlan ingress-node-replication

    Configuring EVPN-VXLAN on Leaf 1

    Step-by-Step Procedure

    1. Enable physical server 1 to be multihomed to leaf 1 and leaf 2 by configuring an aggregated Ethernet interface, specifying an ESI for the interface, and setting the mode so that the connections to both leaf devices are active.

      Note: When configuring the ae202 interface on leaf 2, you must specify the same ESI (00:11:22:33:44:55:66:77:88:99) that is specified for the same interface on leaf 1.

      [edit]
      user@switch# set interfaces et-0/0/53 ether-options 802.3ad ae202
      user@switch# set interfaces ae202 esi 00:11:22:33:44:55:66:77:88:99
      user@switch# set interfaces ae202 esi all-active
      user@switch# set interfaces ae202 aggregated-ether-options lacp active
      user@switch# set interfaces ae202 aggregated-ether-options lacp system-id 00:00:00:04:04:04
      user@switch# set interfaces ae202 unit 0 family ethernet-switching interface-mode trunk
      user@switch# set interfaces ae202 unit 0 family ethernet-switching vlan members 1-2
    2. Configure two IRB interfaces, each with unique IP addresses and the same MAC address.

      [edit]
      user@switch# set interfaces irb unit 1 family inet address 10.1.1.1/24
      user@switch# set interfaces irb unit 1 mac 00:00:5e:00:53:01
      user@switch# set interfaces irb unit 2 family inet address 10.1.2.1/24
      user@switch# set interfaces irb unit 2 mac 00:00:5e:00:53:01
    3. Configure a loopback interface (lo0.0) for the leaf device and a logical loopback address (lo0.1) for the EVPN routing instance (VRF-1).

      [edit]
      user@switch# set interfaces lo0 unit 0 family inet address 192.168.0.11/32
      user@switch# set interfaces lo0 unit 1 family inet address 192.168.10.11/32
    4. Set up the IBGP overlay network.

      [edit]
      user@switch# set protocols bgp group overlay-evpn type internal
      user@switch# set protocols bgp group overlay-evpn local-address 192.168.0.11
      user@switch# set protocols bgp group overlay-evpn family evpn signaling
      user@switch# set protocols bgp group overlay-evpn local-as 65200
      user@switch# set protocols bgp group overlay-evpn multipath
      user@switch# set protocols bgp group overlay-evpn neighbor 192.168.0.22
      user@switch# set protocols bgp group overlay-evpn neighbor 192.168.0.33
    5. Set up the EVPN-VXLAN domain, which entails determining which VNIs are included in the domain, specifying that leaf 1, which is a hardware VTEP, handles the replication and sending of BUM traffic, disabling the advertisement of the redundant default gateway throughout the EVPN control plane, and specifying a route target for each VNI.

      [edit]
      user@switch# set protocols evpn encapsulation vxlan
      user@switch# set protocols evpn extended-vni-list 1001
      user@switch# set protocols evpn extended- vni-list 1002
      user@switch# set protocols evpn multicast-mode ingress-replication
      user@switch# set protocols evpn default-gateway do-not-advertise
      user@switch# set protocols evpn vni-options vni 1001 vrf-target export target:1:1001
      user@switch# set protocols evpn vni-options vni 1002 vrf-target export target:1:1002
    6. Set up communities for the VNIs, and create policies that import and accept the overlay routes.

      [edit]
      user@switch# set policy-options community comm-leaf_esi members target 9999:9999
      user@switch# set policy-options community com1001 members target:1:1001
      user@switch# set policy-options community com1002 members target:1:1002
      user@switch# set policy-options policy-statement LEAF-IN term import_leaf_esi from community comm-leaf_esi
      user@switch# set policy-options policy-statement LEAF-IN term import_leaf_esi then accept
      user@switch# set policy-options policy-statement vrf-1-to-200 term import_vni1001 from community com1001
      user@switch# set policy-options policy-statement vrf-1-to-200 term import_vni1001 then accept
      user@switch# set policy-options policy-statement vrf-1-to-200 term import_vni1002 from community com1002
      user@switch# set policy-options policy-statement vrf-1-to-200 term import_vni1002 then accept
    7. Set up an EVPN routing instance.

      [edit]
      user@switch# set routing-instances VRF_1 instance-type vrf
      user@switch# set routing-instances VRF_1 interface irb.1
      user@switch# set routing-instances VRF_1 interface irb.2
      user@switch# set routing-instances VRF_1 interface lo0.1
      user@switch# set routing-instances VRF_1 route-distinguisher 192.168.0.11:1
      user@switch# set routing-instances VRF_1 vrf-target target:1:1

      Note: In the above EVPN routing instance configuration, a unique logical loopback interface (lo0.1) is specified, and an IP address for the interface is specified using the set interfaces lo0 unit logical-unit-number family inet address ip-address/prefix command. All items configured in the above routing instance except for the logical loopback interface are required for EVPN. However, the configuration of a logical loopback interface and associated IP address are required to ensure that VXLAN control packets are properly processed.

    8. Configure the switch options to use loopback interface lo0.0 as the source interface of the VTEP, set a route distinguisher, and import the route targets for the three communities into the EVPN (MAC) table.

      [edit]
      user@switch# set switch-options vtep-source-interface lo0.0
      user@switch# set switch-options route-distinguisher 192.168.0.11:5000
      user@switch# set switch-options vrf-import LEAF-IN
      user@switch# set switch-options vrf-import vrf-1-to-200
      user@switch# set switch-options vrf-target target:9999:9999
    9. Configure VLANs to which IRB interfaces and VXLAN VNIs are associated.

      [edit]
      user@switch# set vlans bd1 vlan-id 1
      user@switch# set vlans bd1 l3-interface irb.1
      user@switch# set vlans bd1 vxlan vni 1001
      user@switch# set vlans bd2 vlan-id 2
      user@switch# set vlans bd2 l3-interface irb.2
      user@switch# set vlans bd2 vxlan vni 1002
    10. If not already disabled, disable ingress node replication to prevent leaf 1 from needlessly flooding BUM traffic throughout the EVPN-VXLAN overlay network.

      [edit]
      user@switch# delete vlans bd1 vxlan ingress-node-replication
      user@switch# delete vlans bd2 vxlan ingress-node-replication

    Verification

    The section describes the following verifications for this example:

    Verifying the IRB Interfaces

    Purpose

    Verify that the IRB interfaces are up and running.

    Action

    Display the status of the IRB interfaces:

    user@leaf1> show interfaces irb terse
    Interface               Admin Link Proto    Local                 Remote
    irb                     up    up
    irb.1                   up    up   inet     10.1.1.1/24     
    irb.2                   up    up   inet     10.1.2.1/24     
    

    Meaning

    The IRB interfaces are up and running.

    Verifying the VTEP Interfaces

    Purpose

    Verify the status of the VTEP interfaces.

    Action

    Display the status of the VTEP interfaces:

    user@leaf1> show interfaces vtep terse
    Interface               Admin Link Proto    Local                 Remote
    vtep                    up    up
    vtep.32769              up    up   eth-switch
    vtep.32770              up    up   eth-switch
    vtep.32771              up    up   eth-switch

    Meaning

    The interfaces for each of the VTEPs is up. Therefore, the VTEP interfaces are functioning normally.

    Verifying the EVPN Routing Instance

    Purpose

    Verify the routing table for VRF_1.

    Action

    Verify the routing table for the EVPN routing instance VRF_1.

    user@leaf1> show route table VRF_1.inet.0
    VRF_1.inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
    + = Active Route, - = Last Active, * = Both
    
    10.1.1.0/24        *[Direct/0] 00:07:38
                        > via irb.1
    10.1.1.1/32        *[Local/0] 00:07:38
                          Local via irb.110.1.2.0/24        *[Direct/0] 00:07:38
                        > via irb.2
    10.1.2.1/32        *[Local/0] 00:07:38
                          Local via irb.2
    192.168.10.11/32   *[Direct/0] 00:07:38
                        > via lo0.1
    

    Meaning

    The EVPN routing instance is functioning correctly.

    Modified: 2017-11-09