Examples: Manually Configuring VXLANs on QFX Series and EX4600 Switches
The following examples show use cases for manually configuring VXLANs on QFX5100, QFX5110, QFX5200, QFX5210, and EX4600 switches.
Example: Configuring a VXLAN Transit Switch
If a QFX Series or EX4600 switch acts as a transit switch for downstream devices acting as VTEPs, you do not need to configure any VXLAN information on the QFX Series or EX4600 switch. You do need to configure PIM on the switch so that it can form the multicast tree required so that the VTEPs can establish reachablity with each other.
Requirements
This example uses the following hardware and software components:
Two QFX5100 switches
Junos OS Release 14.1X53-D10
Overview
This example shows a simple use case in which QFX5100 switches are connected to downstream servers acting as VTEPs. The QFX5100 switches need to forward VXLAN packets between VM 1 on Server 1 and VM 2 on Server 2. Because this configuration allows Layer 2 connectivity between the VMs through the VXLAN tunnels, applications can VMotion between the VMs.
Topology
Figure 1 shows QFX 5100 switches configured to forward VXLAN packets for downstream VTEPs.
Configuring PIM on the Transit Switches
CLI Quick Configuration
To quickly configure this example, copy the
following commands, paste them into a text file, remove any line breaks,
change any details necessary to match your network configuration,
and then copy and paste the commands into the CLI at the [edit] hierarchy level.
set protocols pim interface all set protocols pim rp static address ip-address
Procedure
Step-by-Step Procedure
If you are not using an SDN controller to create a VXLAN control plane, you must enable PIM on each switch so that the VTEP can use multicast groups to advertise its existence and to learn about other VTEPs. (Configuring PIM automatically enables IGMP.) You do not need to perform any VXLAN-specific configuration. Note that you also do not need to configure VLAN 1 on either switch.
Enable PIM.
[edit] user@switch# set protocols pim interface all
Configure the address of a PIM rendezvous point.
[edit] user@switch# set protocols pim rp static address ip-address
Example: Configuring a VXLAN Layer 2 Gateway
If a QFX Series or EX4600 switch is connected to a downstream server that hosts a VM that needs Layer 2 connectivity with another VM that is reachable only through a Layer 3 network, you must do the following:
Configure the switch to act as a VTEP—that is, a Layer 2 gateway for downstream Layer 2 devices.
Configure PIM on the switch so that it can form the multicast tree required for reachability with other VTEPs and to allow BUM traffic to be forwarded between the VTEPs.
Enable a routing protocol, for example, OSPF, on the VTEP’s loopback interface and Layer 3 interfaces.
Requirements
This example uses the following hardware and software components:
Two QFX5100 switches
Junos OS Release 14.1X53-D10
Overview
This example shows a use case in which QFX5100 switches act as VTEPs that allow Layer 2 connectivity between VM 1 on Server 1 and VM 2 on Server 2 so that VMotion can occur between the VMs. The servers in this example can be in the same or different data centers—the only constraint is that there must be Layer 3 connectivity between the QFX5100 switches. This allows your network to be very agile in response to demand for server usage or changes in bandwidth requirements.
Note that because the same VLAN exists in both Layer 2 domains and both switches encapsulate the VLAN traffic into the same VXLAN, you do not need a gateway for the VXLAN traffic in the Layer 3 network. The Layer 3 VXLAN packets are routed normally and no de-encapsulation or re-encapsulation is required.
Topology
Figure 2 shows QFX5100 switches configured to act as VTEPs.
Configuring the Switches
CLI Quick Configuration
To quickly configure the QFX5100-96S 1 in
this example, copy the following commands, paste them into a text
file, remove any line breaks, change any details necessary to match
your network configuration, and then copy and paste the commands into
the CLI at the [edit] hierarchy level.
set interfaces lo0 unit 0 family inet address 10.1.1.1 set switch-options vtep–source-interface lo0.0 set protocols pim interface lo0.0 set protocols ospf area 0.0.0.0 interface lo0.0 set protocols pim interface xe-0/0/0.0 set protocols ospf area 0.0.0.0 interface xe-0/0/0.0 set protocols pim rp static address 10.2.2.2 set vlans VLAN1 vlan-id 100 vxlan vni 100 multicast-group 233.252.0.2 set vlans VLAN1 vxlan encapsulate-inner-vlan set vlans VLAN1 vxlan unreachable-vtep-aging-timer 600 set protocols l2-learning decapsulate-accept-inner-vlan set interfaces xe-0/0/0 unit 0 family inet address 10.2.2.100/24 set interfaces xe-0/0/1 unit 0 family ethernet-switching interface-mode trunk set interfaces xe-0/0/1 unit 0 family ethernet-switching vlan members all set protocols ospf enable set protocols ospf area 0.0.0.0 interface em0.0 disable set protocols ospf area 0.0.0.0 interface all
The configuration for QFX5100-96S 2 is identical except for changes to a few of the addresses:
set interfaces lo0 unit 0 family inet address 10.1.1.2 set switch-options vtep–source-interface lo0.0 set protocols pim interface lo0.0 set protocols ospf area 0.0.0.0 interface lo0.0 set protocols pim interface xe-0/0/0.0 set protocols ospf area 0.0.0.0 interface xe-0/0/0.0 set protocols pim rp static address 10.2.2.2 set vlans VLAN1 vlan-id 100 vxlan vni 100 multicast-group 233.252.0.2 set vlans VLAN1 vxlan encapsulate-inner-vlan set vlans VLAN1 vxlan unreachable-vtep-aging-timer 600 set protocols l2-learning decapsulate-accept-inner-vlan set interfaces xe-0/0/0 unit 0 family inet address 10.2.2.200/24 set interfaces xe-0/0/1 unit 0 family ethernet-switching interface-mode trunk set interfaces xe-0/0/1 unit 0 family ethernet-switching vlan members all set protocols ospf enable set protocols ospf area 0.0.0.0 interface em0.0 disable set protocols ospf area 0.0.0.0 interface all
You must configure the same multicast group address for VLAN1 on both switches.
Procedure
Step-by-Step Procedure
Perform the following procedure on both switches to set up the example configuration.
Create a reachable IPv4 address on the loopback interface.
[edit] user@switch# set interfaces lo0 unit 0 family inet address 10.1.1.1
For switch QFX5100-96S 2, use address
10.1.1.2.Configure the loopback interface—and therefore, its associated address—to be used as the tunnel source address.
[edit] user@switch# set switch-options vtep-source-interface lo0.0
Enable PIM on the loopback interface.
[edit] user@switch# set protocols pim interface lo0.0
Add the loopback interface to OSPF area 0.0.0.0.
[edit] user@switch# set protocols ospf area 0.0.0.0 interface lo0.0
Enable PIM on the interface that connects to the Layer 3 network.
[edit] user@switch# set protocols pim interface xe-0/0/0.0
Add the interface that connects to the Layer 3 network to OSPF area 0.0.0.0.
[edit] user@switch# set protocols ospf area 0.0.0.0 interface xe-0/0/0.0
Configure the address of a PIM rendezvous point.
[edit] user@switch# set protocols pim rp static address 10.2.2.2
Create a VLAN, map it to a VXLAN, and assign a multicast group address to the VXLAN. All members of a VXLAN must use the same multicast group address.
[edit] user@switch# set vlans VLAN1 vlan-id 100 vxlan vni 100 multicast-group 233.252.0.2
In this example, the
vlan-idandvniare both set to100. This is done only for simplicity and clarity. You do not need to set thevlan-idandvnito the same value.(Optional) Configure the switch to retain the original VLAN tag (in the inner Ethernet packet) after VXLAN encapsulation. By default, the original tag is dropped when the packet is encapsulated.
[edit] user@switch# set vlans VLAN1 vxlan encapsulate-inner-vlan
(Optional) Configure the system to age out the address for the remote VTEP (the other QFX5100 switch) if all the MAC addresses learned from that VTEP age out. The address for the remote VTEP expires the configured number of seconds after the last learned MAC address expires.
[edit] user@switch# set vlans VLAN1 vxlan unreachable-vtep-aging-timer 600
(Optional) Configure the switch to de-encapsulate and accept original VLAN tags in VXLAN packets. By default, a preserved VLAN tag is dropped when the packet is de-encapsulated.
[edit] user@switch# set protocols l2-learning decapsulate-accept-inner-vlan
Configure the interface that connects to the Layer 3 network.
[edit] user@switch# set interfaces xe-0/0/0 unit 0 family inet address 10.2.2.100/24
For switch QFX5100-96S 2, use address
10.2.2.200.Configure the server-facing interface to support multiple VLANs.
[edit] user@switch# set interfaces xe-0/0/1 unit 0 family ethernet-switching interface-mode trunk [edit] user@switch# set interfaces xe-0/0/1 unit 0 family ethernet-switching vlan members all
Note:Because this example shows only one VLAN, this step is not required for the example. In a real-world configuration, however, it would be required in order to support multiple VMs connected to multiple VLANs. In this case you would also need to configure additional VLAN to VXLAN mappings.
Enable OSPF on all interfaces that are mapped to area 0.0.0.0..
[edit] user@switch# set protocols ospf enable [edit] user@switch# set protocols ospf area 0.0.0.0 interface em0.0 disable [edit] user@switch# set protocols ospf area 0.0.0.0 interface all
Results
From configuration mode, confirm your configuration by entering the following commands on QFX5100-96S 1. If the output does not display the intended configuration, repeat the instructions in this example to correct the configuration.
user@switch# show switch-options
vtep-source-interface lo0.0;
user@switch# show vlans
VLAN1 {
vlan-id 100;
vxlan {
vni 100;
multicast-group 233.252.0.2;
encapsulate-inner-vlan;
}
}
user@switch# show interfaces
xe-0/0/0 {
unit 0 {
family inet {
address 10.2.2.100/24;
}
}
}
xe-0/0/1 {
unit 0 {
family ethernet-switching {
interface-mode trunk;
vlan {
members all;
}
}
}
}
lo0 {
unit 0 {
family inet {
address 10.1.1.1/32;
}
}
}
user@switch# show protocols pim
rp {
static {
address 10.2.2.2;
}
}
interface xe-0/0/0.0
Verification
Confirm that the configuration is working properly.
- Verifying VXLAN Reachability
- Verifying That the Local VTEP Is Configured Correctly
- Verifying MAC Learning from the Remote VTEP
- Monitor the Remote Interface
- Verifying OSPF Neighbor
Verifying VXLAN Reachability
Purpose
On QFX5100-96S 1, verify that there is connectivity with the remote VTEP (QFX5100-96S 2).
Action
user@switch> show ethernet-switching vxlan-tunnel-end-point remote Logical System Name Id SVTEP-IP IFL L3-Idx <default> 0 10.1.1.2 lo0.0 0 RVTEP-IP IFL-Idx NH-Id 10.1.1.2 559 1728 VNID MC-Group-IP 100 233.252.0.2
Meaning
The VTEP on QFX5100-96S 2 is reachable because its IP address (the address assigned to the loopback interface) appears in the output. The output also shows that the VXLAN (VNI 100) and corresponding multicast group are configured correctly on the remote VTEP.
Verifying That the Local VTEP Is Configured Correctly
Purpose
On QFX5100-96S 1, verify that the tunnel endpoint is correct.
Action
user@switch> show ethernet-switching vxlan-tunnel-end-point source Logical System Name Id SVTEP-IP IFL L3-Idx <default> 0 10.1.1.1 lo0.0 0 L2-RTT Bridge Domain VNID MC-Group-IP default-switch VLAN1+100 100 233.252.0.2
Meaning
The VTEP on QFX5100-96S 1 shows the correct tunnel source IP address (assigned to the loopback interface), VLAN, and multicast group for the VXLAN.
Verifying MAC Learning from the Remote VTEP
Purpose
On QFX5100-96S 1, verify that it is learning MAC addresses from the remote VTEP.
Action
user@switch> show ethernet-switching table
MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static
SE - statistics enabled, NM - non configured MAC, R - remote PE MAC)
Ethernet switching table : 2 entries, 2 learned
Routing instance : default-switch
Vlan MAC MAC Age Logical
name address flags interface
VLAN1 00:00:00:ff:ff:ff D - vtep.12345
VLAN1 00:10:94:00:00:02 D - xe-0/0/0.0
Meaning
The output shows the MAC addresses learned from the
remote VTEP (in addition to those learned on the normal Layer 2 interfaces).
It also shows the logical name of the remote VTEP interface (vtep.12345 in the above output).
Monitor the Remote Interface
Purpose
On QFX5100-96S 1, monitor traffic details for the remote VTEP interface.
Action
user@switch> show interface vtep.12345 detail
M Flags: Up SNMP-Traps Encapsulation: ENET2
VXLAN Endpoint Type: Remote, VXLAN Endpoint Address: 10.1.1.2, L2 Routing Instance: default-switch, L3 Routing Instance: default
Traffic statistics:
Input bytes : 228851738624
Output bytes : 0
Input packets: 714162415
Output packets: 0
Local statistics:
Input bytes : 0
Output bytes : 0
Input packets: 0
Output packets: 0
Transit statistics:
Input bytes : 228851738624 0 bps
Output bytes : 0 0 bps
Input packets: 714162415 0 pps
Output packets: 0 0 pps
Protocol eth-switch, MTU: 1600, Generation: 277, Route table: 5
Meaning
The output shows traffic details for the remote VTEP
interface. To get this information, you must supply the logical name
of the remote VTEP interface (vtep.12345 in the above output), which
you can learn by using the show ethernet-switching table command.
Verifying OSPF Neighbor
Purpose
On QFX5100-96S 1, verify that the remote VTEP (QFX5100-96S 2) has established itself as an OSPF neighbor.
Action
user@switch> show ospf neighbor Address Interface State ID Pri Dead 10.2.2.200 xe-0/0/0.0 Full 10.2.3.2 128 38
Meaning
The output shows that interface xe-0/0/0.0 on QFX5100-96S
2 is in the Full state, which means
that QFX5100-96S 2 is a fully adjacent neighbor. The Full state also means that Layer 3 connectivity exists
between QFX5100-96S 1 and QFX5100-96S 2.