Example: Setting Up Inter-VXLAN Unicast Routing and OVSDB Connections in a Data Center

 

This example shows how to set up a data center in which virtual machines (VMs) in different Virtual Extensible LANs (VXLANs) need to communicate. The Juniper Networks device that is integrated into this environment functions as a hardware virtual tunnel endpoint (VTEP) that can route VM traffic from one VXLAN (Layer 2) environment to another.

The Juniper Networks device implements the Open vSwitch Database (OVSDB) management protocol and has a connection with a VMware NSX controller, both of which enable the device and the NSX controller to exchange MAC routes to and from VMs in the physical and virtual networks.

This example explains how to configure a Juniper Networks device as a hardware VTEP, set up the routing of unicast packets between VXLANs, and set up an OVSDB connection with an NSX controller. For information about setting up the routing of unicast and multicast packets between VXLANs, see Example: Setting Up Inter-VXLAN Unicast and Multicast Routing and OVSDB Connections in a Data Center.

Requirements

The topology for this example includes the following hardware and software components:

  • A cluster of five NSX controllers.

  • NSX Manager.

  • A service node that handles broadcast, unknown unicast, and multicast (BUM) traffic within each of the two VXLANs.

  • Two hosts, each of which includes VMs managed by a hypervisor. Each hypervisor includes a software VTEP. The VMs on each of the hosts belong to different VXLANs.

  • A Juniper Networks device that routes VM traffic between the two VXLANs. For example, an MX Series router running Junos OS Release 14.1R2 or later, or an EX9200 switch running Junos OS release 14.2 or later. The Juniper Networks device must also run an OVSDB software package, and the release of this package must be the same as the Junos OS release running on the device. This device is configured to function as a hardware VTEP.

Before you begin the configuration of the Juniper Networks device, you need to perform the following tasks:

  • In NSX Manager or the NSX API, specify the IP address of the service node.

  • In NSX Manager or the NSX API, configure a logical switch for each VXLAN that OVSDB will manage. This example implements two OVSDB-managed VXLANs; therefore, you must configure two logical switches. After the configuration of each logical switch, NSX automatically generates a universally unique identifier (UUID) for the logical switch. If you have not already, retrieve the UUID for each logical switch. A sample UUID is 28805c1d-0122-495d-85df-19abd647d772. When configuring the equivalent VXLANs on the Juniper Networks device, you must use the UUID of the logical switch as the bridge domain or VLAN name.

    For more information about logical switches and VXLANs, see Understanding How to Manually Configure OVSDB-Managed VXLANs.

  • Create an SSL private key and certificate, and install them in the /var/db/certs directory of the Juniper Networks device. For more information, see Creating and Installing an SSL Key and Certificate on a Juniper Networks Device for Connection with SDN Controllers.

For information about using NSX Manager or the NSX API to perform these configuration tasks, see the documentation that accompanies the respective products.

Overview and Topology

In the topology shown in Figure 1, VM 1 in VXLAN 1 needs to communicate with VM 3 in VXLAN 2. To enable this communication, hardware VTEP 1, which can be an MX Series router or an EX9200 switch, is configured to route VM unicast traffic between the two VXLANs.

Figure 1: Inter-VXLAN Unicast Routing and OVSDB Topology
Inter-VXLAN
Unicast Routing and OVSDB Topology

On hardware VTEP 1, a routing instance (virtual switch) is set up. Within the routing instance, two VXLANs are configured: VXLAN 1 and VXLAN 2. Each VXLAN has an integrated routing and bridging (IRB) interface associated with it. The IRB interfaces handle the routing of VM unicast traffic between the VXLANs,

Within each of the two VXLANs, a service node replicates Layer 2 BUM packets then forwards the replicas to all interfaces in the VXLANs. Having the service node handle the Layer 2 BUM traffic is the default behavior, and no configuration is required for this Juniper Networks device.

On hardware VTEP 1, a connection with an NSX controller is configured on the management interface (fxp0 for an MX Series router and me0 for an EX9200 switch). This configuration enables the NSX controller to push MAC routes for VM 1 and VM 3 to the hardware VTEP by way of the table for remote unicast MAC addresses in the OVSDB schema for physical devices.

Each VXLAN-encapsulated packet must include a source IP address, which identifies the source hardware or software VTEP, in the outer IP header. In this example, for hardware VTEP 1, the IP address of the loopback interface (lo0.0) is used.

In this example, the tracing of all OVSDB events is configured. The output of the OVSDB events are placed in a file named ovsdb, which is stored in the /var/log directory. By default, a maximum of 10 trace files can exist, and the configured maximum size of each file is 50 MB.

Table 1 describes the components for setting up inter-VXLAN routing and an OVSDB connection.

Table 1: Components for Setting Up Inter-VXLAN Routing and OVSDB Connections in a Data Center

Property

Settings

Routing instance

Name: vx1

Type: virtual switch

OVSDB-managed VXLANs included: VXLAN 1 and VXLAN 2

VXLAN 1

Bridge domain or VLAN associated with: 28805c1d-0122-495d-85df-19abd647d772

Interface: xe-0/0/2.0

VLAN ID: 100

VNI: 100

VXLAN 2

Bridge domain or VLAN associated with: 96a382cd-a570-4ac8-a77a-8bb8b16bde70

Interface: xe-1/2/0.0

VLAN ID: 200

VNI: 200

Inter-VXLAN unicast routing and forwarding with IRB interfaces

VXLAN 1: irb.0; 10.20.20.1/24; associated with routing interface vx1, and bridge domain or VLAN 28805c1d-0122-495d-85df-19abd647d772

VXLAN 2: irb.1; 10.10.10.3/24; associated with routing interface vx1, and bridge domain or VLAN 96a382cd-a570-4ac8-a77a-8bb8b16bde70

Handling of BUM traffic in each VXLAN

Service node

Note: By default, one or more service nodes handle Layer 2 BUM traffic in a VXLAN; therefore, no configuration is required.

NSX controller

IP address: 10.94.184.1

Hardware VTEP source identifier

Source interface: loopback (lo0.0)

Source IP address: 10.19.19.19/32

OVSDB tracing operations

Filename: /var/log/ovsdb

File size: 50 MB

Flag: All

Configuration

An MX Series router or an EX9200 switch can function as hardware VTEP 1 in this example. Because the configuration for each device is slightly different, a separate configuration is provided for each device.

To configure inter-VXLAN unicast routing and OVSDB connections in a data center topology, you need to perform one of these tasks:

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your configuration (for example, IP addresses, interface names, and UUIDs), copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

Note

After completing this configuration, you must configure a gateway, which is the NSX-equivalent of a hardware VTEP. This example implements one hardware VTEP; therefore, you must configure one gateway, a gateway service, and a logical switch port using NSX Manager or the NSX API. For more information about the tasks you must perform and key NSX Manager configuration details, see VMware NSX Configuration for Juniper Networks Devices Functioning as Virtual Tunnel Endpoints.

MX Series router configuration:

EX9200 switch configuration:

Configuring an MX Series Router as a Hardware VTEP with an OVSDB Connection

Step-by-Step Procedure

To configure an MX Series router as hardware VTEP 1 with an OVSDB connection to an NSX controller, follow these steps:

  1. Create the Layer 3 network.

  2. Create an access interface for VXLAN 1, and associate the interface with the VXLAN.

  3. Create an access interface for VXLAN 2, and associate the interface with the VXLAN.

  4. Create an IRB interface to handle inter-VXLAN unicast traffic for VXLAN 1.

  5. Create an IRB interface to handle inter-VXLAN unicast traffic for VXLAN 2.

  6. Set up the virtual switch routing instance.

  7. Specify an IP address for the loopback interface. This IP address serves as the source IP address in the outer header of any VXLAN-encapsulated packets.

  8. Set up OVSDB tracing operations.

  9. Configure a connection with an NSX controller.

  10. Configure interfaces xe-0/0/2.0 and xe-1/2/0.0 to be managed by OVSDB.

    Note

    After completing this configuration, you must configure a gateway, which is the NSX-equivalent of a hardware VTEP. This example implements one hardware VTEP; therefore, you must configure one gateway, a gateway service, and a logical switch port by using NSX Manager or the NSX API. For more information about the tasks you must perform and key NSX Manager configuration details, see VMware NSX Configuration for Juniper Networks Devices Functioning as Virtual Tunnel Endpoints.

Configuring an EX9200 Switch as a Hardware VTEP with an OVSDB Connection

Step-by-Step Procedure

To configure an EX9200 switch as hardware VTEP 1 with an OVSDB connection to an NSX controller, follow these steps:

  1. Create the Layer 3 network.

  2. Create an access interface for VXLAN 1, and associate the interface with the VXLAN.

  3. Create an access interface for VXLAN 2, and associate the interface with the VXLAN.

  4. Create an IRB interface to handle inter-VXLAN unicast traffic for VXLAN 1.

  5. Create an IRB interface to handle inter-VXLAN unicast traffic for VXLAN 2.

  6. Set up the virtual switch routing instance.

  7. Specify an IP address for the loopback interface. This IP address serves as the source IP address in the outer header of any VXLAN-encapsulated packets.

  8. Set up tracing operations to be performed for the OVSDB management protocol.

  9. Configure a connection with an NSX controller.

  10. Configure interfaces xe-0/0/2.0 and xe-1/2/0.0 to be managed by OVSDB.

    Note

    After completing this configuration, you must configure a gateway, which is the NSX-equivalent of a hardware VTEP. This example implements one hardware VTEP; therefore, you must configure one gateway, a gateway service, and a logical switch port by using NSX Manager or the NSX API. For more information about the tasks you must perform and key NSX Manager configuration details, see VMware NSX Configuration for Juniper Networks Devices Functioning as Virtual Tunnel Endpoints.

Verification

Verifying the Logical Switches

Purpose

Verify that logical switches with the UUIDs of 28805c1d-0122-495d-85df-19abd647d772 and 96a382cd-a570-4ac8-a77a-8bb8b16bde70 are configured in NSX Manager or in the NSX API, and that information about the logical switches is published in the OVSDB schema.

Action

Issue the show ovsdb logical-switch operational mode command.

user@host> show ovsdb logical-switch

Meaning

The output verifies that information about the logical switches is published in the OVSDB schema. The Created by both state indicates that the logical switches are configured in NSX Manager or the NSX API, and the corresponding VXLANs are configured on the Juniper Networks device. In this state, the logical switches and VXLANs are operational.

If the state of the logical switches is something other than Created by both, see Troubleshooting a Nonoperational Logical Switch and Corresponding Junos OS OVSDB-Managed VXLAN.

Verifying the MAC Addresses of VM 1 and VM 3

Purpose

Verify that the MAC addresses of VM 1 and VM 3 are present in the OVSDB schema.

Action

Issue the show ovsdb mac remote operational mode command to verify that the MAC addresses for VM 1 and VM 3 are present.

user@host> show ovsdb mac remote

Meaning

The output shows that the MAC addresses for VM 1 and VM 3 are present and are associated with logical switches with the UUIDs of 28805c1d-0122-495d-85df-19abd647d772 and 96a382cd-a570-4ac8-a77a-8bb8b16bde70, respectively. Given that the MAC addresses are present, VM 1 and VM 3 are reachable through hardware VTEP 1.

Verifying the NSX Controller Connection

Purpose

Verify that the connection with the NSX controller is up.

Action

Issue the show ovsdb controller operational mode command, and verify that the controller connection state is up.

user@host> show ovsdb controller

Meaning

The output shows that the connection state of the NSX controller is up, in addition to other information about the controller. When this connection is up, OVSDB is enabled on the Juniper Networks device.