Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Navigation
Guide That Contains This Content
[+] Expand All
[-] Collapse All

    Setting Up and Using a Simple Virtual Gateway with Contrail

    Introduction to the Simple Gateway

    Every virtual network has a routing instance associated with it. The routing instance defines the network connectivity for the virtual machines in the virtual network. By default, the routing instance contains routes only for virtual machines spawned within the virtual network. Connectivity between virtual networks is controlled by defining network policies.

    The public network is the IP fabric or the external networks across the IP fabric. The virtual networks do not have access to the public network, and a gateway is used to provide connectivity to the public network from a virtual network. In traditional deployments, a routing device such as a Juniper Networks MX Series router can act as a gateway.

    The simple virtual gateway for Contrail is a restricted implementation of a gateway that can be used for experimental purposes. The simple gateway provides the Contrail virtual networks with access to the public network, and is represented as vgw.

    How the Simple Gateway Works

    The following sections illustrate how the simple gateway works, first, by showing a virtual network setup with no simple gateway, then illustrating the same setup with a simple gateway configured.

    Setup Without Simple Gateway

    The following shows a virtual network setup when the simple gateway is not configured.

    • A virtual network, default-domain:admin:net1, is configured with the subnet 192.168.1.0/24.
    • The routing instance default-domain:admin:net1:net1 is associated with the virtual network default-domain:admin:net1.
    • A virtual machine with the IP 192.168.1.253 is spawned in net1.
    • A virtual machine is spawned on compute server 1.
    • An interface, vhost0, is in the host OS of server 1 and is assigned the IP 10.1.1.1/24.
    • The interface vhost0 is added to the vRouter in the routing instance fabric.
    • The simple gateway is not configured.

    Setup With Simple Gateway

    The following diagram shows a virtual network setup with the simple gateway configured for the virtual network default-domain:admin:net1.

    The simple gateway configuration uses a gateway interface (vgw) to provide connectivity between the routing instance Fabric and the default-domain:admin:net1:net1.

    The following shows the packet flows between Fabric and the default-domain:admin:net1:net1.

    In the diagram, routes marked with (*) are added by the simple gateway feature.

    Simple Gateway Configuration Features

    The simple gateway configuration has the following features.

    • The simple gateway is configured for the virtual network default-domain:admin:net1.
      • The gateway interface vgw provides connectivity between the routing instance default-domain:admin:net1:net1 and the fabric. ​
      • An IP address is not configured for the gateway interface vgw.
    • The host OS is configured with the following:
      • Two INET interfaces are added to the host OS: vgw and vhost0
      • The host OS is not aware of the routing instances, so vgw and vhost0 are part of the same routing instance in the host OS.
      • The simple gateway adds the route 192.168.1.0/24, pointing to the vgw interface, and that setup is added to the host OS. This route ensures that any packet destined to the virtual machine is sent to the vrouter on the vgw interface.
    • The vRouter is configured with the following:
      • The routing instance named Fabric is created for the fabric network.
      • The interface vhost0 is added to the routing instance Fabric.
      • The interface eth0, which is connected to the fabric network, is added to the routing instance named Fabric.
      • The simple gateway adds the route 192.168.1.0/24 => vhost0, consequently, packets destined to the virtual network default-domain:admin:net1 are sent to the host OS.
    • The routing instance default-domain:admin:net1:net1 is created for the virtual network default-domain:admin:net1.
      • The interface vgw is added to the routing instance default-domain:admin:net1:net1.
      • The simple gateway adds a default route 0.0.0.0/0 that points to the interface vgw. Packets in the routing instance default-domain:admin:net1:net that hit this route are sent to the host OS on the vgw interface. The host OS routes the packets to the Fabric network over the vhost0 interface.

    Simple Gateway Restrictions

    The following are restrictions of the simple gateway.

    • A single compute node can have the simple gateway configured for multiple virtual networks, however, there cannot be overlapping subnets. The host OS does not support routing instances, therefore, all gateway interfaces in the host OS are in the same routing instance. Consequently, the subnets in the virtual networks must not overlap.
    • Each virtual network can have a single simple gateway interface. ECMP is not supported.

    Packet Flows with the Simple Gateway

    The following sections describe the packet flow process when the simple gateway is configured on a Contrail system.

    First, the packet flow process from the virtual network to the public network is described. Next, the packet flow process from the public network to the virtual network is described.

    Packet Flow Process From the Virtual Network to the Public Network

    The following describes the procedure used to move a packet from the virtual network (net1) to the public network.

    1. A packet with source-ip=192.168.1.253 and destination-ip=10.1.1.253 comes from a virtual machine and is received by the vRouter on interface tap0.
    2. The interface tap0 is in the routing instance of default-domain:admin:net1:net1.
    3. The route lookup for 10.1.1.253 in the routing instance default-domain:admin:net1:net1 finds the default route pointing to the tap interface named vgw.
    4. The vRouter transmits the packet toward vgw and it is received by the networking stack of the host OS.
    5. The host OS performs forwarding based on its routing table and forwards the packet on the vhost0 interface.
    6. Packets transmitted on vhost0 are received by the vRouter.
    7. The vhost0 interface is added to the routing instance Fabric.
    8. The routing table for 10.1.1.253 in the routing instance Fabric indicates that the packet is to be transmitted on the eth0 interface.
    9. The vRouter transmits the packet on the eth0 interface.
    10. The host 10.1.1.253 on Fabric receives the packet.

    Packet Flow Process From the Public Network to the Virtual Network

    The following describes the procedure used to move a packet from the public network to the virtual network (net1).

    1. A packet with source-ip=10.1.1.253 and destination-ip=192.168.1.253 coming from the public network is received on interface eth0.
    2. The interface tap0 is in the routing instance of default-domain:admin:net1:net1.
    3. The vRouter receives the packet from eth0 in the routing instance Fabric.
    4. The route lookup for 192.168.1.253 in Fabric points to the interface vhost0.
    5. The vRouter transmits the packet on vhost0 and it is received by the networking stack of the host OS.
    6. The host OS performs forwarding according to its routing table and forwards the packet on the vgw interface.
    7. The vRouter receives the packet on the vgw interface into the routing instance default-domain:admin:net1:net1.
    8. The route lookup for 192.168.1.253 in the routing instance default-domain:admin:net1:net1 points to the tap0 interface.
    9. The vRouter transmits the packet on the tap0 interface.
    10. The virtual machine receives the packet destined to 192.168.1.253.

    Four Methods for Configuring the Simple Gateway

    There are four different methods that can be used to configure the simple gateway. Each of the methods is described in the following sections.

    Using Fab Provisioning to Configure the Simple Gateway

    You can provision the simple virtual gateway (vgw) during system provisioning with fab commands by enabling the vgw knob in the Contrail testbed.py file. Select some or all of the compute nodes to be configured as vgw by identifying vgw roles in the env.roledefs section, along with other role definitions.

    The following example configuration shows three host nodes (host4, host5, and host6) configured as compute nodes. Two of the compute nodes (host4 and host5) are also configured for vgw.

    In the file section env.vgw, two vgw interfaces (vgw1, vgw2) are configured in host4, and the two interfaces are associated with virtual network public and public1, respectively.

    For each vgw interface, the key ipam-subnets designates the subnets used by each virtual network. If the same vgw interface is configured in a different compute node, it must be associated with the same virtual network ipam-subnets. This is illustrated in the following example, where vgw2 is configured in two compute nodes, host4 and host5. In both host4 and host5, vgw2 is associated with the same ipam-subnets.

    The key gateway-routes is an optional parameter. If gateway-routes is configured, the corresponding vgw will only publish the list of routes identified for gateway routes.

    If the vgw interfaces are defined in env.roledefs, when provisioning the system nodes with the command fab setup_all, the vgw interfaces will be provisioned, along with all of the other nodes.

    Example: Testbed.py Env.roledefs for vgw

    env.roledefs = { 'all': [host1, host2, host3, host4, host5, host6],
    
    'cfgm': [host1, host2, host3],
    
    'openstack': [host2],
    
    'webui': [host3],
    
    'control': [host1, host3],
    
    'compute': [host4, host5, host6],
    
    'vgw': [host4, host5], >>>>>>>>>Add section VGW in one or multiple compute node
    
    'collector': [host1, host3],
    
    'database': [host1],
    
    'build': [host_build],
    
    }
    
    env.vgw = {
    
          host4: {
    
                 'vgw1': {
    
                             'vn':'default-domain:admin:public:public',
    
                             'ipam-subnets': ['10.204.220.128/29', '10.204.220.136/29']
    
                             'gateway-routes': ['8.8.8.0/24', '1.1.1.0/24']
    
                          },
    
                                 'vgw2':{
                              'vn':'default-domain:admin:public1:public1',
    
                              'ipam-subnets': ['10.204.220.144/29']}},
    
          host5: {
    
                 'vgw2':{
    
                              'vn':'default-domain:admin:public1:public1',
    
                              ipam-subnets': ['10.204.220.144/29']
    
                        }
    
                 }
    
                      }
    
    }

    Using the vRouter Configuration File to Configure the Simple Gateway

    Another way to enable a simple gateway is to configure one or more vgw interfaces within the contrail-vrouter-agent.conf file.

    Any changes made in this file for simple gateway configuration are implemented upon the next restart of the vrouter agent. To configure the simple gateway in the contrail-vrouter-agent.conf file, each simple gateway interface uses the following parameters:

    • interface=vgwxx— Simple gateway interface name
    • routing_instance=default-domain:admin:public xx:public xx— Name of the routing instance for which the simple gateway is being configured.
    • ip_block=1.1.1.0/24— List of the subnet addresses allocated for the virtual network. Routes within this subnet are added to both the host OS and routing instance for the fabric instance. Represent multiple subnets in the list by separating each with a space.
    • routes=10.10.10.1/24 11.11.11.1/24— List of subnets in the public network that are reachable from the virtual network. Routes within this subnet are added to the routing instance configured for the vgw interface. Represent multiple subnets in the list by separating each with a space.

    Using Thrift Messages to Dynamically Configure the Simple Gateway

    Another way to configure the simple gateway is to dynamically send create and delete thrift messages to the vrouter agent.

    Starting with Contrail Release 1.10 and greater, the following thrift messages are available:

    • AddVirtualGateway—add a virtual gateway
    • DeleteVirtualGateway —delete a virtual gateway
    • ConnectForVirtualGateway —allows audit of the virtual gateway configuration by stateful clients. Upon a new ConnectForVirtualGateway request, one minute is allowed for the configuration to be redone. Any older virtual gateway configuration remaining after this time is deleted.

    How to Dynamically Create a Virtual Gateway

    To dynamically create a simple virtual gateway, you run a script on the compute node where the virtual gateway will be created.

    When run, the script does the following:

    1. Enables forwarding on the node.
    2. Creates the required interface.
    3. Adds the interface to the vRouter.
    4. Adds required routes to the host OS.
    5. Sends the thrift message AddVirtualGateway to the vRouter agent telling it to create the virtual gateway.

    Example: Dynamically Create a Virtual Gateway

    The following procedure dynamically creates the interface vgw1, with subnets 20.30.40.0/24 and 30.40.50.0/24 in the vrf default-domain:admin:vn1:vn1.

    1. Set the PYTHONPATH to the location of InstanceService.py and types.py, for example:

      export PYTHONPATH=/usr/lib/python2.7/dist-packages/nova_contrail_vif/gen_py/instance_service

      export PYTHONPATH=/usr/lib/python2.6/site-packages/contrail_vrouter_api/gen_py/instance_service

    2. Run the vgw provision command with the oper create option.

      Use the option subnets to specify the subnets defined for virtual network vn1.

      Use the option routes to specify the routes in the public network that are injected into vn1.

      In the following example, the virtual machines in vn1 can access subnets 8.8.8.0/24 and 9.9.9.0/24 in the public network:

      python /opt/contrail/utils/provision_vgw_interface.py --oper create --interface vgw1 --subnets 20.30.40.0/24 30.40.50.0/24 --routes 8.8.8.0/24 9.9.9.0/24 --vrf default-domain:admin:vn1:vn1

    How to Dynamically Delete a Virtual Gateway

    To dynamically delete a virtual gateway, you run a script on the compute node where the virtual gateway was created.

    When run, the script does the following:

    1. Sends the DeleteVirtualGateway thrift message to the vRouter agent, telling it to delete the virtual gateway.
    2. Deletes the vgw interface from the vRouter.
    3. Deletes the vgw routes that were added in the host OS when the vgw was created.

    Example: Dynamically Create a Virtual Gateway

    The following procedure dynamically deletes the interface vgw1, and also deletes the subnets 20.30.40.0/24 and 30.40.50.0/24 in the vrf default-domain:admin:vn1:vn1.

    1. Set the PYTHONPATH to the location of InstanceService.py and types.py, for example:

      export PYTHONPATH=/usr/lib/python2.7/dist-packages/nova_contrail_vif/gen_py/instance_service

      export PYTHONPATH=/usr/lib/python2.6/site-packages/contrail_vrouter_api/gen_py/instance_service

    2. Run the vgw provision command with the oper delete option.

      python /opt/contrail/utils/provision_vgw_interface.py --oper delete --interface vgw1 --subnets 20.30.40.0/24 30.40.50.0/24 --routes 8.8.8.0/24 9.9.9.0/24

    3. (optional) If using a stateful client, send the ConnectForVirtualGateway thrift message to the vRouter agent when the client starts.

    Note: If the the vRouter agent restarts or if the compute node reboots, it is expected that the client will reconfigure again.

    Using Devstack to Configure the Simple Gateway

    Another way to configure the simple gateway is to set configuration parameters in the devstack localrc file.

    The following parameters are available:

    • CONTRAIL_VGW_PUBLIC_NETWORK —The name of the routing instance for which the simple gateway is being configured.
    • CONTRAIL_VGW_PUBLIC_SUBNET —A list of subnet addresses allocated for the virtual network. Routes containing these addresses are added to both the host OS and the routing instance for the fabric. List multiple subnets by separating each with a space.
    • CONTRAIL_VGW_INTERFACE —A list of subnets in the public network that are reachable from the virtual network. Routes containing these subnets are added to the routing instance configured for the simple gateway. List multiple subnets by separating each with a space.

    This method can only add the default route 0.0.0.0/0 into the routing instance specified in CONTRAIL_VGW_PUBLIC_NETWORK.

    Example: Devstack Configuration for Simple Gateway

    Add following lines in the localrc file for stack.sh:

    CONTRAIL_VGW_INTERFACE=vgw1
    
    CONTRAIL_VGW_PUBLIC_SUBNET=192.168.1.0/24
    
    CONTRAIL_VGW_PUBLIC_NETWORK=default-domain:admin:net1:net1

    Note: This method can only add default route 0.0.0.0/0 into the routing instance specified in CONTRAIL_VGW_PUBLIC_NETWORK.

    Common Issues with Simple Gateway Configuration

    The following are common problems you might encounter when a simple gateway is configured.

    • Packets from the external network are not reaching the compute node.

      The devices in the fabric network must be configured with static routes for the IP addresses defined in the public subnet (192.168.1.0/24 in the example) to reach the compute node that is running as a simple gateway.

    • Packets are reaching the compute node, but are not routed from the host OS to the virtual machine.

      Check to see if the firewall_driver in /etc/nova/nova.conf file is set to nova.virt.libvirt.firewall.IptablesFirewallDriver, which enables IPTables. IPTables can discard packets.

      Resolutions include disabling IPTables during runtime or setting the firewall_driver in localrc: LIBVIRT_FIREWALL_DRIVER=nova.virt.firewall.NoopFirewallDriver

    Modified: 2015-09-02