Agent Modules in Contrail Networking

Virtual-Machine Ports

Agent opens a thrift service (name InstanceService) to listen for Port-Add/Port-Delete message. Port-Add informs agent about a virtual machine (VM) interface created on the compute node. The Port-Add message also contains the following information:

• Name of virtual machine port.

• Virtual machine for the port.

• Mac and IP address for the port.

• Virtual network for the port.

Once agent knows about the creation of a port, it will send a subscribe message to contrail-controller for virtual-machine. When contrail-controller receives the subscribe message for a virtual-machine, it walks through the IF-MAP graph and sends all configuration relevant for the virtual-machine to the Agent. The module invoking port add message is platform dependent. In the case of OpenStack, nova-compute service invokes the message.

IF-MAP

All of the Contrail Virtual Network Controller (VNC) configuration is stored as a Metadata Access Point (MAP) database. The MAP database is accessed using IF-MAP protocol.

Agent does not access the MAP database directly. Instead, Agent opens an XMPP connection to contrail-controller to get the MAP configuration. The contrail-controller works on a subscription model. Agent must subscribe to the virtual machines of interest and contrail-controller will download all of the configuration relevant to the virtual-machine. As a result, Agent receives only the minimal configuration needed. Agent subscribes to a virtual-machine when it receives a port add message for a virtual-machine-interface.

Agent uses the ifmap-agent-client library to parse the IF-MAP messages from the XMPP channel to the contrail-controller. The ifmap-agent-client defines a DBTable for every IF-MAP node type. A special DBTable is defined to store the IF-MAP links. The ifmap-agent-client also creates a graph for ease of navigating the IF-MAP configuration. An IF-MAP node is vertex in the graph and links form the edges in the graph.

Configuration Management

Config module registers DBTables of interest from the ifmap-agent-client library. Any add, delete, or update of the configuration results in a callback to the Config module. The Config module then does basic validation on the config nodes and then triggers the operational module to process the configuration.

Redundancy

Agent connects to two different control nodes for redundancy. When the XMPP connection for one of the control node fails, it will subscribe to the other control node for configuration. When connecting to the new control node, Config module audits the configuration to remove stale configuration.

Virtual Network

Table of all virtual-networks with UUID as the key. It contains the following information:

VRF

The virtual routing and forwarding (VRF) table represents a routing-instance in configuration. Each virtual-network has a "native" VRF. Other than the per virtual-network VRF, there can be other internal VRFs. The internal VRFs are used in features, such as service chaining.

Each VRF has a set of routing tables as its members.

Virtual Machine

The virtual machine table stores all virtual-machines created on the compute node.

Interface

The interface table contains all of the interfaces in Agent. Based on the type of interface, the trigger to create an interface can vary. Also, the key fields used to uniquely identify the interface and the data fields in an interface can vary based on the type of interface.

Agent supports the following different types of interfaces:

An interface is in Active state if all of the necessary configuration for the interface is available and it can be made operational.

An interface is in Inactive state if it cannot be made operational. The reason can be missing configuration, the link-state down, and so on.

Routes

Every VRF has a set of routing tables for inet4 unicast routes, inet4 mulitcast routes, EVPN routes, and bridge MAC entries.

Every route specifies the forwarding action for a destination. Agent has multiple modules that can have different views of forwarding action for a destination. Each forwarding action is specified in the form of a path. Each module that adds a path is identified by a peer in the path.

Route keeps the list of paths sorted. The head of this list is treated as the Active path for the route.

Every path contains next hop that describes forwarding action.

The unicast routing table also maintains route entries in the Patricia tree form to support longest prefix match (LPM) on the tree.

Next Hop

Next hop describes the forwarding action for routes pointing to it. When route lookup for an address hits the route, the forwarding action for the packet is defined by the next hop.

The different types of next hop supported in Agent are:

MPLS

The MPLS label defines the forwarding action for MPLS tunneled packets received on the fabric network.

Agent assigns the following labels:

• Two labels are allocated for every VM interface.

  • A label for layer 3 packets.

  • A label for bridge packets.

• A label for every ECMP composite next hop.

• A label for every multicast composite next hop.

The label-range for multicast composite next hop is preallocated and does not overlap with other labels.

Multicast

Multicast module is responsible for managing multicast routes.

VXLAN

The VXLAN table contains an entry for every VXLAN ID created.

Configuration Channel

The Contrail controller uses this channel to send IF-MAP configuration to Agent. Agent subscribes to configuration from only one of the XMPP channels at a time. If the subscribed channel fails, it will switch subscription to the other channel.

Route Channel

This channel is used to exchange routes between Agent and Contrail controller. Agent connects to two Contrail controllers at a time and routes are exchanged between both of the channels. Routes from each of the channels is added with a different "Route Peer." When one of the channels fails, it only deletes "Route Path" from the channel that failed.

Headless Mode

When the XMPP channel from Agent to the Contrail controller fails, Agent flushes all of the "Route Paths" added by the controller. If the connection to both of the Contrail controllers fail, this can result in deleting routes distributed by the controller.

Connections to Contrail controllers can fail for many reasons including network failure, Contrail controller node failing, and so on. Deleting paths can result in connectivity loss between virtual machines.

Headless mode is introduced as a resilient mode of operation for Agent. When running in headless mode, Agent retains the last "Route Path" from Contrail controller. The "Route Paths" are held until a new stable connection is established to one of the Contrail controllers. Once the XMPP connection is up and is stable for a predefined duration, the "Route Paths" from the old XMPP connection are flushed.