Understanding Contrail with Mesos Architecture
From Contrail Release 5.1.x, Contrail overlay and non-overlay network virtualization features are also available in Apache Mesos environment. The features are available in the commercial version of Mesosphere DC/OS.
Contrail with Mesos Architecture Diagram
Setup information
Setup is performed in two parts:
DC/OS installation.
For DC/OS setup, refer to https://dcos.io/install.
Contrail installation.
For Contrail installation, refer to https://github.com/Juniper/contrail-ansible-deployer.
You must update the inventory file and set the orchestrator as mesos.
Master nodes consists of:
DC/OS master components.
For details, refer to https://docs.mesosphere.com/1.11/overview/architecture/components/.
Contrail master components including Contrail Controller, Analytics, Config, and UI.
Slave/Agent nodes consists of:
Contrail Agent.
Contrail vRouter kernel module.
Contrail CNI.
Contrail Mesos Manager.
DC/OS slave components.
For details, refer to https://docs.mesosphere.com/1.11/overview/architecture/components/.
Components
The following components are a part of the architecture:
Contrail Controller
Contrail controller performs all the decision making. It includes config management, analytics, UI and control plane components for network virtualization. For further details, refer to https://github.com/Juniper/contrail-controller.
Contrail controller exposes APIs for creating configuration and updating virtual network components. In Mesos, mesos manager updates the task (universal docker) information to the Contrail controller via API server. All Contrail controller components are microservice docker containers.
Mesos Manager
Mesos manager consists of two sub modules:
VNC server.
Interaction with CNI and mesos agent.
Mesos manager application runs inside a docker on every slave node.
Mesos manager creates two networking by default: mesos-default-pod-task network and ip-fabric network.
All the pods and tasks are created in the mesos-default-pod-task network.
ip-fabric network is created in the respective domains of mesos-default and project-default.
CNI receives the task information and posts it to the Mesos manager. On receiving the task information, the Mesos manager creates the contrail-vnc objects.
Configuration information for the Mesos manager is present in contrail-mesos.conf file. The file is located at /etc/contrail/contrail-mesos.conf in the mesos manager docker.
Sample of contrail-mesos.conf file:
[MESOS] listen_ip_addr=127.0.0.1 listen_port=6991 pod_task_subnets=10.x.x.0/12 ip_fabric_subnets=10.x.x.0/12 [VNC] vnc_endpoint_ip=127.0.0.1 vnc_endpoint_port=8082 admin_user=admin admin_password=admin admin_tenant=admin rabbit_server=127.0.0.1 rabbit_port=5673 cassandra_server_list=127.0.0.1:9161 [DEFAULTS] disc_server_ip=127.0.0.1 disc_server_port=5998 log_local=1 log_level=SYS_NOTICE log_file=/var/log/contrail/contrail-mesos-manager.log [SANDESH] #sandesh_ssl_enable=False #introspect_ssl_enable=False #sandesh_keyfile=/etc/contrail/ssl/private/server-privkey.pem #sandesh_certfile=/etc/contrail/ssl/certs/server.pem #sandesh_ca_cert=/etc/contrail/ssl/certs/ca-cert.pem
You can add the network to pod or task through annotation. You can set the network using labels.
Sample task/pod input json file:
networks": [
{
"name": "contrail-cni-plugin",
"mode": "container",
"labels": {
"networks": "default-domain:default:blue-network",
"pod-subnets": "default-domain:default:blue-network"
}
}
Introspect for mesos-manager objects on the port 8109.
Contrail Container Network Interface (CNI)
The Container Network Interface (CNI) is located at /opt/mesosphere/active/cni/contrail-cni-plugin. It is a run to completion executable file.
The config file is located at /opt/mesosphere/etc/dcos/network/cni/contrail-cni-plugin.conf.
Sample contrail-cni-plugin.conf file:
{
"cniVersion": "0.2.0",
"contrail" : {
"vrouter-ip" : "slave-ip",
"vrouter-port" : 9091,
"cluster-name" : "slave-hostname",
"config-dir" : "/var/lib/contrail/ports/vm",
"poll-timeout" : 15,
"poll-retries" : 5,
"log-file" : "/var/log/contrail/cni/opencontrail.log",
"log-level" : "debug",
"mesos-ip" : "localhost",
"mesos-port" : "6991",
"mode" : "mesos"
},
"name": "contrail-cni-plugin",
"type": "contrail-cni-plugin"
}
Mesos agent invokes Contrail CNI when custom/host network provider is mentioned as contrail-cni-plugin in the task description.