Contrail Cloud Software

Contrail Cloud Software Overview

Juniper Networks Contrail Cloud is an integrated cloud platform that uses the Red Hat OpenStack Platform for orchestration, Juniper Contrail Networking for network automation, AppFormix for monitoring and analytics, and Red Hat Ceph Storage for storage. All Contrail Cloud software packages are deployed using a single installer. Contrail Cloud is commonly deployed in Service Provider and Enterprise environments.

Node Types

Nodes in Contrail Cloud are software packages that run in servers and in virtual machines. Figure 1 illustrates the nodes that are deployed in Contrail Cloud 13.1.

Table 3 provides a summary description of each node.

Table 3: Node Summary

Node	Description
Compute node	Server provisioned to support virtualization; hosts virtual machines. Networking of the VMs on a compute node is provided by the Contrail Networking vRouter.
Control Host	Server that hosts controller nodes. Controller nodes are VMs responsible for controlling Contrail Cloud functions. For additional information on control hosts and controller nodes, see Contrail Cloud Configuration.
Jumphost	A server that performs the following functions in Contrail Cloud: hosts the undercloud. The undercloud runs as a VM on the jumphost and is responsible for provisioning and managing all controls hosts, compute nodes, and storage nodes. hosts Contrail Cloud automation. All Contrail Cloud scripts and YAML configuration files are stored and instantiated on the jumphost. hosts the Contrail Command web user interface virtual machine. serves as the entry point for access to the undercloud or overcloud. You can “jump” between the overcloud and the undercloud on the jumphost.
Storage node	Server whose purpose is storing data. Storage nodes run Red Hat Ceph storage software in Contrail Cloud.

Note:

Storage nodes must be separate from compute nodes in a Contrail Cloud environment. Hyperconverged nodes are not supported.

Note:

TSN stands for ToR Service Node. A TSN is an optional node used when Contrail Networking is managing bare metal servers. In the current version of Contrail Networking, the TSN name has been changed to Contrail Service Node (CSN). The Red Hat OOO Heat templates, however, still contain the term TSN. Both terms are used in this guide.

Table 4 illustrates which nodes are deployed as virtual machines and which nodes must run in the operating system of a server.

Table 4: Virtual Machine Summary

Function		Deployed as VM
Jumphost		No
	Undercloud	Yes
	Contrail Command	Yes
Controller Hosts	OpenStack	Yes
	Contrail Controller	Yes
	Contrail Analytics	Yes
	Contrail Analytics DB	Yes
	Appformix	Yes
	Contrail Service Node (TOR Service Node)	Yes
Compute Nodes	vRouter in kernel mode	No
	vRouter using DPDK	No
	vRouter using SR-IOV	No
Storage Nodes (Ceph Software)		No

All hosts in a Contrail Cloud deployment must run on servers dedicated to the Contrail Cloud deployment since Contrail Cloud provisions the servers with a new operating system. The jumphost can only run the undercloud and Contrail Command VMs.

The hardware management for nodes deployed in bare-metal servers is typically done using IPMI while VMs are typically deployed using VirtualBMC. Note that a Red Hat support exception has been made to use VirtualBMC in Contrail Cloud production environments.

In the most common deployment architectures, an instance of each controller function runs in each of three controller node servers. This style of deployment is described in detail in this reference architecture. See Contrail Cloud Configuration.

Additional descriptions on how to configure variable deployments—such as splitting OpenStack, Contrail Controller, and Contrail Analytics onto separate sets of servers for improved performance and reliability—are also provided in this reference architecture. See Reference Architecture Variations.

Server Hardware

The servers used for the jumphost, controller nodes, compute nodes, and storage nodes have different physical server requirements.

Physical server requirements can vary between environments. The following table provides the recommended minimum physical server specifications for each node type.

Table 5: Physical Server Minimum Specifications

Role	Cores	Memory	Disk	Network
Jumphost	20 (2x10)	128 GB	2 x 1 TB HDD (SW RAID) — Host OS Note: HW Raid recommended if RAID controller is available in your environment. 3 x 1 TB HDD — Data	1G — IPMI 1G/10G — Provisioning 1G/10G — Intranet
Control Host	36 (2x18)	256 GB	1 x 1 TB SSD (Cassandra Journaling) Note: 1x1TB is the recommended disk memory option for control hosts. 2 x 1 TB HDD (HW RAID)—Host OS 3 x 2 TB HDD (HW RAID)—Data	1 x 1G — IPMI 1x1G/10G — Provisioning 1x1G/10G — Management (optional) 4 x 10G/25G/40G ports on two Intel Fortville (XL710/X710/XXV710) family NICs. Both high-speed NICs must be on NUMA 0.
Compute Node	28 (2x14)	256 GB	2 x 1 TB HDD (HW RAID) — Host OS 4 x 1 TB HDD — Data (only needed if Ceph storage not used to provide storage)	1 x 1G — IPMI 1x1G/10G — Provisioning 1x1G/10G — Management (optional) 4 x 10G/25G/40G ports on two Intel Fortville (XL710/X710/XXV710) family NICs. Both high-speed NICs must be on NUMA 0.
Storage Node	16 (2x8)	128 GB	2 x 1 TB HDD (HW RAID) — Host OS 2 x 480 GB SSDs — Ceph Journaling 10 x 1 TB HDDs — Ceph OSDs	1 x 1G — IPMI 1x1G/10G — Provisioning 1x1G/10G — Management (optional) 4 x 10G/25G/40G ports on two Intel Fortville (XL710/X710/XXV710) family NICs. Both high-speed NICs must be on NUMA 0.

Note:

Virtual machines and processes located on NUMA 1 communicate with NICs via the QPI bus, which is less costly than balancing traffic across NICs in both NUMAs.

Note:

For improved controller performance, you can use 6 SSDs. Configure 5 drives as RAID 5 and maintain a spare drive when using 6 SSDs.

Note:

The number of drives and capacity in storage nodes can be increased as needed. The Red Hat recommendation of a 1:5 ratio of SSD:HDD drives should be followed. For additional information, see the Throughput-optimized Solutions section in the Red Hat Storage Hardware Selection Guide).

The sizing and configuration recommendations provided in this reference architecture are intended to enable a Contrail Cloud environment to operate at the highest possible performance. The configuration file examples provided in this reference architecture assume these server configuration recommendations are followed. Other server configurations are supported; see Reference Architecture Variations.

Networking for Contrail Cloud 13.1 and 13.2

Traffic for different purposes is segregated onto different physical logical interfaces in Contrail Cloud 13.1 and 13.2. The configuration of server-facing ports in the IP Fabric must match the configuration of the corresponding server ports as specified in the Contrail Cloud configuration files.

Table 6 lists the server-layer networks used in Contrail Cloud 13.1 and 13.2.

Table 6: Networking in Contrail Cloud Summary

Network	Purpose
IPMI	Provides hardware management of servers. IPMI services are mostly used by the Openstack Ironic service and are established outside of the Contrail Cloud installation. IPMI services can be used by the server nodes in Contrail Cloud.
Intranet	Provides user access to the jumphost. Provides access to satellite repositories for the jumphost. Provides access to the Contrail Command web user interface. Provides external network access via SNAT from the control plane network.
Provisioning or Control Plane	Deploys new nodes using PXE booting and to install software packages on overcloud bare metal servers. The provisioning/control plane network is used by Red Hat Director and is predefined before the creation of the undercloud.
Internal API	Provides communication with the OpenStack and Contrail Networking services, including Keystone, Nova, and Neutron using API communication, RPC messages, and database communication.
External	Provides tenant administrators access to the OpenStack Dashboard (Horizon) graphic interface, the public APIs for Openstack services, public Contrail APIs, and the Appformix WebUI. Commonly used as a path to the intranet.
Tenant	Supports overlay data-plane traffic—VXLAN and MPLS over UDP—and Contrail Controller to Contrail vRouter control plane traffic.
Storage	Supports storage data traffic for Ceph, block storage, NFS, iSCSI, and any other storage types.
Storage Management	Provides Ceph control, management, and replication traffic.
Management	(Optional) Provides direct access to compute nodes without having to send traffic through the jumphost. Can be used for DNS and NTP services.

Note:

Depending on the context, Red Hat documentation uses the names Provisioning and Control Plane for the same network. In this document, the Provisioning network name is used to refer to the network in the undercloud and the Control Plane network name is used to refer to the network in the overcloud. The Control Plane network name, notably, is used in configuration files.

All networks, except IPMI and Intranet, are configured on nodes by Contrail Cloud 13.1 and 13.2 using information contained in YAML configuration files. Table 7 lists the networking requirements by role in Contrail Cloud 13.1 and 13.2.

Table 7: Device & VM Network Summary

Role	Intranet	Provisioning	Internal API	External	Tenant	Storage	Storage Mgmt	Mgmt
Jumphost (Undercloud VM)	✓	✓
Openstack Controller		✓	✓	✓	Optional	✓	✓	Optional
Contrail Controller		✓	✓	Optional	✓			Optional
Contrail Analytics		✓	✓	Optional	✓			Optional
Contrail Analytics DB		✓	✓	Optional	✓			Optional
Contrail Service Node (ToR Service Node)		✓	✓	Optional	✓			Optional
Appformix Controller		✓	✓	✓	Optional		✓	Optional
Control Host (Running all Controller VMs)		✓						Optional
Compute Node		✓	✓		✓	✓		Optional
Storage Node		✓				✓	✓	Optional

Table 8 provides the networking requirements for each host type in a Contrail Cloud. The table assumes that each listed host is running all applicable controller functions.

Table 8: Host Network Summary

Host	Intranet	Provisioning	Internal API	External	Tenant	Storage	Storage Mgmt	Mgmt
Jumphost	✓	✓
Control Host		✓	✓	✓	✓	✓	✓	Optional
Compute Node		✓	✓		✓	✓		Optional
Storage Node		✓				✓	✓	Optional

The server port assignments depend on the available NICs in each server.

The use of a management network to enable direct access to nodes without going through the jumphost is optional. It is included in this reference architecture for convenience.

The diagram, below, shows the layout for a Contrail Cloud environment in three racks, with controller nodes that are running all controller functions.

Note:

Red Hat configuration files use the term leaf to refer to a group of servers—generally in the same rack—which share switches for network connectivity. In this reference architecture, connectivity for servers in the same Red Hat leaf is implemented by a pair of switches with leaf roles in the IP Fabric and by a management switch.

Note:

Some Contrail Cloud documentation shows the controller nodes all placed in the same rack. The nodes are placed in separate racks for resilience in this reference architecture. The architecture shown in the other Contrail documentation is a supported variance for this reference architecture. See Reference Architecture Variations.

The IPMI, Management, and Provisioning networks are connected via management switches and the networks are stretched between switches using trunk connections for VLANs, or by using a separate VXLAN for each network.

The external network contains the externally-routable API and UI IP addresses for the various controller functions, and generally these addresses reside in the same subnet. The VLAN for the external network is configured to be in a VXLAN, which terminates in a VRF which is configured to route between the network where tenant user packets arrive from the external network. The network naming convention for these rack-specific subnets is network_name(leaf #), where the leaf number is appended to the network name. The name of the network without an appended leaf number is used to identify the subnets that are used by control hosts.

The rest of the networks are connected into the leaf switches within the IP Fabric, where EVPN-VXLAN is used to connect switches between racks. The diagram shows connectivity for these networks within supernets that span racks, but these supernets are subdivided on a per rack basis using the leaf-id as part of each subnet identifier. This setup is described in more detail later in this document.

The other networks connected to the IP Fabric have a separate subnet for each rack. Routing is used to connect the subnets together using VXLAN. The routing can be configured in leaf devices (edge routing) or spine devices (central routing). Edge routing is used in this reference architecture.

Table 9 summarizes recommended network configurations options.

Table 9: Recommended Network Subnets and Implementations

Network	Subnet Description	Implementation
IPMI	IPMI is typically an external service that can be used by devices that can route to it. IPMI is not established by Contrail Cloud.	IPMI is typically not spanned over racks. It can be spanned over racks using VXLAN.
Intranet	Administrative access to jumphost.	Specific port configuration
Provisioning	Layer 2 single subnet	Span over racks using VLAN or VXLAN
Internal API	Layer 2 single subnet	Span over racks using VLAN or VXLAN
Internal API leaf-number	One subnet per rack leaf-number is the Red Hat Openstack leaf number.	Layer 3 routing between racks
External	Layer 2 single subnet	Span over racks using VLAN or VXLAN
Tenant	Layer 2 single subnet	Span over racks using VLAN or VXLAN
Tenant leaf-number	One subnet per rack leaf-number is the Red Hat Openstack leaf number.	Layer 3 routing between racks
Storage	Layer 2 single subnet	Span over racks using VLAN or VXLAN
Storage leaf-number	One subnet per rack leaf-number is the Red Hat Openstack leaf number.	Layer 3 routing between racks
Storage Management	Layer 2 single subnet	Span over racks using VLAN or VXLAN
Storage Management leaf-number	One subnet per rack leaf-number is the Red Hat Openstack leaf number.	Layer 3 routing between racks
Management	Layer 2 single subnet	Span over racks using VLAN or VXLAN

When a network is composed of multiple subnets—one subnet per rack—the subnet that contains all the rack subnets is termed the supernet. The subnet address of the supernet is used to configure static routes on servers to ensure proper traffic flow. It is important that each supernet address range is large enough to accommodate future growth in the number of racks.

For networks that have a subnet per rack, the base subnet that is used for control hosts is stretched across the racks containing such nodes. A naming convention is used in configuration files where the name of the subnet in each rack has a suffix of the number of the rack (leaf in Red Hat terminology) and the base subnet has no suffix. For instance, the tenant network is stretched across the racks containing control hosts, and tenant0, tenant1, and is used for compute and storage nodes in each rack. This is explained in detail in Internal API, Tenant, Storage, and Storage Management Network sections.

The network configuration described in the table is designed to provide maximum robustness and performance. Variations are supported where networks span racks instead.

The upcoming diagrams illustrate how VLANs, VTEPs, IRBs, and VRFs are configured in gateway, spine, and leaf devices. Illustrations of these components in edge-routed bridging (ERB) environments—where routing is done on the leaf devices—and centrally-routed bridging (CRB)—where routing is done on the spine devices—are provided.

For additional information on ERB and CRB architectures, see Centrally-Routed Bridging Overlay Design and Implementation and Edge-Routed Bridging Overlay Design and Implementation in the Cloud Data Center Architecture Guide.

Note:

The diagram shows a logical view of the networks. All traffic between leaf switches passes through a spine switch and not directly between leaf switches.

The Virtual Routing and Forwarding (VRF) instances are created on the spine devices and provide VXLAN tunnels to each leaf device when centralized routing and bridging (CRB) is used.

Figure 5 illustrates VXLAN connectivity for a leaf device in a CRB environment.

The main features of this network design:

Table 10: VLAN Summaries

Device	VLAN Summary
Management Switch	The IPMI, Management, Provisioning, and Intranet networks all connect the management switch to a server on different ports. The port traffic for each network arrives on the management switch untagged and is configured into the same VLAN. The VLAN is extended between the management switches on different racks.
Leaf Switch (EVPN-VXLAN Fabric)	Traffic from the external, internal API, tenant, storage, and storage management networks arrives on the leaf switch’s high-speed interfaces from the servers in different VLANs. The VLANs are configured in VTEPs. Leaf ports are configured with logical interfaces that have VLANs for the networks that will be attached by servers to that port. Each VLAN on each switch is configured into a VXLAN VTEP and EVPN advertises host routes for each connected server to the spines. The VLANs used for each network are specified in the file overcloud-nics.yml.
Spine Switch (EVPN-VXLAN Fabric)	The spine switches are configured with a VTEP for each of the Internal API, Tenant, Storage and Storage-Mgt networks, and each of these are connected to an IRB interface whose IP address is that of the supernet. Each spine switch has a VRF that receives routes to each host from the leaf switches.
SDN Gateway Router	The SDN gateways are configured with a VTEP and VRF for the external network. Each SDN gateway will advertise a route for the external network to peers outside the Layer 2 network.

This reference architecture does not provide fabric configuration options and details.

To configure an EVPN-VXLAN fabric that can be used by the nodes in a Contrail Cloud to transport Layer 2 and Layer 3 traffic, see Data Center Fabric Architecture Guide.

Interface Addressing and Settings

This section covers interface addressing and other interface settings.

• Leaf Subnets and Supernets
• Address Allocation Within Networks
• Use of IPv6 in Contrail Cloud networks
• MTU settings
• Interface Naming Conventions