Setting Up the Overcloud
Configuring the Overcloud
Use this example procedure on the undercloud to set up the configuration for the overcloud.
Customizing the Contrail Service with Templates (contrail-services.yaml)
This section contains information to customize Contrail services for your network by modifying the contrail-services.yaml file.
Contrail Services customization
vi ~/tripleo-heat-templates/environments/contrail-services.yaml parameter_defaults: ContrailSettings: VROUTER_GATEWAY: 10.0.0.1 # KEY1: value1 # KEY2: value2 VXLAN_VN_ID_MODE: "configured" ENCAP_PRIORITY: "VXLAN,MPLSoUDP,MPLSoGRE" ContrailControllerParameters: AAAMode: rbac
Contrail registry settings
vi ~/tripleo-heat-templates/environments/contrail-services.yaml
Here are few examples of default values for various registries:
Public Juniper registry
parameter_defaults: ContrailRegistry: hub.juniper.net/contrail ContrailRegistryUser: <USER> ContrailRegistryPassword: <PASSWORD>
Insecure registry
parameter_defaults: ContrailRegistryInsecure: true DockerInsecureRegistryAddress: 10.87.64.32:5000,192.168.24.1:8787 ContrailRegistry: 10.87.64.32:5000
Private secure registry
parameter_defaults: ContrailRegistryCertUrl: http://device.example.net/pub/device.example.net.crt ContrailRegistry: device.example.net:5443
Contrail Container image settings
parameter_defaults: ContrailImageTag: queens-5.0-104-rhel-queens
Customizing the Contrail Network with Templates
- Overview
- Roles Configuration (roles_data_contrail_aio.yaml)
- Network Parameter Configuration (contrail-net.yaml)
- Network Interface Configuration (*-NIC-*.yaml)
- Advanced vRouter Kernel Mode Configuration
- Advanced vRouter DPDK Mode Configuration
- Advanced vRouter SRIOV + Kernel Mode Configuration
- Advanced vRouter SRIOV + DPDK Mode Configuration
- Advanced Scenarios
Overview
In order to customize the network, define different networks and configure the overcloud nodes NIC layout. TripleO supports a flexible way of customizing the network.
The following networking customization example uses network as:
Network |
VLAN |
overcloud Nodes |
---|---|---|
provisioning |
- |
All |
internal_api |
710 |
All |
external_api |
720 |
OpenStack CTRL |
storage |
740 |
OpenStack CTRL, Computes |
storage_mgmt |
750 |
OpenStack CTRL |
tenant |
- |
Contrail CTRL, Computes |
Roles Configuration (roles_data_contrail_aio.yaml)
The networks must be activated per role in the roles_data file:
vi ~/tripleo-heat-templates/roles_data_contrail_aio.yaml
OpenStack Controller
############################################################################### # Role: Controller # ############################################################################### - name: Controller description: | Controller role that has all the controler services loaded and handles Database, Messaging and Network functions. CountDefault: 1 tags: - primary - controller networks: - External - InternalApi - Storage - StorageMgmt
Compute Node
############################################################################### # Role: Compute # ############################################################################### - name: Compute description: | Basic Compute Node role CountDefault: 1 networks: - InternalApi - Tenant - Storage
Contrail Controller
############################################################################### # Role: ContrailController # ############################################################################### - name: ContrailController description: | ContrailController role that has all the Contrail controler services loaded and handles config, control and webui functions CountDefault: 1 tags: - primary - contrailcontroller networks: - InternalApi - Tenant
Compute DPDK
############################################################################### # Role: ContrailDpdk # ############################################################################### - name: ContrailDpdk description: | Contrail Dpdk Node role CountDefault: 0 tags: - contraildpdk networks: - InternalApi - Tenant - Storage
Compute SRIOV
############################################################################### # Role: ContrailSriov ############################################################################### - name: ContrailSriov description: | Contrail Sriov Node role CountDefault: 0 tags: - contrailsriov networks: - InternalApi - Tenant - Storage
Compute CSN
############################################################################### # Role: ContrailTsn ############################################################################### - name: ContrailTsn description: | Contrail Tsn Node role CountDefault: 0 tags: - contrailtsn networks: - InternalApi - Tenant - Storage
Network Parameter Configuration (contrail-net.yaml)
cat ~/tripleo-heat-templates/environments/contrail/contrail-net.yaml resource_registry: OS::TripleO::Controller::Net::SoftwareConfig: ../../network/config/contrail/controller-nic-config.yaml OS::TripleO::ContrailController::Net::SoftwareConfig: ../../network/config/contrail/contrail-controller-nic-config.yaml OS::TripleO::ContrailControlOnly::Net::SoftwareConfig: ../../network/config/contrail/contrail-controller-nic-config.yaml OS::TripleO::Compute::Net::SoftwareConfig: ../../network/config/contrail/compute-nic-config.yaml OS::TripleO::ContrailDpdk::Net::SoftwareConfig: ../../network/config/contrail/contrail-dpdk-nic-config.yaml OS::TripleO::ContrailSriov::Net::SoftwareConfig: ../../network/config/contrail/contrail-sriov-nic-config.yaml OS::TripleO::ContrailTsn::Net::SoftwareConfig: ../../network/config/contrail/contrail-tsn-nic-config.yaml parameter_defaults: # Customize all these values to match the local environment TenantNetCidr: 10.0.0.0/24 InternalApiNetCidr: 10.1.0.0/24 ExternalNetCidr: 10.2.0.0/24 StorageNetCidr: 10.3.0.0/24 StorageMgmtNetCidr: 10.4.0.0/24 # CIDR subnet mask length for provisioning network ControlPlaneSubnetCidr: '24' # Allocation pools TenantAllocationPools: [{'start': '10.0.0.10', 'end': '10.0.0.200'}] InternalApiAllocationPools: [{'start': '10.1.0.10', 'end': '10.1.0.200'}] ExternalAllocationPools: [{'start': '10.2.0.10', 'end': '10.2.0.200'}] StorageAllocationPools: [{'start': '10.3.0.10', 'end': '10.3.0.200'}] StorageMgmtAllocationPools: [{'start': '10.4.0.10', 'end': '10.4.0.200'}] # Routes ControlPlaneDefaultRoute: 192.168.24.1 InternalApiDefaultRoute: 10.1.0.1 ExternalInterfaceDefaultRoute: 10.2.0.1 # Vlans InternalApiNetworkVlanID: 710 ExternalNetworkVlanID: 720 StorageNetworkVlanID: 730 StorageMgmtNetworkVlanID: 740 TenantNetworkVlanID: 3211 # Services EC2MetadataIp: 192.168.24.1 # Generally the IP of the undercloud DnsServers: ["172.x.x.x"] NtpServer: 10.0.0.1
Network Interface Configuration (*-NIC-*.yaml)
NIC configuration files exist per role in the following directory:
cd ~/tripleo-heat-templates/network/config/contrail
OpenStack Controller
heat_template_version: queens description: > Software Config to drive os-net-config to configure multiple interfaces for the compute role. This is an example for a Nova compute node using Contrail vrouter and the vhost0 interface. parameters: ControlPlaneIp: default: '' description: IP address/subnet on the ctlplane network type: string ExternalIpSubnet: default: '' description: IP address/subnet on the external network type: string InternalApiIpSubnet: default: '' description: IP address/subnet on the internal_api network type: string InternalApiDefaultRoute: # Not used by default in this template default: '10.0.0.1' description: The default route of the internal api network. type: string StorageIpSubnet: default: '' description: IP address/subnet on the storage network type: string StorageMgmtIpSubnet: default: '' description: IP address/subnet on the storage_mgmt network type: string TenantIpSubnet: default: '' description: IP address/subnet on the tenant network type: string ManagementIpSubnet: # Only populated when including environments/network-management.yaml default: '' description: IP address/subnet on the management network type: string ExternalNetworkVlanID: default: 10 description: Vlan ID for the external network traffic. type: number InternalApiNetworkVlanID: default: 20 description: Vlan ID for the internal_api network traffic. type: number StorageNetworkVlanID: default: 30 description: Vlan ID for the storage network traffic. type: number StorageMgmtNetworkVlanID: default: 40 description: Vlan ID for the storage mgmt network traffic. type: number TenantNetworkVlanID: default: 50 description: Vlan ID for the tenant network traffic. type: number ManagementNetworkVlanID: default: 60 description: Vlan ID for the management network traffic. type: number ControlPlaneSubnetCidr: # Override this via parameter_defaults default: '24' description: The subnet CIDR of the control plane network. type: string ControlPlaneDefaultRoute: # Override this via parameter_defaults description: The default route of the control plane network. type: string ExternalInterfaceDefaultRoute: # Not used by default in this template default: '10.0.0.1' description: The default route of the external network. type: string ManagementInterfaceDefaultRoute: # Commented out by default in this template default: unset description: The default route of the management network. type: string DnsServers: # Override this via parameter_defaults default: [] description: A list of DNS servers (2 max for some implementations) that will be added to resolv.conf. type: comma_delimited_list EC2MetadataIp: # Override this via parameter_defaults description: The IP address of the EC2 metadata server. type: string resources: OsNetConfigImpl: type: OS::Heat::SoftwareConfig properties: group: script config: str_replace: template: get_file: ../../scripts/run-os-net-config.sh params: $network_config: network_config: - type: interface name: nic1 use_dhcp: false dns_servers: get_param: DnsServers addresses: - ip_netmask: list_join: - '/' - - get_param: ControlPlaneIp - get_param: ControlPlaneSubnetCidr routes: - ip_netmask: 169.x.x.x/32 next_hop: get_param: EC2MetadataIp - default: true next_hop: get_param: ControlPlaneDefaultRoute - type: vlan vlan_id: get_param: InternalApiNetworkVlanID device: nic1 addresses: - ip_netmask: get_param: InternalApiIpSubnet - type: vlan vlan_id: get_param: ExternalNetworkVlanID device: nic1 addresses: - ip_netmask: get_param: ExternalIpSubnet - type: vlan vlan_id: get_param: StorageNetworkVlanID device: nic1 addresses: - ip_netmask: get_param: StorageIpSubnet - type: vlan vlan_id: get_param: StorageMgmtNetworkVlanID device: nic1 addresses: - ip_netmask: get_param: StorageMgmtIpSubnet outputs: OS::stack_id: description: The OsNetConfigImpl resource. value: get_resource: OsNetConfigImpl
Contrail Controller
heat_template_version: queens description: > Software Config to drive os-net-config to configure multiple interfaces for the compute role. This is an example for a Nova compute node using Contrail vrouter and the vhost0 interface. parameters: ControlPlaneIp: default: '' description: IP address/subnet on the ctlplane network type: string ExternalIpSubnet: default: '' description: IP address/subnet on the external network type: string InternalApiIpSubnet: default: '' description: IP address/subnet on the internal_api network type: string InternalApiDefaultRoute: # Not used by default in this template default: '10.0.0.1' description: The default route of the internal api network. type: string StorageIpSubnet: default: '' description: IP address/subnet on the storage network type: string StorageMgmtIpSubnet: default: '' description: IP address/subnet on the storage_mgmt network type: string TenantIpSubnet: default: '' description: IP address/subnet on the tenant network type: string ManagementIpSubnet: # Only populated when including environments/network-management.yaml default: '' description: IP address/subnet on the management network type: string ExternalNetworkVlanID: default: 10 description: Vlan ID for the external network traffic. type: number InternalApiNetworkVlanID: default: 20 description: Vlan ID for the internal_api network traffic. type: number StorageNetworkVlanID: default: 30 description: Vlan ID for the storage network traffic. type: number StorageMgmtNetworkVlanID: default: 40 description: Vlan ID for the storage mgmt network traffic. type: number TenantNetworkVlanID: default: 50 description: Vlan ID for the tenant network traffic. type: number ManagementNetworkVlanID: default: 60 description: Vlan ID for the management network traffic. type: number ControlPlaneSubnetCidr: # Override this via parameter_defaults default: '24' description: The subnet CIDR of the control plane network. type: string ControlPlaneDefaultRoute: # Override this via parameter_defaults description: The default route of the control plane network. type: string ExternalInterfaceDefaultRoute: # Not used by default in this template default: '10.0.0.1' description: The default route of the external network. type: string ManagementInterfaceDefaultRoute: # Commented out by default in this template default: unset description: The default route of the management network. type: string DnsServers: # Override this via parameter_defaults default: [] description: A list of DNS servers (2 max for some implementations) that will be added to resolv.conf. type: comma_delimited_list EC2MetadataIp: # Override this via parameter_defaults description: The IP address of the EC2 metadata server. type: string resources: OsNetConfigImpl: type: OS::Heat::SoftwareConfig properties: group: script config: str_replace: template: get_file: ../../scripts/run-os-net-config.sh params: $network_config: network_config: - type: interface name: nic1 use_dhcp: false dns_servers: get_param: DnsServers addresses: - ip_netmask: list_join: - '/' - - get_param: ControlPlaneIp - get_param: ControlPlaneSubnetCidr routes: - ip_netmask: 169.x.x.x/32 next_hop: get_param: EC2MetadataIp - default: true next_hop: get_param: ControlPlaneDefaultRoute - type: vlan vlan_id: get_param: InternalApiNetworkVlanID device: nic1 addresses: - ip_netmask: get_param: InternalApiIpSubnet - type: interface name: nic2 use_dhcp: false addresses: - ip_netmask: get_param: TenantIpSubnet outputs: OS::stack_id: description: The OsNetConfigImpl resource. value: get_resource: OsNetConfigImpl
Compute Node
heat_template_version: queens description: > Software Config to drive os-net-config to configure multiple interfaces for the compute role. This is an example for a Nova compute node using Contrail vrouter and the vhost0 interface. parameters: ControlPlaneIp: default: '' description: IP address/subnet on the ctlplane network type: string ExternalIpSubnet: default: '' description: IP address/subnet on the external network type: string InternalApiIpSubnet: default: '' description: IP address/subnet on the internal_api network type: string InternalApiDefaultRoute: # Not used by default in this template default: '10.0.0.1' description: The default route of the internal api network. type: string StorageIpSubnet: default: '' description: IP address/subnet on the storage network type: string StorageMgmtIpSubnet: default: '' description: IP address/subnet on the storage_mgmt network type: string TenantIpSubnet: default: '' description: IP address/subnet on the tenant network type: string ManagementIpSubnet: # Only populated when including environments/network-management.yaml default: '' description: IP address/subnet on the management network type: string ExternalNetworkVlanID: default: 10 description: Vlan ID for the external network traffic. type: number InternalApiNetworkVlanID: default: 20 description: Vlan ID for the internal_api network traffic. type: number StorageNetworkVlanID: default: 30 description: Vlan ID for the storage network traffic. type: number StorageMgmtNetworkVlanID: default: 40 description: Vlan ID for the storage mgmt network traffic. type: number TenantNetworkVlanID: default: 50 description: Vlan ID for the tenant network traffic. type: number ManagementNetworkVlanID: default: 60 description: Vlan ID for the management network traffic. type: number ControlPlaneSubnetCidr: # Override this via parameter_defaults default: '24' description: The subnet CIDR of the control plane network. type: string ControlPlaneDefaultRoute: # Override this via parameter_defaults description: The default route of the control plane network. type: string ExternalInterfaceDefaultRoute: # Not used by default in this template default: '10.0.0.1' description: The default route of the external network. type: string ManagementInterfaceDefaultRoute: # Commented out by default in this template default: unset description: The default route of the management network. type: string DnsServers: # Override this via parameter_defaults default: [] description: A list of DNS servers (2 max for some implementations) that will be added to resolv.conf. type: comma_delimited_list EC2MetadataIp: # Override this via parameter_defaults description: The IP address of the EC2 metadata server. type: string resources: OsNetConfigImpl: type: OS::Heat::SoftwareConfig properties: group: script config: str_replace: template: get_file: ../../scripts/run-os-net-config.sh params: $network_config: network_config: - type: interface name: nic1 use_dhcp: false dns_servers: get_param: DnsServers addresses: - ip_netmask: list_join: - '/' - - get_param: ControlPlaneIp - get_param: ControlPlaneSubnetCidr routes: - ip_netmask: 169.x.x.x/32 next_hop: get_param: EC2MetadataIp - default: true next_hop: get_param: ControlPlaneDefaultRoute - type: vlan vlan_id: get_param: InternalApiNetworkVlanID device: nic1 addresses: - ip_netmask: get_param: InternalApiIpSubnet - type: vlan vlan_id: get_param: StorageNetworkVlanID device: nic1 addresses: - ip_netmask: get_param: StorageIpSubnet - type: contrail_vrouter name: vhost0 use_dhcp: false members: - type: interface name: nic2 use_dhcp: false addresses: - ip_netmask: get_param: TenantIpSubnet outputs: OS::stack_id: description: The OsNetConfigImpl resource. value: get_resource: OsNetConfigImpl
Advanced vRouter Kernel Mode Configuration
In addition to the standard NIC configuration, the vRouter kernel mode supports VLAN, Bond, and Bond + VLAN modes. The configuration snippets below only show the relevant section of the NIC template configuration for each mode.
VLAN
- type: vlan vlan_id: get_param: TenantNetworkVlanID device: nic2 - type: contrail_vrouter name: vhost0 use_dhcp: false members: - type: interface name: str_replace: template: vlanVLANID params: VLANID: {get_param: TenantNetworkVlanID} use_dhcp: false addresses: - ip_netmask: get_param: TenantIpSubnet
Bond
- type: linux_bond name: bond0 bonding_options: "mode=4 xmit_hash_policy=layer2+3" use_dhcp: false members: - type: interface name: nic2 - type: interface name: nic3 - type: contrail_vrouter name: vhost0 use_dhcp: false members: - type: interface name: bond0 use_dhcp: false addresses: - ip_netmask: get_param: TenantIpSubnet
Bond + VLAN
- type: linux_bond name: bond0 bonding_options: "mode=4 xmit_hash_policy=layer2+3" use_dhcp: false members: - type: interface name: nic2 - type: interface name: nic3 - type: vlan vlan_id: get_param: TenantNetworkVlanID device: bond0 - type: contrail_vrouter name: vhost0 use_dhcp: false members: - type: interface name: str_replace: template: vlanVLANID params: VLANID: {get_param: TenantNetworkVlanID} use_dhcp: false addresses: - ip_netmask: get_param: TenantIpSubnet
Advanced vRouter DPDK Mode Configuration
In addition to the standard NIC configuration, the vRouter DPDK mode supports Standard, VLAN, Bond, and Bond + VLAN modes.
Network Environment Configuration:
vi ~/tripleo-heat-templates/environments/contrail/contrail-services.yaml
Enable the number of hugepages:
parameter_defaults: ContrailDpdkHugepages1GB: 10
See the following NIC template configurations for vRouter DPDK mode. The configuration snippets below only show the relevant section of the NIC configuration for each mode.
Standard
- type: contrail_vrouter_dpdk name: vhost0 use_dhcp: false driver: uio_pci_generic cpu_list: 0x01 members: - type: interface name: nic2 use_dhcp: false addresses: - ip_netmask: get_param: TenantIpSubnet
VLAN
- type: contrail_vrouter_dpdk name: vhost0 use_dhcp: false driver: uio_pci_generic cpu_list: 0x01 vlan_id: get_param: TenantNetworkVlanID members: - type: interface name: nic2 use_dhcp: false addresses: - ip_netmask: get_param: TenantIpSubnet
Bond
- type: contrail_vrouter_dpdk name: vhost0 use_dhcp: false driver: uio_pci_generic cpu_list: 0x01 bond_mode: 4 bond_policy: layer2+3 members: - type: interface name: nic2 use_dhcp: false - type: interface name: nic3 use_dhcp: false addresses: - ip_netmask: get_param: TenantIpSubnet
Bond + VLAN
- type: contrail_vrouter_dpdk name: vhost0 use_dhcp: false driver: uio_pci_generic cpu_list: 0x01 vlan_id: get_param: TenantNetworkVlanID bond_mode: 4 bond_policy: layer2+3 members: - type: interface name: nic2 use_dhcp: false - type: interface name: nic3 use_dhcp: false addresses: - ip_netmask: get_param: TenantIpSubnet
Advanced vRouter SRIOV + Kernel Mode Configuration
vRouter SRIOV + Kernel mode can be used in the following combinations:
Standard
VLAN
Bond
Bond + VLAN
Network environment configuration:
vi ~/tripleo-heat-templates/environments/contrail/contrail-services.yaml
Enable the number of hugepages:
parameter_defaults: ContrailSriovHugepages1GB: 10
SRIOV PF/VF settings:
NovaPCIPassthrough: - devname: "ens2f1" physical_network: "sriov1" ContrailSriovNumVFs: ["ens2f1:7"]
The SRIOV NICs are not configured in the NIC templates. However, vRouter NICs must still be configured. See the following NIC template configurations for vRouter kernel mode. The configuration snippets below only show the relevant section of the NIC configuration for each mode.
VLAN
- type: vlan vlan_id: get_param: TenantNetworkVlanID device: nic2 - type: contrail_vrouter name: vhost0 use_dhcp: false members: - type: interface name: str_replace: template: vlanVLANID params: VLANID: {get_param: TenantNetworkVlanID} use_dhcp: false addresses: - ip_netmask: get_param: TenantIpSubnet
Bond
- type: linux_bond name: bond0 bonding_options: "mode=4 xmit_hash_policy=layer2+3" use_dhcp: false members: - type: interface name: nic2 - type: interface name: nic3 - type: contrail_vrouter name: vhost0 use_dhcp: false members: - type: interface name: bond0 use_dhcp: false addresses: - ip_netmask: get_param: TenantIpSubnet
Bond + VLAN
- type: linux_bond name: bond0 bonding_options: "mode=4 xmit_hash_policy=layer2+3" use_dhcp: false members: - type: interface name: nic2 - type: interface name: nic3 - type: vlan vlan_id: get_param: TenantNetworkVlanID device: bond0 - type: contrail_vrouter name: vhost0 use_dhcp: false members: - type: interface name: str_replace: template: vlanVLANID params: VLANID: {get_param: TenantNetworkVlanID} use_dhcp: false addresses: - ip_netmask: get_param: TenantIpSubnet
Advanced vRouter SRIOV + DPDK Mode Configuration
vRouter SRIOV + DPDK can be used in the following combinations:
Standard
VLAN
Bond
Bond + VLAN
Network environment configuration:
vi ~/tripleo-heat-templates/environments/contrail/contrail-services.yaml
Enable the number of hugepages
parameter_defaults: ContrailSriovMode: dpdk ContrailDpdkHugepages1GB: 10 ContrailSriovHugepages1GB: 10
SRIOV PF/VF settings
NovaPCIPassthrough: - devname: "ens2f1" physical_network: "sriov1" ContrailSriovNumVFs: ["ens2f1:7"]
The SRIOV NICs are not configured in the NIC templates. However, vRouter NICs must still be configured. See the following NIC template configurations for vRouter DPDK mode. The configuration snippets below only show the relevant section of the NIC configuration for each mode.
Standard
- type: contrail_vrouter_dpdk name: vhost0 use_dhcp: false driver: uio_pci_generic cpu_list: 0x01 members: - type: interface name: nic2 use_dhcp: false addresses: - ip_netmask: get_param: TenantIpSubnet
VLAN
- type: contrail_vrouter_dpdk name: vhost0 use_dhcp: false driver: uio_pci_generic cpu_list: 0x01 vlan_id: get_param: TenantNetworkVlanID members: - type: interface name: nic2 use_dhcp: false addresses: - ip_netmask: get_param: TenantIpSubnet
Bond
- type: contrail_vrouter_dpdk name: vhost0 use_dhcp: false driver: uio_pci_generic cpu_list: 0x01 bond_mode: 4 bond_policy: layer2+3 members: - type: interface name: nic2 use_dhcp: false - type: interface name: nic3 use_dhcp: false addresses: - ip_netmask: get_param: TenantIpSubnet
Bond + VLAN
- type: contrail_vrouter_dpdk name: vhost0 use_dhcp: false driver: uio_pci_generic cpu_list: 0x01 vlan_id: get_param: TenantNetworkVlanID bond_mode: 4 bond_policy: layer2+3 members: - type: interface name: nic2 use_dhcp: false - type: interface name: nic3 use_dhcp: false addresses: - ip_netmask: get_param: TenantIpSubnet
Advanced Scenarios
Remote Compute
Remote Compute extends the data plane to remote locations (POP) whilest keeping the control plane central. Each POP will have its own set of Contrail control services, which are running in the central location. The difficulty is to ensure that the compute nodes of a given POP connect to the Control nodes assigned to that POC. The Control nodes must have predictable IP addresses and the compute nodes have to know these IP addresses. In order to achieve that the following methods are used:
Custom Roles
Static IP assignment
Precise Node placement
Per Node hieradata
Each overcloud node has a unique DMI UUID. This UUID is known on the undercloud node as well as on the overcloud node. Hence, this UUID can be used for mapping node specific information. For each POP, a Control role and a Compute role has to be created.
Overview
Mapping Table
Nova Name |
Ironic Name |
UUID |
KVM |
IP Address |
POP |
---|---|---|---|---|---|
overcloud-contrailcontrolonly-0 |
control-only-1-5b3s30 |
Ironic UUID: 7d758dce-2784-45fd-be09-5a41eb53e764 DMI UUID: 73F8D030-E896-4A95-A9F5-E1A4FEBE322D |
5b3s30 |
10.0.0.11 |
POP1 |
overcloud-contrailcontrolonly-1 |
control-only-2-5b3s30 |
Ironic UUID: d26abdeb-d514-4a37-a7fb-2cd2511c351f DMI UUID: 14639A66-D62C-4408-82EE-FDDC4E509687 |
5b3s30 |
10.0.0.14 |
POP2 |
overcloud-contrailcontrolonly-2 |
control-only-1-5b3s31 |
Ironic UUID: 91dd9fa9-e8eb-4b51-8b5e-bbaffb6640e4 DMI UUID: 28AB0B57-D612-431E-B177-1C578AE0FEA4 |
5b3s31 |
10.0.0.12 |
POP1 |
overcloud-contrailcontrolonly-3 |
control-only-2-5b3s31 |
Ironic UUID: 09fa57b8-580f-42ec-bf10-a19573521ed4 DMI UUID: 09BEC8CB-77E9-42A6-AFF4-6D4880FD87D0 |
5b3s31 |
10.0.0.15 |
POP2 |
overcloud-contrailcontrolonly-4 |
control-only-1-5b3s32 |
Ironic UUID: 4766799-24c8-4e3b-af54-353f2b796ca4 DMI UUID: 3993957A-ECBF-4520-9F49-0AF6EE1667A7 |
5b3s32 |
10.0.0.13 |
POP1 |
overcloud-contrailcontrolonly-5 |
control-only-2-5b3s32 |
Ironic UUID: 58a803ae-a785-470e-9789-139abbfa74fb DMI UUID: AF92F485-C30C-4D0A-BDC4-C6AE97D06A66 |
5b3s32 |
10.0.0.16 |
POP2 |
ControlOnly preparation
Add ControlOnly overcloud VMs to overcloud KVM host
This has to be done on the overcloud KVM hosts
Two ControlOnly overcloud VM definitions will be created on each of the overcloud KVM hosts.
ROLES=control-only:2 num=4 ipmi_user=<user> ipmi_password=<password> libvirt_path=/var/lib/libvirt/images port_group=overcloud prov_switch=br0 /bin/rm ironic_list IFS=',' read -ra role_list <<< "${ROLES}" for role in ${role_list[@]}; do role_name=`echo $role|cut -d ":" -f 1` role_count=`echo $role|cut -d ":" -f 2` for count in `seq 1 ${role_count}`; do echo $role_name $count qemu-img create -f qcow2 ${libvirt_path}/${role_name}_${count}.qcow2 99G virsh define /dev/stdin <<EOF $(virt-install --name ${role_name}_${count} \ --disk ${libvirt_path}/${role_name}_${count}.qcow2 \ --vcpus=4 \ --ram=16348 \ --network network=br0,model=virtio,portgroup=${port_group} \ --network network=br1,model=virtio \ --virt-type kvm \ --cpu host \ --import \ --os-variant rhel7 \ --serial pty \ --console pty,target_type=virtio \ --graphics vnc \ --print-xml) EOF vbmc add ${role_name}_${count} --port 1623${num} --username ${ipmi_user} --password ${ipmi_password} vbmc start ${role_name}_${count} prov_mac=`virsh domiflist ${role_name}_${count}|grep ${prov_switch}|awk '{print $5}'` vm_name=${role_name}-${count}-`hostname -s` kvm_ip=`ip route get 1 |grep src |awk '{print $7}'` echo ${prov_mac} ${vm_name} ${kvm_ip} ${role_name} 1623${num}>> ironic_list num=$(expr $num + 1) done done
The generated ironic_list will be needed on the undercloud to import the nodes to Ironic.
Get the ironic_lists from the overcloud KVM hosts and combine them.
cat ironic_list_control_only 52:54:00:3a:2f:ca control-only-1-5b3s30 10.87.64.31 control-only 16234 52:54:00:31:4f:63 control-only-2-5b3s30 10.87.64.31 control-only 16235 52:54:00:0c:11:74 control-only-1-5b3s31 10.87.64.32 control-only 16234 52:54:00:56:ab:55 control-only-2-5b3s31 10.87.64.32 control-only 16235 52:54:00:c1:f0:9a control-only-1-5b3s32 10.87.64.33 control-only 16234 52:54:00:f3:ce:13 control-only-2-5b3s32 10.87.64.33 control-only 16235
Import:
ipmi_password=<password> ipmi_user=<user> DEPLOY_KERNEL=$(openstack image show bm-deploy-kernel -f value -c id) DEPLOY_RAMDISK=$(openstack image show bm-deploy-ramdisk -f value -c id) num=0 while IFS= read -r line; do mac=`echo $line|awk '{print $1}'` name=`echo $line|awk '{print $2}'` kvm_ip=`echo $line|awk '{print $3}'` profile=`echo $line|awk '{print $4}'` ipmi_port=`echo $line|awk '{print $5}'` uuid=`openstack baremetal node create --driver ipmi \ --property cpus=4 \ --property memory_mb=16348 \ --property local_gb=100 \ --property cpu_arch=x86_64 \ --driver-info ipmi_username=${ipmi_user} \ --driver-info ipmi_address=${kvm_ip} \ --driver-info ipmi_password=${ipmi_password} \ --driver-info ipmi_port=${ipmi_port} \ --name=${name} \ --property capabilities=boot_option:local \ -c uuid -f value` openstack baremetal node set ${uuid} --driver-info deploy_kernel=$DEPLOY_KERNEL --driver-info deploy_ramdisk=$DEPLOY_RAMDISK openstack baremetal port create --node ${uuid} ${mac} openstack baremetal node manage ${uuid} num=$(expr $num + 1) done < <(cat ironic_list_control_only)
ControlOnly node introspection
openstack overcloud node introspect --all-manageable --provide
Get the ironic UUID of the ControlOnly nodes
openstack baremetal node list |grep control-only | 7d758dce-2784-45fd-be09-5a41eb53e764 | control-only-1-5b3s30 | None | power off | available | False | | d26abdeb-d514-4a37-a7fb-2cd2511c351f | control-only-2-5b3s30 | None | power off | available | False | | 91dd9fa9-e8eb-4b51-8b5e-bbaffb6640e4 | control-only-1-5b3s31 | None | power off | available | False | | 09fa57b8-580f-42ec-bf10-a19573521ed4 | control-only-2-5b3s31 | None | power off | available | False | | f4766799-24c8-4e3b-af54-353f2b796ca4 | control-only-1-5b3s32 | None | power off | available | False | | 58a803ae-a785-470e-9789-139abbfa74fb | control-only-2-5b3s32 | None | power off | available | False |
The first ControlOnly node on each of the overcloud KVM hosts will be used for POP1, the second for POP2, and so and so forth.
Get the ironic UUID of the POP compute nodes:
openstack baremetal node list |grep compute | 91d6026c-b9db-49cb-a685-99a63da5d81e | compute-3-5b3s30 | None | power off | available | False | | 8028eb8c-e1e6-4357-8fcf-0796778bd2f7 | compute-4-5b3s30 | None | power off | available | False | | b795b3b9-c4e3-4a76-90af-258d9336d9fb | compute-3-5b3s31 | None | power off | available | False | | 2d4be83e-6fcc-4761-86f2-c2615dd15074 | compute-4-5b3s31 | None | power off | available | False |
The first two compute nodes belong to POP1 the second two compute nodes belong to POP2.
Create an input YAML using the ironic UUIDs:
~/subcluster_input.yaml --- - subcluster: subcluster1 asn: "65413" control_nodes: - uuid: 7d758dce-2784-45fd-be09-5a41eb53e764 ipaddress: 10.0.0.11 - uuid: 91dd9fa9-e8eb-4b51-8b5e-bbaffb6640e4 ipaddress: 10.0.0.12 - uuid: f4766799-24c8-4e3b-af54-353f2b796ca4 ipaddress: 10.0.0.13 compute_nodes: - uuid: 91d6026c-b9db-49cb-a685-99a63da5d81e vrouter_gateway: 10.0.0.1 - uuid: 8028eb8c-e1e6-4357-8fcf-0796778bd2f7 vrouter_gateway: 10.0.0.1 - subcluster: subcluster2 asn: "65414" control_nodes: - uuid: d26abdeb-d514-4a37-a7fb-2cd2511c351f ipaddress: 10.0.0.14 - uuid: 09fa57b8-580f-42ec-bf10-a19573521ed4 ipaddress: 10.0.0.15 - uuid: 58a803ae-a785-470e-9789-139abbfa74fb ipaddress: 10.0.0.16 compute_nodes: - uuid: b795b3b9-c4e3-4a76-90af-258d9336d9fb vrouter_gateway: 10.0.0.1 - uuid: 2d4be83e-6fcc-4761-86f2-c2615dd15074 vrouter_gateway: 10.0.0.1
Only control_nodes, compute_nodes, dpdk_nodes and sriov_nodes are supported.
Generate subcluster environment:
~/tripleo-heat-templates/tools/contrail/create_subcluster_environment.py -i ~/subcluster_input.yaml \ -o ~/tripleo-heat-templates/environments/contrail/contrail-subcluster.yaml
Check subcluster environment file:
cat ~/tripleo-heat-templates/environments/contrail/contrail-subcluster.yaml parameter_defaults: NodeDataLookup: 041D7B75-6581-41B3-886E-C06847B9C87E: contrail_settings: CONTROL_NODES: 10.0.0.14,10.0.0.15,10.0.0.16 SUBCLUSTER: subcluster2 VROUTER_GATEWAY: 10.0.0.1 09BEC8CB-77E9-42A6-AFF4-6D4880FD87D0: contrail_settings: BGP_ASN: '65414' SUBCLUSTER: subcluster2 14639A66-D62C-4408-82EE-FDDC4E509687: contrail_settings: BGP_ASN: '65414' SUBCLUSTER: subcluster2 28AB0B57-D612-431E-B177-1C578AE0FEA4: contrail_settings: BGP_ASN: '65413' SUBCLUSTER: subcluster1 3993957A-ECBF-4520-9F49-0AF6EE1667A7: contrail_settings: BGP_ASN: '65413' SUBCLUSTER: subcluster1 73F8D030-E896-4A95-A9F5-E1A4FEBE322D: contrail_settings: BGP_ASN: '65413' SUBCLUSTER: subcluster1 7933C2D8-E61E-4752-854E-B7B18A424971: contrail_settings: CONTROL_NODES: 10.0.0.14,10.0.0.15,10.0.0.16 SUBCLUSTER: subcluster2 VROUTER_GATEWAY: 10.0.0.1 AF92F485-C30C-4D0A-BDC4-C6AE97D06A66: contrail_settings: BGP_ASN: '65414' SUBCLUSTER: subcluster2 BB9E9D00-57D1-410B-8B19-17A0DA581044: contrail_settings: CONTROL_NODES: 10.0.0.11,10.0.0.12,10.0.0.13 SUBCLUSTER: subcluster1 VROUTER_GATEWAY: 10.0.0.1 E1A809DE-FDB2-4EB2-A91F-1B3F75B99510: contrail_settings: CONTROL_NODES: 10.0.0.11,10.0.0.12,10.0.0.13 SUBCLUSTER: subcluster1 VROUTER_GATEWAY: 10.0.0.1
Deployment
Add contrail-subcluster.yaml, contrail-ips-from-pool-all.yaml and contrail-scheduler-hints.yaml to the OpenStack deploy command:
openstack overcloud deploy --templates ~/tripleo-heat-templates \ -e ~/overcloud_images.yaml \ -e ~/tripleo-heat-templates/environments/network-isolation.yaml \ -e ~/tripleo-heat-templates/environments/contrail/contrail-plugins.yaml \ -e ~/tripleo-heat-templates/environments/contrail/contrail-services.yaml \ -e ~/tripleo-heat-templates/environments/contrail/contrail-net.yaml \ -e ~/tripleo-heat-templates/environments/contrail/contrail-subcluster.yaml \ -e ~/tripleo-heat-templates/environments/contrail/contrail-ips-from-pool-all.yaml \ -e ~/tripleo-heat-templates/environments/contrail/contrail-scheduler-hints.yaml \ --roles-file ~/tripleo-heat-templates/roles_data_contrail_aio.yaml