Contrail Deployment with Helm
This procedure describes how to deploy Contrail with Helm charts, but without OpenStack.
Nodes should be configured so the primary can ssh
into Minion. If ssh keys are needed, these should be specified in
the inventory file.
Follow these steps to deploy Contrail with Helm:
Download the file contrail-helm-deployer-release-tag.tgz onto your provisioning host. It contains the required two required Helm repositories: /opt/openstack-helm-infra (which contains code to deploy k8s) and /opt/contrail-helm-deployer.
Run the command
scp contrail-helm-deployer-release-tag.tgz
for all nodes in the cluster.Untar contrail-helm-deployer-release-tag.tgz on all nodes:
tar -zxf contrail-helm-deployer-release-tag.tgz -C /opt/
Using any node in the cluster, export the following variables:
export BASE_DIR=/opt export OSH_INFRA_PATH=${BASE_DIR}/openstack-helm-infra export CHD_PATH=${BASE_DIR}/contrail-helm-deployer
In this step, all the required packages are installed and Kubernetes is deployed. If you want to install a different version of Kubernetes or CNI, edit the file ${OSH_INFRA_PATH}/tools/gate/devel/multinode-vars.yaml. Doing this overrides the default values in ${OSH_INFRA_PATH}/playbooks/vars.yaml. Following is an example multinode-vars.yaml file, with sample values indicated for the
private_registries
section:version: kubernetes: v1.9.3 helm: v2.7.2 cni: v0.6.0 docker: # list of insecure_registries, from where you will be pulling container images insecure_registries: - "10.87.65.243:5000" # list of private secure docker registry auth info, from where you will be pulling container images #private_registries: # - name: docker-registry-name # username: username@abc.xyz # email: username@abc.xyz # password: password # secret_name: contrail-image-secret # namespace: openstack kubernetes: network: default_device: ens3 cluster: cni: calico pod_subnet: 192.168.0.0/16 domain: cluster.local
Install the dependent packages using
sudo apt-get
.sudo apt-get update sudo apt-get install --no-install-recommends -y ca-certificates make jq nmap curl uuid-runtime ipcalc linux-headers-$(uname -r)
Prepare the nodes definition in $OSH_INFRA_PATH/tools/gate/devel/multinode-inventory.yaml, similar to this example:
all: children: primary: hosts: controller1: ansible_port: 22 ansible_host: 10.10.0.1 ansible_user: root ansible_ssh_extra_args: -o StrictHostKeyChecking=no ansible_ssh_private_key_file: /path/to/ssh/key/file nodes: hosts: controller2: ansible_port: 22 ansible_host: 10.10.0.2 ansible_user: root ansible_ssh_extra_args: -o StrictHostKeyChecking=no ansible_ssh_private_key_file: /path/to/ssh/key/file
Deploy k8s to the nodes and use the
kubectl get nodes
command to verify the deployment is successful.cd ${OSH_INFRA_PATH} make dev-deploy setup-host multinode make dev-deploy k8s multinode nslookup kubernetes.default.svc.cluster.local || /bin/true kubectl get nodes -o wide
Set the correct labels for the nodes.
kubectl label node controller1.localdomain --overwrite openstack-compute-node=disable kubectl label node controller1.localdomain opencontrail.org/controller=enabled kubectl label node controller2.localdomain --overwrite openstack-compute-node=disable kubectl label node controller2.localdomain opencontrail.org/controller=enabled
Deploy the OpenContrail charts.
cd $CHD_PATH make # Change k8s rbac settings kubectl replace -f ${CHD_PATH}/rbac/cluster-admin.yaml
Prepare the values for Contrail in /tmp/contrail.yml, similar to the following example.
Note:This example uses bash variables you should replace with exact values using any preferred means (sed, eval, cat, and so on). Similarly, replace the other variables with actual values where indicated, including
IPDATA_SERVICE_HOST
,METADATA_PROXY_SECRET
, and keystone IP/VIP details.global: images: tags: kafka: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-external-kafka:${CONTRAIL_TAG:-latest}" cassandra: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-external-cassandra:${CONTRAIL_TAG:-latest}" redis: "redis:4.0.2" zookeeper: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-external-zookeeper:${CONTRAIL_TAG:-latest}" contrail_control: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-control-control:${CONTRAIL_TAG:-latest}" control_dns: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-control-dns:${CONTRAIL_TAG:-latest}" control_named: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-control-named:${CONTRAIL_TAG:-latest}" config_api: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-config-api:${CONTRAIL_TAG:-latest}" config_devicemgr: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-config-devicemgr:${CONTRAIL_TAG:-latest}" config_schema_transformer: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-config-schema:${CONTRAIL_TAG:-latest}" config_svcmonitor: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-config-svcmonitor:${CONTRAIL_TAG:-latest}" webui_middleware: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-webui-job:${CONTRAIL_TAG:-latest}" webui: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-webui-web:${CONTRAIL_TAG:-latest}" analytics_api: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-analytics-api:${CONTRAIL_TAG:-latest}" contrail_collector: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-analytics-collector:${CONTRAIL_TAG:-latest}" analytics_alarm_gen: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-analytics-alarm-gen:${CONTRAIL_TAG:-latest}" analytics_query_engine: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-analytics-query-engine:${CONTRAIL_TAG:-latest}" analytics_snmp_collector: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-analytics-snmp-collector:${CONTRAIL_TAG:-latest}" contrail_topology: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-analytics-topology:${CONTRAIL_TAG:-latest}" build_driver_init: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-vrouter-kernel-build-init:${CONTRAIL_TAG:-latest}" vrouter_agent: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-vrouter-agent:${CONTRAIL_TAG:-latest}" vrouter_init_kernel: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-vrouter-kernel-init:${CONTRAIL_TAG:-latest}" vrouter_dpdk: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-vrouter-agent-dpdk:${CONTRAIL_TAG:-latest}" vrouter_init_dpdk: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-vrouter-kernel-init-dpdk:${CONTRAIL_TAG:-latest}" nodemgr: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-nodemgr:${CONTRAIL_TAG:-latest}" contrail_status: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-status:${CONTRAIL_TAG:-latest}" node_init: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-node-init:${CONTRAIL_TAG:-latest}" dep_check: quay.io/stackanetes/kubernetes-entrypoint:v0.2.1 contrail_env: CONTROLLER_NODES: 10.10.0.1,10.10.0.2 LOG_LEVEL: SYS_DEBUG CLOUD_ORCHESTRATOR: openstack JVM_EXTRA_OPTS: "-Xms1g -Xmx2g" BGP_PORT: "1179" CONFIG_DATABASE_NODEMGR__DEFAULTS__minimum_diskGB: "2" DATABASE_NODEMGR__DEFAULTS__minimum_diskGB: "2" IPFABRIC_SERVICE_HOST: metadata IP of old OpenStack setup METADATA_PROXY_SECRET: metadata proxy secret of old OpenStack setup endpoints: keystone: auth: username: admin password: password project_name: admin user_domain_name: admin_domain project_domain_name: admin_domain region_name: RegionOne hosts: default: keystone IP/VIP path: default: /v3 port: admin: default: 35357 api: default: 5000 scheme: default: http host_fqdn_override: default: keystone IP/VIP namespace: null
If you are using a private registry, add the username and password under the
imageCredentials
section as follows:global: images: imageCredentials: registry: ${CONTRAIL_REGISTRY:-opencontrailnightly} username: ${CONTRAIL_REG_USERNAME} password: ${CONTRAIL_REG_PASSWORD}
Finally, deploy the Contrail charts:
helm install --name contrail-thirdparty ${CHD_PATH}/contrail-thirdparty --namespace=contrail --values=/tmp/contrail.yaml helm install --name contrail-analytics ${CHD_PATH}/contrail-analytics --namespace=contrail --values=/tmp/contrail.yaml helm install --name contrail-controller ${CHD_PATH}/contrail-controller --namespace=contrail --values=/tmp/contrail.yaml
After all containers are deployed, you can check cluster status
using the contrail-status
command. You can also use the
Contrail web browser interface to view and verify the cluster status.