Contrail Namespaces and Isolation
In Chapter 3 you read about namespace
or NS in Kubernetes, and at the beginning of this chapter we mentioned
object mappings between Kubernetes and Contrail. In this section you’ll
see how namespace works in Contrail environments and how Contrail
extends the feature set even further.
One analogy given when introducing the namespace
concept is OpenStack project
, or tenant
. And that is exactly how Contrail is looking
at it. Whenever a new namespace
object
is created, contrail-kube-manager
(KM)
gets a notice about the object creation event and it will create the
corresponding project in Contrail.
To differentiate between multiple Kubernetes clusters in Contrail,
a Kubernetes cluster name will be added to the Kubernetes namespace
or project name. The default Kubernetes cluster name is k8s
. So if you create a Kubernetes namespace ns-user-1
, a k8s-ns-user-1
project will be created in Contrail just as you can see here in Figure 1, which shows
the Contrail GUI.
The Kubernetes cluster name
is configurable,
but only during the deployment process. If you don’t configure
it k8s
will be the default. Once the cluster
is created, the name cannot be changed. To view the cluster
name
, you have to go to the contrail-kube-manager
(KM) docker and check its configuration file.
To locate the KM docker container:
$ docker ps -a | grep kubemanager 2260c7845964 ...snipped... ago Up 2 minutes kubemanager_kubemanager_1
To log in to the KM container:
$ docker exec -it kubemanager_kubemanager_1 bash
To find the cluster_name
option:
$ grep cluster /etc/contrail/contrail-kubernetes.conf cluster_name=k8s #<--- cluster_project={} cluster_network={}
The rest of this book will refer to all these terms namespace, NS, tenant,
and project
interchangeably.
Non-Isolated Namespaces
You should be aware that one Kubernetes basic networking requirement is for a flat/NAT-less network – any pod can talk to any pod in any namespace – and any CNI provider must ensure that. Consequently, in Kubernetes, by default, all namespaces are not isolated:
The term isolated and non-isolated are in the context of (Contrail) networking only.
k8s-default-pod-network and k8s-default-service-network
To provide networking for all non-isolated namespaces, there should be a common VRF (virtual routing and forwarding) table or routing instance. In the Contrail Kubernetes environment, two default virtual networks are pre-configured in k8s’ default namespace, for pod and for service, respectively. Correspondingly, there are two VRF tables, each with the same name as their corresponding virtual network.
The name of the two virtual networks/VRF tables is in this format:
<k8s-cluster-name>-<namespace name>-[pod|service]-network
So, for the default namespace with a default cluster name, k8s
, the two Virtual network/VRF table names are:
k8s-default-pod-network
: the pod virtual network/VRF table, with the default subnet10.32.0.0/12
k8s-default-service-network
: the service virtual network /VRF table, with a default subnet10.96.0.0/12
The default subnet for pod or service is configurable.
It is important to know that these two default virtual networks are shared between all of the non-isolated namespaces. What that means is that they will be available for any new non-isolated namespace that you create, implicitly. That’s why pods from all non-isolated namespaces, including default namespaces, can talk to each other.
On the other hand, any virtual networks that you create will be isolated with other virtual networks, regardless of the same or different namespaces. Communication between pods in two different virtual networks requires Contrail network policy.
Later, when you read about Kubernetes service , you may wonder why packets destined for the service virtual network/VRF table can reach the backend pod in pod virtual network/VRF table. Again, the good news is because of Contrail network policy. By default, Contrail network policy is enabled between the service and pod networks, which allows packets arriving to the service virtual network/VRF table to reach the pod, and vice versa.
Isolated Namespaces
In contrast, isolated namespaces have their own default pod-network
and service-network, and accordingly, two new VRF tables are also
created for each isolated namespace. The same flat-subnets 10.32.0.0/12
and 10.96.0.0/12
are shared by the pod and service networks in the isolated namespaces.
However, since the networks are with a different VRF table, by default
it is isolated with another namespace. Pods launched in isolated namespaces
can only talk to service and pods on the same namespace. Additional
configurations, for example, policy, are required to make the pod
able to reach the network outside of the current namespace.
To illustrate this concept, let’s use an example. Suppose
you have three namespaces: the default
namespace,
and two user namespaces: ns-non-isolated
and ns-isolated
. In each namespace you
can create one user virtual network: vn-left-1
. You will end up following virtual network/VRF tables in Contrail:
default-domain:k8s-default:k8s-default-pod-network
default-domain:k8s-default:k8s-default-service-network
default-domain:k8s-default:k8s-vn-left-1-pod-network
default-domain:k8s-ns-non-isolated:k8s-vn-left-1-pod-network
default-domain:k8s-ns-isolated:k8s-ns-isolated-pod-network
default-domain:k8s-ns-isolated:k8s-ns-isolated-service-network
default-domain:k8s-ns-isolated:k8s-vn-left-1-pod-network
The above names are listed in FQDN format. In Contrail, domain is the top-level object, followed by project/tenant, and then followed by virtual networks.
Figure 2 expertly illustrates all this.
Here is the YAML file to create an isolated namespace:
$ cat ns-isolated.yaml apiVersion: v1 kind: Namespace metadata: annotations: "opencontrail.org/isolation" : "true" name: ns-isolated
And to create the NS:
kubectl create -f ns-isolated.yaml $ kubectl get ns NAME STATUS AGE contrail Active 8d default Active 8d ns-isolated Active 1d #<--- kube-public Active 8d kube- system Active 8d
The annotations under metadata are an additional way to compare standard (non-isolated) k8s namespace. The value of true indicates this is an isolated namespace:
annotations: "opencontrail.org/isolation" : "true"
You can see that this part of the definition is Juniper’s
extension. The contrail-kube-manager (KM)
reads the namespace metadata
from kube-apiserver,
parses the information defined in the annotations
object, and sees that the isolation
flag
is set to true. It then creates the tenant with the corresponding
routing instance (one for pod and one for service) instead of using
the default namespace routing instances for the isolated namespace.
Fundamentally that is how the isolation is implemented.
The following sections will verify that the routing isolation is working.
Pods Communication Across NS
Create a non-isolated namespace and an isolated namespace:
$ cat ns-non-isolated.yaml apiVersion: v1 kind: Namespace metadata: name: ns-non-isolated $ cat ns-isolated.yaml apiVersion: v1 kind: Namespace metadata: annotations: "opencontrail.org/isolation": "true" name: ns-isolated $ kubectl apply -f ns-non-isolated.yaml namespace/ns-non-isolated created $ kubectl apply -f ns-isolated.yaml namespace/ns-isolated created $ kubectl get ns | grep isolate ns-isolated Active 79s ns-non-isolated Active 73s
In both namespaces and the default namespace, create a deployment to launch a webserver pod:
#deploy-webserver-do.yaml apiVersion: apps/v1 - {key: app, operator: In, values: [webserver]} $ kubectl apply -f deploy-webserver-do.yaml -n default deployment.extensions/webserver created $ kubectl apply -f deploy-webserver-do.yaml -n ns-non-isolated deployment.extensions/webserver created $ kubectl apply -f deploy-webserver-do.yaml -n ns-isolated deployment.extensions/webserver created $ kubectl get pod -o wide -n default NAME READY STATUS ... IP NODE ... webserver-85fc7dd848-tjfn6 1/1 Running ... 10.47.255.242 cent333 ... $ kubectl get pod -o wide -n ns-non-isolated... NAME READY STATUS ... IP NODE ... webserver-85fc7dd848-nrxq6 1/1 Running ... 10.47.255.248 cent222 ... $ kubectl get pod -o wide -n ns-isolated NAME READY STATUS ... IP NODE ... webserver-85fc7dd848-6l7j2 1/1 Running ... 10.47.255.239 cent222 ...
Ping between all pods in three namespaces:
#default ns to non-isolated new ns: succeed $ kubectl -n default exec -it webserver-85fc7dd848-tjfn6 -- ping 10.47.255.248 PING 10.47.255.248 (10.47.255.248): 56 data bytes 64 bytes from 10.47.255.248: seq=0 ttl=63 time=1.600 ms ^C --- 10.47.255.248 ping statistics --- 1 packets transmitted, 1 packets received, 0% packet loss round-trip min/avg/max = 1.600/1.600/1.600 ms #default ns to isolated new ns: fail $ kubectl -n default exec -it webserver-85fc7dd848-tjfn6 -- ping 10.47.255.239 PING 10.47.255.239 (10.47.255.239): 56 data bytes ^C --- 10.47.255.239 ping statistics --- 3 packets transmitted, 0 packets received, 100% packet loss
The test result shows that bidirectional communication between
two non-isolated namespaces (namespace ns-non-isolated
and default
, in this case) works, but
traffic from a non-isolated namespace (default
ns) toward an isolated namespace does not pass through. What about
traffic within the same isolated namespace?
With the power of deployment
you
can quickly test it out: in isolated namespace ns-isolated
, clone one more pod by scale the deployment with replicas=2
and ping between the two pods:
$ kubectl scale deployment webserver --replicas=2 $ kubectl get pod -o wide -n ns-isolated NAME READY STATUS RESTARTS AGE IP NODE webserver-85fc7dd848-6l7j2 1/1 Running 0 8s 10.47.255.239 cent222 webserver-85fc7dd848-215k8 1/1 Running 0 8s 10.47.255.238 cent333 $ kubectl -n ns-isolated exec -it webserver-85fc7dd848-6l7j2 -- ping 10.47.255.238 PING 10.47.255.238 (10.47.255.238): 56 data bytes 64 bytes from 10.47.255.238: seq=0 ttl=63 time=1.470 ms ^C --- 10.47.255.238 ping statistics --- 1 packets transmitted, 1 packets received, 0% packet loss round-trip min/avg/max = 1.470/1.470/1.470 ms
The ping packet passes through now. To summarize the test results:
Traffic is not isolated between non-isolated namespace.
Traffic is isolated between an isolated namespace and all other tenants in the cluster.
Traffic is not isolated in the same namespace.
Pod-level isolation can be achieved via Kubernetes network policy, or security groups in Contrail, all covered later in this chapter.