Kubernetes Network Policy
The Kubernetes networking model requires all pods to be able to access all other pods by default. This is called a flat network because it follows an allow-any-any model. It significantly simplifies the design and implementation of Kubernetes networking and makes it much more scalable.
Chapter 4 details the requirements that Kubernetes enforces on network implementations.
Security is an important concern. In reality, in many cases a certain level of network segmentation methods is required to ensure that only certain pods can talk to each other, and that is when Kubernetes network policy comes into the picture. A Kubernetes network policy defines the access permissions for groups of pods the same way a security group in the cloud is used to control access to VM instances.
Kubernetes supports network policy via the NetworkPolicy object, which is a Kubernetes resource just like pod, service, ingress, and many others you’ve learned about earlier in this chapter. The role of the Network Policy object is to define how groups of pods are allowed to communicate with each other.
Let’s examine how Kubernetes network policy works:
1Initially, in a Kubernetes cluster, all pods are non-isolated by default and they work in an allow-any-any model so any pod can talk to any other pod.
2. Now apply a network policy named policy1 to pod A. In policy policy1 you define a rule to explicitly allow pod A to talk to pod B. In this case let’s call pod A a target pod because it is the pod that the network policy will act on.
3. From this moment on, a few things happen:
Target pod A can talk to pod B, and can talk to pod B only, because B is the only pod you allowed in the policy. Due to the nature of the policy rules, you can call the rule a whitelist.
For target pod A only, any connections that are not explicitly allowed by the whitelist of this network policy policy1 will be rejected. You don’t need to explicitly define this in policy1, because it will be enforced by the nature of Kubernetes network policy. Let’s call this implicit policy the deny all policy.
As for other non-targeted pods, for example, pod B or pod C, which are not applied with policy1, nor to any other network policies, will continue to follow the allow-any-any model. Therefore they are not affected and can continue to communicate to all other pods in the cluster. This is another implicit policy, an allow all policy.
Assuming you also want pod A to be able to communicate to pod C, you need to update the network policy policy1 and its rules to explicitly allow it. In other words, you need to keep updating the whitelist to allow more traffic types.
As you can see, when you define a policy, at least three policies will be applied in the cluster:
Explicit policy1: This is the network policy you defined, with the whitelist rules allowing certain types of traffic for the selected (target) pod.
An implicit deny all network policy: This denies all other traffic that is not in the whitelist of the target pod.
An implicit allow all network policy: This allows all other traffic for the other non-targeted pods that are not selected by policy1. We’ll see deny all and allow all policies again in Chapter 8.
Here are some highlights of the Kubernetes network policy.
Pod specific: Network policy specification applies to one pod or a group of pods based on label, same way as rc or Deploy do.
Whitelist-based rules: explicit rules that compose a whitelist, and each rule describes a certain type of traffic to be allowed. All other traffic not described by any rules in the whitelist will be dropped for the target pod.
Implicit allow all: A pod will be affected only if it is selected as the target by a network policy, and it will be affected only by the selecting network policy. The absence of a network policy applied on a pod indicates an implicit allow all policy to this pod. In other words, if a non-targeted pod continues its allow-any-any networking model.
Separation of ingress and egress: Policy rules need to be defined for a specific direction. The direction can be Ingress, Egress, none, or both.
Flow-based (vs. packet-based): Once the initiating packet is allowed, the return packet in the same flow will also be allowed. For example, suppose an ingress policy applied on pod A allows an ingress HTTP request, then the whole HTTP interaction will be allowed for pod A. This includes the three-way TCP connection establishment and all data and acknowledgments in both directions.
Network policies are implemented by the network component, so you must be using a network solution that supports network policy. Simply creating the NetworkPolicy resource without a controller to implement it will have no effect. In this book Contrail is such a network component with network policy implemented. In Chapter 8, you’ll see how these network policies work in Contrail.
Network Policy Definition
Like all other objects in Kubernetes, network policy can be defined in a YAML file. Let’s look at an example (the same example will be used in Chapter 8):
#policy1-do.yaml apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: policy1 namespace: dev spec: podSelector: matchLabels: app: webserver-dev policyTypes: - Ingress - Egress ingress: - from: - ipBlock: cidr: 10.169.25.20/32 - namespaceSelector: matchLabels: project: jtac - podSelector: matchLabels: app: client1-dev ports: - protocol: TCP port: 80 egress: - to: - podSelector: matchLabels: app: dbserver-dev ports: - protocol: TCP port: 80
Let’s look at the spec part of this YAML file since the other sections are somewhat self-explanatory. The spec has the following structure:
spec: podSelector: ...... policyTypes: - Ingress - Egress ingress: - from: ...... egress: - to: ......
Here you can see that a network policy definition YAML file can logically be divided into four sections:
podSelector: This defines the pods selection. It identifies the pods to which the current network policy will be applied.
policyTypes: Specifies the type of policy rules: Ingress, Egress or both.
ingress: Defines the ingress policy rules for the target pods.
egress: Defines the egress policy rules for the target pods.
Next we’ll look at each section in more detail.
Selecting Target Pods
When you define a network policy, Kubernetes needs to know which pods you want this policy to act on. Similar to how service selects its backend pods, the network policy selects pods to which it will be applied based on labels:
podSelector: matchLabels: app: webserver-dev
Here, all pods that have the label
app: webserver-dev are selected to be the target pods by the network policy. All of
the following content in spec will apply to only the target pods.
The second section defines the
policyTypes for the target pods:
policyTypes: - Ingress - Egress
PolicyTypes can either be ingress, egress, or both. And both types define specific traffic types in the form of one or more rules, as discussed next.
The ingress and egress sections define the direction of traffic, from the selected target pods’ perspective. For example, consider the following simplified example:
ingress: - from: - podSelector: matchLabels: app: client1-dev ports: - protocol: TCP port: 80 egress: - to: - podSelector: matchLabels: app: client1-dev ports: - protocol: TCP port: 8080
Assuming the target pod is webserver-dev pod, and there is only one pod client1-dev in the cluster having a matching label client1-dev, two things will happen:
The ingress direction: the pod webserver-dev can accept a TCP session with a destination port 80, initiated from pod client1-dev. This explains why we said Kubernetes network policy is flow-based instead of packet-based. The TCP connection could not be established if the policy would have been packet-based designed because on receiving the incoming TCP sync, the returning outgoing TCP sync-ack would have been rejected without a matching egress policy.
The egress direction: pod webserver-dev can initiate a TCP session with a destination port 8080, towards pod client1-dev.
For the egress connection to go through, the other end needs to define an ingress policy to allow the incoming connection.
Network Policy Rules
Each from or to statement defines a rule in the network policy:
A from statement defines an ingress policy rule.
A to statement defines an egress policy rule
Both rules can optionally have ports statements, which will be discussed later.
So you can define multiple rules to allow complex traffic modes for each direction:
ingress: INGRESS RULE1 INGRESS RULE2 egress: EGRESS RULE1 EGRESS RULE2
Each rule identifies the network endpoints where the target pods can communicate. Network endpoints can be identified by different methods:
ipBlock: Selects pods based on an IP address block.
namespaceSelector: Selects pods based on the label of the namespace.
podSelector: Selects pods based on label of the pod.
The podSelector selects different things when it is used in different places of a YAML file. Previously (under spec) it selected pods that the network policy applies to, which we’ve called target pods. Here, in a rule (under from or to), it selects which pods the target pod is communicating with. Sometimes we call these pods peering pods, or endpoints.
So the YAML structure for a rule can look like this:
ingress: - from: - ipBlock: ..... - namespaceSelector: ..... - podSelector: ..... ports: ......
ingress: - from: - ipBlock: cidr: 10.169.25.20/32 - namespaceSelector: matchLabels: project: jtac - podSelector: matchLabels: app: client1-dev ports: - protocol: TCP port: 80 egress: - to: - podSelector: matchLabels: app: dbserver-dev ports: - protocol: TCP port: 80
Here, the ingress network endpoints are subnet 10.169.25.20/32; or all pods in namespaces that have the label project: jtac; or pods which have the label app: client1-dev in current namespace (namespace of target pod), and the egress network point is pod dbserver-dev. We’ll come to the ports part soon.
AND versus OR
It’s also possible to specify only a few pods from namespaces, instead of communicating with all pods. In our example, podSelector is used all along, which assumes the same namespace as the target pod. Another method is to use podSelector along with a namespaceSelector. In that case, the namespaces that the pods belong to are those with matching labels with namespaceSelector, instead of the same as the target pod’s namespace.
For example, assuming that the target pod is webserver-dev and its namespace is dev, and only namespace qa has a label project=qa matching to the namespaceSelector:
ingress: - from: - namespaceSelector: matchLabels: project: qa podSelector: matchLabels: app: client1-qa
Here, the target pod can only communicate with those pods that
are in namespace qa, AND (not OR) with the label
Be careful here because it is totally different than the definition
below, which allows the target pod to talk to those pods that are:
qa, OR (not AND) with label
app: client1-qa in the target pod’s namespace
ingress: - from: - namespaceSelector: matchLabels: project: qa - podSelector: matchLabels: app: client1-qa
Protocol and Ports
It is also possible to specify ports for an ingress and egress rule. The protocol type can also be specified along with a protocol port. For example:
egress: - to: - podSelector: matchLabels: app: dbserver-dev ports: - protocol: TCP port: 80
The ports in ingress say that the target pods can allow incoming traffic for the specified ports and protocol. Ports in egress say that target pods can initiate traffic to specified ports and protocol. If port are not mentioned, all ports and protocols are allowed.
Let’s look at our example again in detail:
podSelector: matchLabels: app: webserver-dev policyTypes: - Ingress - Egress ingress: - from: - ipBlock: cidr: 10.169.25.20/32 - namespaceSelector: matchLabels: project: jtac - podSelector: matchLabels: app: client1-dev ports: - protocol: TCP port: 80 egress: - to: - podSelector: matchLabels: app: dbserver-dev ports: - protocol: TCP port: 80
You should now know exactly what the network policy is trying to enforce.
Lines 1-3: pod webserver-dev is selected by the policy, so it is the target pod; all following policy rules will apply on it, and on it alone.
Lines 4-6: the policy will define rules for both Ingress and Egress traffic.
Lines 7-19: ingress: section defines the ingress policy.
Line 8: from: and line 17: ports, these two sections define one policy rule on ingress policy.
Lines 9-16: these eight lines under the from: section compose an ingress whitelist:
Lines 9-10: any incoming data with source IP being 10.169.25.20/32 can access the target pod webserver-dev.
Lines 11-13: any pods under namespace jtac can access target pod webserver-dev.
Lines 14-16: any pods with label client1-dev can access target pod webserver-dev.
Lines 17-19: ports section is second (and optional) part of the same policy rule. Only TCP port 80 (web service) on target pod webserver-dev is exposed and accessible. Access to all other ports will be denied.
Lines 20-26: egress: section defines the egress policy.
Lines 21: to: and line 24: ports, these two sections define one policy rule in egress policy.
Lines 21-24: these four lines under to: section compose an egress whitelist, here the target pod can send egress traffic to pod dbserver-dev.
Line 25: ports section is second part of the same policy rule. The target pod webserver-pod can only start TCP session with a destination port of 80 to other pods.
And that’s not all. If you remember at the beginning of this chapter, we talked about the Kubernetes default allow-any-any network model and the implicit deny-all, allow-all policies, you will realize that so far we just explained the explicit part of it (policy1 in our network policy introduction section). After that, there are two more implicit policies:
The deny all network policy: for the target pod webserver-dev, deny all other traffic that is other than what is explicitly allowed in the above whitelists, this implies at least two rules:
ingress: deny all incoming traffic destined to the target pod webserver-dev, other than what is defined in the ingress whitelist.
egress: deny all outgoing traffic sourcing from the target pod webserver-dev, other than what is defined in the egress whitelist.
An allow all network policy allows all traffic for other pods that are not target of this network policy, on both ingress and egress direction.
In Chapter 8 we’ll take a more in depth look at these implicit network policies and their rules in Contrail implementation.
Create Network Policy
You can create and verify the network policy the same way that you create other Kubernetes objects:
$ kubectl apply -f policy1-do.yaml networkpolicy.networking.k8s.io/policy1-do created $ kubectl get netpol -n dev NAME POD-SELECTOR AGE policy1 app=webserver-dev 6s $ kubectl describe netpol policy -n dev Name: policy1 Namespace: dev Created on: 2019-10-01 11:18:19 -0400 EDT Labels: <none> Annotations: <none> Spec: PodSelector: app=webserver-dev Allowing ingress traffic: To Port: 80/TCP From: IPBlock: CIDR: 10.169.25.20/32 Except: From: NamespaceSelector: project=jtac From: PodSelector: app=client1-dev Allowing egress traffic: To Port: 80/TCP To: PodSelector: app=dbserver-dev Policy Types: Ingress, Egress
In Chapter 8 we’ll set up a test environment to verify the effect of this network policy in more detail.