Deploy Kubevirt DPDK Dataplane Support for VMs
SUMMARY Cloud-Native Contrail® Networking™ supports the deployment of the vRouter DPDK dataplane (Kubevirt) for high-performance VM and container networking in Kubernetes.
Kubevirt Overview
Kubevirt is an open source Kubernetes project that enables the management (scheduling) of virtual machine workloads (VMs) alongside container workloads within a Kubernetes cluster. Kubevirt provides a unified development platform where developers build, modify, and deploy applications residing in both application containers and VMs within a common, shared environment. Kubevirt provides the following additional functions to your Kubernetes cluster:
- Kubevirt adds additional types of pods, or Custom Resource Definitions (CRDs), to the Kubernetes API server
- Additional controllers for cluster-wide logic to support the new types of pods
- Additional daemons for node-specific logic to support the new types of pods
As a result of this new functionality, Kubevirt creates and manages
VirtualMachineInstance (VMI) objects. VMIs contain a workload controller
called a VirtualMachine (VM). The VM maintains the persistent state of it’s
VMI. This enables users to terminate and initiate VMs at another time with no change in data
or state. Additionally, you can deploy Kubevirt on top of a Kubernetes cluster which lets
you manage traditional container workloads along with VMIs managed by Kubevirt. VMs have
access to Kubernetes cluster features with no additional permissions required.
Kubevirt DPDK Implementation
Kubevirt does not typically support user space networking for fast packet processing. In
Cloud-Native Contrail Networking however, enhancements enable Kubevirt to support
vhostuser interface types for VMs. These interfaces perform user space
networking with the DPDK vRouter and give pods access to the increased performance and
packet processing the DPDK vRouter provides.
The following are some of the benefits of the DPDK vRouter application:
- Packet processing occurs in user space and bypasses kernel space. This increases packet-processing efficiency.
- Kernel interrupts and context switches do not occur because packets bypass kernel space. This results in less CPU overhead and increased data throughput.
- DPDK enhances the forwarding plane of the vRouter in user space, increasing performance.
- DPDK Lcores run in poll mode. This enables the Lcores to receive and process packets immediately upon receiving them.
Deploy Kubevirt
Prerequisites
You must have an active Kubernetes cluster and the ability to use the
kubectl client in order to deploy Kubevirt.
Deploy the Cloud-Native Contrail Networking Kubevirt Fork
The current Kubevirt release (v0.48.0) doesn't support enhancements to enable the
vhostuser interface. Juniper maintains a Kubevirt fork that supports
this DPDK interface. This Kubevirt fork is for use in environments running Cloud-Native
Contrail Networking. Changes to the fork are committed on top of Kubevirt release version
(v0.40.0) and tagged/released as version (v.0.40.0-jnpr) from the fork.
The Kubevirt operator and Kubevirt CR are also released from the fork repository. The Kubevirt operator manages the life cycle of core Kubevirt components. The Kubevirt operator and CR enable virtualization in your cluster.
Use the following commands to deploy the Kubevirt fork, Kubevirt CR, and Kubevirt operator. Note the release version (v0.40.0) and installation file paths.
export RELEASE=v0.40.0-jnpr
kubectl apply -f https://github.com/cijohnson/kubevirt/releases/download/${RELEASE}/kubevirt-operator.yaml
kubectl apply -f https://github.com/cijohnson/kubevirt/releases/download/${RELEASE}/kubevirt-cr.yaml
kubectl -n kubevirt wait kv kubevirt --for condition=Available
Cloud-Native Contrail Networking Kubevirt fork.
Launch a VM Alongside a Container
With Kubevirt, launching and managing a VM in Kubernetes is similar to deploying a pod. You
can create a VM object using kubectl. After creating a VM object, that VM
is active and running in your cluster.
Use the following high-level steps to launch a VM alongside a container:
- Create a
VirtualNetwork - Launch a VM
Launch a VM
The following VirtualMachine specs are examples of
VirtualMachine instances with a varying amount of interfaces.
- Single
vhostuserinterface VM:apiVersion: kubevirt.io/v1alpha3 kind: VirtualMachine metadata: name: vm-single-virtio namespace: contrail spec: running: true template: metadata: labels: kubevirt.io/size: small kubevirt.io/domain: vm-single-virtio app: vm-single-virtio-app spec: nodeSelector: master: master terminationGracePeriodSeconds: 30 domain: cpu: sockets: 1 cores: 8 threads: 2 #dedicatedCpuPlacement: true memory: hugepages: pageSize: "2Mi" resources: requests: memory: "512Mi" devices: disks: - name: containerdisk disk: bus: virtio - name: cloudinitdisk disk: bus: virtio interfaces: - name: default bridge: {} - name: vhost-user-vn-blue vhostuser: {} useVirtioTransitional: true networks: - name: default pod: {} - name: vhost-user-vn-blue multus: networkName: vn-blue volumes: - name: containerdisk containerDisk: image: svl-artifactory.juniper.net/atom-docker/dpdk-pktgen/vmdisks/dpdk-pktgen-auto:latest - name: cloudinitdisk cloudInitNoCloud: userDataBase64: SGkuXG4= - Multi
vhostuserinterface:apiVersion: kubevirt.io/v1alpha3 kind: VirtualMachine metadata: name: vm-multi-virtio namespace: contrail spec: running: true template: metadata: labels: kubevirt.io/size: small kubevirt.io/domain: vm-multi-virtio app: vm-multi-virtio-app spec: nodeSelector: worker: worker terminationGracePeriodSeconds: 30 domain: cpu: sockets: 1 cores: 8 threads: 2 #dedicatedCpuPlacement: true memory: hugepages: pageSize: "2Mi" resources: requests: memory: "512Mi" devices: disks: - name: containerdisk disk: bus: virtio - name: cloudinitdisk disk: bus: virtio interfaces: - name: default bridge: {} - name: vhost-user-vn-blue vhostuser: {} - name: vhost-user-vn-green vhostuser: {} useVirtioTransitional: true networks: - name: default pod: {} - name: vhost-user-vn-blue multus: networkName: vn-blue - name: vhost-user-vn-green multus: networkName: vn-green volumes: - name: containerdisk containerDisk: image: svl-artifactory.juniper.net/atom-docker/dpdk-pktgen/vmdisks/dpdk-pktgen-auto:latest - name: cloudinitdisk cloudInitNoCloud: userDataBase64: SGkuXG4= - Bridge/
vhostuserinterface VM:apiVersion: kubevirt.io/v1alpha3 kind: VirtualMachine metadata: name: vm-virtio-veth namespace: contrail spec: running: true template: metadata: labels: kubevirt.io/size: small kubevirt.io/domain: vm-virtio-veth app: vm-virtio-veth-app spec: nodeSelector: master: master terminationGracePeriodSeconds: 30 domain: cpu: sockets: 1 cores: 8 threads: 2 #dedicatedCpuPlacement: true memory: hugepages: pageSize: "2Mi" resources: requests: memory: "512Mi" devices: disks: - name: containerdisk disk: bus: virtio - name: cloudinitdisk disk: bus: virtio interfaces: - name: default bridge: {} - name: vhost-user-vn-blue vhostuser: {} - name: vhost-user-vn-green bridge: {} useVirtioTransitional: true networks: - name: default pod: {} - name: vhost-user-vn-blue multus: networkName: vn-blue - name: vhost-user-vn-green multus: networkName: vn-green volumes: - name: containerdisk containerDisk: image: svl-artifactory.juniper.net/atom-docker/dpdk-pktgen/vmdisks/dpdk-pktgen-auto:latest - name: cloudinitdisk cloudInitNoCloud: userDataBase64: SGkuXG4=
Create a Virtual Network
The following net-attach-def object is an example of a virtual
network.
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
name: vn-blue
namespace: contrail
annotations:
juniper.net/networks: '{
"ipamV4Subnet": "19.1.1.0/24"
}'
labels:
vn: vn-blue-vn-green
spec:
config: '{
"cniVersion": "0.3.1",
"name": "nad-blue",
"type": "contrail-k8s-cni"
}'
Launch a VM
The following VirtualMachine specs are examples of
VirtualMachine instances with a varying amount of interfaces.
- Single
vhostuserinterface VM:apiVersion: kubevirt.io/v1alpha3 kind: VirtualMachine metadata: name: vm-single-virtio namespace: contrail spec: running: true template: metadata: labels: kubevirt.io/size: small kubevirt.io/domain: vm-single-virtio app: vm-single-virtio-app spec: nodeSelector: master: master terminationGracePeriodSeconds: 30 domain: cpu: sockets: 1 cores: 8 threads: 2 #dedicatedCpuPlacement: true memory: hugepages: pageSize: "2Mi" resources: requests: memory: "512Mi" devices: disks: - name: containerdisk disk: bus: virtio - name: cloudinitdisk disk: bus: virtio interfaces: - name: default bridge: {} - name: vhost-user-vn-blue vhostuser: {} useVirtioTransitional: true networks: - name: default pod: {} - name: vhost-user-vn-blue multus: networkName: vn-blue volumes: - name: containerdisk containerDisk: image: svl-artifactory.juniper.net/atom-docker/dpdk-pktgen/vmdisks/dpdk-pktgen-auto:latest - name: cloudinitdisk cloudInitNoCloud: userDataBase64: SGkuXG4= - Multi
vhostuserinterface:apiVersion: kubevirt.io/v1alpha3 kind: VirtualMachine metadata: name: vm-multi-virtio namespace: contrail spec: running: true template: metadata: labels: kubevirt.io/size: small kubevirt.io/domain: vm-multi-virtio app: vm-multi-virtio-app spec: nodeSelector: worker: worker terminationGracePeriodSeconds: 30 domain: cpu: sockets: 1 cores: 8 threads: 2 #dedicatedCpuPlacement: true memory: hugepages: pageSize: "2Mi" resources: requests: memory: "512Mi" devices: disks: - name: containerdisk disk: bus: virtio - name: cloudinitdisk disk: bus: virtio interfaces: - name: default bridge: {} - name: vhost-user-vn-blue vhostuser: {} - name: vhost-user-vn-green vhostuser: {} useVirtioTransitional: true networks: - name: default pod: {} - name: vhost-user-vn-blue multus: networkName: vn-blue - name: vhost-user-vn-green multus: networkName: vn-green volumes: - name: containerdisk containerDisk: image: svl-artifactory.juniper.net/atom-docker/dpdk-pktgen/vmdisks/dpdk-pktgen-auto:latest - name: cloudinitdisk cloudInitNoCloud: userDataBase64: SGkuXG4= - Bridge/
vhostuserinterface VM:apiVersion: kubevirt.io/v1alpha3 kind: VirtualMachine metadata: name: vm-virtio-veth namespace: contrail spec: running: true template: metadata: labels: kubevirt.io/size: small kubevirt.io/domain: vm-virtio-veth app: vm-virtio-veth-app spec: nodeSelector: master: master terminationGracePeriodSeconds: 30 domain: cpu: sockets: 1 cores: 8 threads: 2 #dedicatedCpuPlacement: true memory: hugepages: pageSize: "2Mi" resources: requests: memory: "512Mi" devices: disks: - name: containerdisk disk: bus: virtio - name: cloudinitdisk disk: bus: virtio interfaces: - name: default bridge: {} - name: vhost-user-vn-blue vhostuser: {} - name: vhost-user-vn-green bridge: {} useVirtioTransitional: true networks: - name: default pod: {} - name: vhost-user-vn-blue multus: networkName: vn-blue - name: vhost-user-vn-green multus: networkName: vn-green volumes: - name: containerdisk containerDisk: image: svl-artifactory.juniper.net/atom-docker/dpdk-pktgen/vmdisks/dpdk-pktgen-auto:latest - name: cloudinitdisk cloudInitNoCloud: userDataBase64: SGkuXG4=