ON THIS PAGE
L3 VLAN Sub-Interface Configuration Example
SUMMARY Read this topic to learn how to add a user pod with a Layer 3 VLAN sub-interface to an instance of the cloud-native router.
Overview
You can configure a user pod with a Layer 3 VLAN sub-interface and attach it to the JCNR instance. The Juniper Cloud-Native Router must have an L3 interface configured at the time of deployment. The cRPD must be configured with the valid VLAN configuration for the fabric interface. For example:
set interfaces ens1f1v1 unit 201 vlan-id 201 set interfaces ens1f1v1 unit 201 family inet address 192.168.123.1/24 set interfaces ens1f1v1 unit 201 family inet6 address abcd:192:168:123::1/64 set routing-instance blue interface ens1f1v1.201
Your high-level tasks are:
-
Define and apply a network attachment definition (NAD)—The NAD file defines the required configuration for Multus to invoke the JCNR-CNI and create a network to attach the pod interface to.
-
Define and apply a pod YAML file to your cloud-native router cluster—The pod YAML contains the pod specifications and an annotation to the network created by the JCNR-CNI
Note:Please review the JCNR Use-Cases and Configuration Overview topic for more information on NAD and pod YAML files.
Configuration Example
- Here are example NADs to create a Layer 3 VLAN
sub-interface:
The NAD defines virtual-router instancesapiVersion: "k8s.cni.cncf.io/v1" kind: NetworkAttachmentDefinition metadata: name: vrf201 spec: config: '{ "cniVersion":"0.4.0", "name": "vrf201", "plugins": [ { "type": "jcnr", "args": { "instanceName": "vrf201", "instanceType": "virtual-router", "parentInterface":"net1", "vlanId": "201" }, "ipam": { "type": "static", "addresses":[ { "address":"99.61.0.2/16", "gateway":"99.61.0.1" }, { "address":"1234::99.61.0.2/120", "gateway":"1234::99.61.0.1" } ] }, "kubeConfig":"/etc/kubernetes/kubelet.conf" } ] }'
vrf201
with the parent interfacenet1
and VLAN ID201
. Avirtual-router
instance type is similar to a VPN routing and forwarding instance type, but used for non-VPN-related applications. There are no virtual routing and forwarding (VRF) import, VRF export, VRF target, or route distinguisher requirements for this instance type. The pod VLAN sub-interface is attached tovrf201
instance. The NAD also defines static IP addresses to be assigned to the pod interface. - Apply the NAD manifests to create the networks.
kubectl apply -f nad_l3_vlan_subinterface_201.yaml networkattachmentdefinition.k8s.cni.cncf.io/vrf201 created
- Verify the NADs are created.
kubectl get net-attach-def NAME AGE vrf201 30s
- Here is an example yaml to create a pod attached to the
vrf201
andvrf202
networks: The pod attaches to the router instances using theapiVersion: v1 kind: Pod metadata: name: pod1 annotations: k8s.v1.cni.cncf.io/networks: | [ { "name": "vrf201", "interface":"net1.201" } ] spec: containers: - name: pod1 image: ubuntu:latest imagePullPolicy: IfNotPresent securityContext: privileged: false env: - name: KUBERNETES_POD_UID valueFrom: fieldRef: fieldPath: metadata.uid volumeMounts: - name: dpdk mountPath: /dpdk subPathExpr: $(KUBERNETES_POD_UID) volumes: - name: dpdk hostPath: path: /var/run/jcnr/containers
k8s.v1.cni.cncf.io/networks
annotation. - Apply the pod manifest.
[root@jcnr-01]# kubectl apply -f pod_l3_subinterface.yaml pod/pod1 created
- Verify the pod is running.
[root@jcnr-01 ~]# kubectl get pods NAME READY STATUS RESTARTS AGE pod1 1/1 Running 0 38s
- Describe the pod to verify a secondary interface is created and attached to the
vrf201
network. (The output is trimmed for brevity).[root@jcnr-01 ~]# kubectl describe pod pod1 Name: pod1 Namespace: default Priority: 0 Node: jcnr-01/10.100.20.25 Start Time: Mon, 26 Jun 2023 09:53:31 -0400 Labels: <none> Annotations: cni.projectcalico.org/containerID: 90de252886b3e0a97526ac175544078fb03debf05650946d759e2de0d5179c17 cni.projectcalico.org/podIP: 10.233.91.126/32 cni.projectcalico.org/podIPs: 10.233.91.126/32 jcnr.juniper.net/dpdk-interfaces: [ { "name": "net1.201", "vhost-adaptor-path": "/dpdk/vhost-net1.sock", "vhost-adaptor-mode": "client", "ipv4-address": "99.61.0.2/16", "ipv6-address": "1234::633d:2/120", "mac-address": "02:00:00:8C:97:A2", "vlan-id": "201" } ] k8s.v1.cni.cncf.io/network-status: [{ "name": "k8s-pod-network", "ips": [ "10.233.91.126" ], "default": true, "dns": {} },{ "name": "default/vrf201", "interface": "net1.201", "ips": [ "99.61.0.2", "1234::633d:2" ], "mac": "02:00:00:8C:97:A2", "dns": {} }] ...
- Verify the vRouter has the corresponding interface created. Access the vRouter CLI and issue the
vif --list
command. You can see the IP addresses assigned to the sub-interfaces for the corresponding valid VRF numbers.vif0/11 PCI: 0000:b3:11.1 (Speed 10000, Duplex 1) NH: 16 MTU: 9014 ---> fabric interface Type:Physical HWaddr:b2:56:78:5c:af:fa IPaddr:0.0.0.0 DDP: OFF SwLB: ON Vrf:0 Mcast Vrf:0 Flags:L3L2Vof QOS:0 Ref:42 RX port packets:10988509 errors:0 RX queue errors to lcore 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Fabric Interface: 0000:b3:11.1 Status: UP Driver: net_iavf RX packets:10988509 bytes:5582067106 errors:0 TX packets:10988484 bytes:5581953776 errors:0 Drops:0 TX port packets:10988484 errors:0 vif0/17 PMD: ens1f1v1 NH: 44 MTU: 9000 ---> tap interface Type:Host HWaddr:b2:56:78:5c:af:fa IPaddr:0.0.0.0 DDP: OFF SwLB: ON Vrf:0 Mcast Vrf:0 Flags:L3L2 QOS:0 Ref:41 TxXVif:11 RX device packets:2201 bytes:935980 errors:0 RX queue packets:2201 errors:0 RX queue errors to lcore 0 0 0 0 0 0 0 0 0 0 0 0 0 0 RX packets:2201 bytes:935980 errors:0 TX packets:493 bytes:161906 errors:0 Drops:0 TX queue packets:493 errors:0 TX device packets:493 bytes:161906 errors:0 vif0/48 Virtual: ens1f1v1.201 Vlan(o/i)(,S): 201/201 NH: 161 MTU: 1514 Parent:vif0/11 Sub-type: physical-tap ---> L3 sub-interface, parent is a physical interface Type:Virtual(Vlan) HWaddr:b2:56:78:5c:af:fa IPaddr:192.168.123.1 IP6addr:abcd:192:168:123::1 DDP: OFF SwLB: ON Vrf:201 Mcast Vrf:201 Flags:L3DProxyEr QOS:-1 Ref:4 RX queue errors to lcore 0 0 0 0 0 0 0 0 0 0 0 0 0 0 RX packets:0 bytes:0 errors:0 TX packets:18 bytes:1836 errors:0 Drops:0 vif0/49 Virtual: ens1f1v1.201 Vlan(o/i)(,S): 201/201 NH: 156 MTU: 9000 Parent:vif0/17 Sub-type: Host-tap ---> L3 sub-interface, parent is a tap interface Type:Virtual(Vlan) HWaddr:b2:56:78:5c:af:fa IPaddr:192.168.123.1 IP6addr:abcd:192:168:123::1 DDP: OFF SwLB: ON Vrf:201 Mcast Vrf:65535 Flags:L3DProxyEr QOS:-1 Ref:4 TxXVif:48 RX queue errors to lcore 0 0 0 0 0 0 0 0 0 0 0 0 0 0 RX packets:18 bytes:1908 errors:0 TX packets:0 bytes:0 errors:0 Drops:0 vif0/50 PMD: vhostnet1-9403fd77-648a-47 NH: 177 MTU: 9160 ---> pod interface Type:Virtual HWaddr:00:00:5e:00:01:00 IPaddr:0.0.0.0 DDP: OFF SwLB: ON Vrf:65535 Mcast Vrf:65535 Flags:L3DProxyEr QOS:-1 Ref:20 RX queue errors to lcore 0 0 0 0 0 0 0 0 0 0 0 0 0 0 RX packets:0 bytes:0 errors:0 TX packets:0 bytes:0 errors:0 Drops:0 vif0/51 Virtual: vhostnet1-9403fd77-648a-47.202 Vlan(o/i)(,S): 202/202 NH: 17 MTU: 1514 Parent:vif0/50 ---->L3 pod sub-interface, parent is the pod interface Type:Virtual(Vlan) HWaddr:00:00:5e:00:01:00 IPaddr:99.62.0.2 IP6addr:1234::633e:2 DDP: OFF SwLB: ON Vrf:2 Mcast Vrf:2 Flags:PL3DProxyEr QOS:-1 Ref:4 RX queue errors to lcore 0 0 0 0 0 0 0 0 0 0 0 0 0 0 RX packets:0 bytes:0 errors:0 TX packets:0 bytes:0 errors:0 Drops:0