ON THIS PAGE
Static Routes
SUMMARY Juniper Cloud-Native Contrail Networking (CN2) release 23.1 supports static routes for your cluster. This article provides information about how to configure static routes for your CN2 cluster.
Understanding Static Routes
You can use static routes when a network doesn't require the complexity of a dynamic routing protocol. Routes that are permanent fixtures in routing and forwarding tables are often configured as static routes.
The route consists of a destination prefix and a next-hop forwarding address. The static route is activated in the routing table and inserted into the forwarding table when the next-hop address is reachable. Traffic that matches the static route is forwarded to the specified next-hop address.
Static Routes in CN2
CN2 implements static routes through the following two custom resources (CRs):
-
RouteTable: Contains a user-defined network destination prefix, along with an associated next hop (nextHop). ThenextHopIP address must be an IP address of a VMI object. A prefix defines the destination network that is reachable via the next hop. ARouteTablelets you define a static route and next-hop pairing, which you can then associate with aRouteTableand a virtual network (VN). The following is an example of aRouteTableCR. Note that the namespace field is not required in this instance.apiVersion: core.contrail.juniper.net/v3 kind: RouteTable metadata: name: static-rt namespace: static-route spec: routes: route: - nextHop: 10.20.30.2 nextHopType: ip-address prefix: 10.20.30.0/24 communityAttributes: communityAttribute: - accept-own - no-advertiseNote that the field nextHopType must have the value ip-address. Any other value results in a user input error. The communityAttributes field allows you to set BGP communities on the route, which are advertised via BGP.
-
InterfaceRouteTable: TheInterfaceRouteTableconfigures static routing with the next hop of a virtual machine interface (VMI). AnInterfaceRouteTablecontains the destination prefix without the need for a statically-configured next hop entry. As with aRouteTable, the prefix defines the destination network. Unlike aRouteTable, you do not need to define anextHopIP address because when you associate anInterfaceRouteTablewith a VMI, the associated VMI acts as the next hop for this prefix. -
The following is an example of an
InterfaceRouteTableCR. Note that the namespace field is not required in this instance.apiVersion: core.contrail.juniper.net/v3 kind: InterfaceRouteTable metadata: name: static-rt namespace: static-route spec: interfaceRouteTableRoutes: route: - nextHopType: ip-address prefix: 10.20.30.0/24 communityAttributes: communityAttribute: - accept-ownNote that the field
nextHopTypemust have the valueip-address. Any other value results in a user input error.These CRs are scoped to their respective namespaces and enable you to configure required attributes for static routes.
Configure Static Routes for a Virtual Network
Configure the RouteTable CR to apply static routes to a virtual network
(VN). A VN references a RouteTable in it's spec. As a result, when the
RouteTable is associated with that VN, the static route is applied. The
following is a VN object with an associated RouteTable:
apiVersion: core.contrail.juniper.net/v3
kind: VirtualNetwork
metadata:
namespace: static-route
name: vn-route
spec:
v4SubnetReference:
apiVersion: core.contrail.juniper.net/v1alpha1
kind: Subnet
namespace: static-route
name: vn-subnet
routeTableReferences:
- apiVersion: core.contrail.juniper.net/v3
kind: RouteTable
namespace: static-route
name: static-rtConfigure Static Routes for a VMI
Configure the interface-route-table annotation on a VMI in order to apply an
InterfaceRouteTable to a VMI.
apiVersion: v3
kind: VirtualMachineInterface
metadata:
name: static-route-pod
namespace: static-route
annotations:
core.juniper.net/interface-route-table: '[{"name": "static-rt", "namespace": "static-route"}]'
spec:
<VMI_SPEC>
status:
interfaceRouteTableReferences:
- apiVersion: core.contrail.juniper.net/v3
kind: InterfaceRouteTable
namespace: static-route
name: static-rtThe following is an example kubectl describe vmi output:
Name: forwarder-6dff5888fd-59c9l-da8f57ef
Namespace: static-route
Labels: back-reference.core.juniper.net/28ffe95511d099080e315d0bb633ec5334f9250a40c8c9c61825d185=Tag_t2-kubernetes.io_metadata.name-static-route
back-reference.core.juniper.net/5b136ca6d41e33dfb79a3f066e3db71186ade2514d1de62cf7d3a2b7=Tag_t1-core.juniper.net_clusterName-contrail-k8s-kubemanager-ku
back-reference.core.juniper.net/83323440b6198bd1f37a71f0ad4afde199fffd2742ed93cdeea56d4f=InterfaceRouteTable_static-route_to-zone-1
back-reference.core.juniper.net/84befcc656f5c3b04ece96e4d040cd168c3470825052cd9ef6903755=Tag_t1-core.juniper.net_namespace-static-route
back-reference.core.juniper.net/92b2adff92e4dea99659da131af95ac5623ac10275e0a581fcab8c77=InterfaceRouteTable_static-route_to-right
back-reference.core.juniper.net/9793fa23186c25d750b1a724d088a2cd46bff77cc54f819860847259=VirtualNetwork_static-route_left-vn
back-reference.core.juniper.net/bcaf7cc535f4f34dd8d9e4a425cebeea7c229e4efc8e2f8ba3821bc8=VirtualMachine_contrail-k8s-kubemanager-kubernetes-forwarder-6d
back-reference.core.juniper.net/ec2031060f3699dd2f7dd888d858d0a51de65fc2e5e62100b85df999=RoutingInstance_static-route_left-vn
back-reference.core.juniper.net/f48f4e9c4f6c1dfccc439d752da4c9ee8eb7472a73d7bf5749a050af=Tag_t1-app-forwarder
back-reference.core.juniper.net/f6b670baed4845c6640bb4db030b0d22cb3165a5e3929aad3a91fff6=Tag_t1-pod-template-hash-6dff5888fd
Annotations: core.juniper.net/interface-route-table: [{"Namespace":"static-route","Name":"to-right"},{"Namespace":"static-route","Name":"to-zone-1"}]
index: 1/5
interface: eth1
kube-manager.juniper.net/pod-cluster-name: contrail-k8s-kubemanager-kubernetes
kube-manager.juniper.net/pod-name: forwarder-6dff5888fd-59c9l
kube-manager.juniper.net/pod-namespace: static-route
network: left-vn
vmi-address-family: dualStack
API Version: core.contrail.juniper.net/v3
Kind: VirtualMachineInterface
Metadata:
Creation Timestamp: 2023-05-04T06:05:15Z
Finalizers:
virtualmachineinterface.finalizers.core.juniper.net
Generation: 2
Managed Fields:
API Version: core.contrail.juniper.net/v3
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:finalizers:
.:
v:"virtualmachineinterface.finalizers.core.juniper.net":
f:labels:
f:back-reference.core.juniper.net/ec2031060f3699dd2f7dd888d858d0a51de65fc2e5e62100b85df999:
f:spec:
f:virtualMachineInterfaceMacAddresses:
f:macAddress:
Manager: manager
Operation: Update
Time: 2023-05-04T06:05:15Z
API Version: core.contrail.juniper.net/v3
Fields Type: FieldsV1
fieldsV1:
f:status:
f:interfaceRouteTableReferences:
f:observation:
f:routingInstanceReferences:
f:state:
Manager: manager
Operation: Update
Subresource: status
Time: 2023-05-04T06:05:16Z
API Version: core.contrail.juniper.net/v3
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:core.juniper.net/interface-route-table:
f:index:
f:interface:
f:kube-manager.juniper.net/pod-cluster-name:
f:kube-manager.juniper.net/pod-name:
f:kube-manager.juniper.net/pod-namespace:
f:network:
f:vmi-address-family:
f:ownerReferences:
.:
k:{"uid":"a48fb78d-710b-4d66-9c29-73b77d5db8c6"}:
f:spec:
f:portSecurityEnabled:
f:tagReferences:
f:virtualMachineReferences:
f:virtualNetworkReference:
.:
f:apiVersion:
f:kind:
f:name:
f:namespace:
f:resourceVersion:
f:uid:
Manager: kubemanager
Operation: Update
Time: 2023-05-04T06:05:17Z
Owner References:
API Version: core.contrail.juniper.net/v3
Block Owner Deletion: true
Controller: true
Kind: VirtualMachine
Name: contrail-k8s-kubemanager-kubernetes-forwarder-6dff5888fd-59c9l-65da579d
UID: a48fb78d-710b-4d66-9c29-73b77d5db8c6
Resource Version: 43839
UID: bbe79a87-fda4-4a2d-be5f-52ecad9863e2
Spec:
Allowed Address Pairs:
Fq Name:
default-domain
static-route
forwarder-6dff5888fd-59c9l-da8f57ef
Parent:
Port Security Enabled: true
Properties:
Tag References:
API Version: core.contrail.juniper.net/v3
Fq Name:
t1-core.juniper.net_namespace-static-route
Kind: Tag
Name: t1-core.juniper.net_namespace-static-route
Resource Version: 43153
UID: c6bc3209-03c0-4c42-92fe-a71bcf5d2268
API Version: core.contrail.juniper.net/v3
Fq Name:
t1-core.juniper.net_clusterName-contrail-k8s-kubemanager-kubernetes
Kind: Tag
Name: t1-core.juniper.net_clusterName-contrail-k8s-kubemanager-kubernetes
Resource Version: 5169
UID: 72bf6479-90c2-4c57-97ca-214f0ddeec38
API Version: core.contrail.juniper.net/v3
Fq Name:
t1-app-forwarder
Kind: Tag
Name: t1-app-forwarder
Resource Version: 43297
UID: c31f28bb-efba-4793-bd2f-567fd3ff396d
API Version: core.contrail.juniper.net/v3
Fq Name:
t1-pod-template-hash-6dff5888fd
Kind: Tag
Name: t1-pod-template-hash-6dff5888fd
Resource Version: 43298
UID: 7add9d84-8db2-4ddc-b32a-6f2ba06182ee
API Version: core.contrail.juniper.net/v3
Fq Name:
t2-kubernetes.io_metadata.name-static-route
Kind: Tag
Name: t2-kubernetes.io_metadata.name-static-route
Resource Version: 43178
UID: ed588b5a-bc07-4ab4-b163-016ab16924ee
Virtual Machine Interface Mac Addresses:
Mac Address:
02:bb:e7:9a:87:fd
Virtual Machine References:
API Version: core.contrail.juniper.net/v3
Fq Name:
contrail-k8s-kubemanager-kubernetes-forwarder-6dff5888fd-59c9l-65da579d
Kind: VirtualMachine
Name: contrail-k8s-kubemanager-kubernetes-forwarder-6dff5888fd-59c9l-65da579d
Resource Version: 43434
UID: a48fb78d-710b-4d66-9c29-73b77d5db8c6
Virtual Network Reference:
API Version: core.contrail.juniper.net/v3
Fq Name:
default-domain
static-route
left-vn
Kind: VirtualNetwork
Name: left-vn
Namespace: static-route
Resource Version: 42804
UID: 41ab021d-0b55-4598-bc32-abf954708fd0
Status:
Interface Route Table References:
API Version: core.contrail.juniper.net/v3
Fq Name:
static-route
to-right
Kind: InterfaceRouteTable
Name: to-right
Namespace: static-route
Resource Version: 42473
UID: 912aa23d-fb26-4181-8c42-880ebe9f1a9b
API Version: core.contrail.juniper.net/v3
Fq Name:
static-route
to-zone-1
Kind: InterfaceRouteTable
Name: to-zone-1
Namespace: static-route
Resource Version: 42475
UID: a2cfce5c-cb31-4a24-b432-16bdb8318bd6
Observation:
Routing Instance References:
API Version: core.contrail.juniper.net/v3
Attributes:
Direction: both
Fq Name:
default-domain
static-route
left-vn
left-vn
Kind: RoutingInstance
Name: left-vn
Namespace: static-route
UID: 300cda6a-a7fa-4f79-a815-dc3d7c0a46e7
State: Success
Events: <none>]Note the Interface Route Table References section. This section shows the
association between the listed InterfaceRouteTables and VMIs.
Configure Static Routes on Pod Interfaces
You can use the annotation section of a pod's manifest to configure static routes for a
pod's default or secondary interface. The pod reconciler processes the annotation section to
create a VMI object with an associated InterfaceRouteTable. The reconciler
looks for the string key: "core.juniper.net/interface-route-table" in the annotation
section. The pod's VMI uses that string as a metadata label to associate with an
InterfaceRouteTable.
The following is an example of a pod manifest with an InterfaceRouteTable
defined for the default interface:
apiVersion: v1
kind: Pod
metadata:
name: static-route-pod
namespace: static-route
annotations:
core.juniper.net/interface-route-table: '[{"name": "vmi-rt", "namespace": "static-route"}]'
spec:
containers:
- name: praqma
image: <image-repository>:<tag>
imagePullPolicy: Always
securityContext:
capabilities:
add:
- NET_ADMIN
privileged: trueNote the securityContext fields. These fields are necessary because the
pod must have permission to change it's routing table in order to utilize the configured
route.
The following is an example of a pod manifest with an InterfaceRouteTable
defined for the secondary interface:
apiVersion: v1
kind: Pod
metadata:
name: static-route-pod
namespace: static-route
annotations:
k8s.v1.cni.cncf.io/networks: |
[
{
"name": "vn-route",
"namespace": "static-route",
"cni-args": {
"core.juniper.net/interface-route-table": "[{\"name\": \"vmi-rt\", \"namespace\": \"static-route\"}]"
}
}
]
spec:
containers:
- name: praqma
image: <image-repository>:<tag>
imagePullPolicy: Always
securityContext:
capabilities:
add:
- NET_ADMIN
privileged: trueNote that the name for the primary interface InterfaceRouteTable is
vmi-rt and that the name for the secondary interface is
vn-route. Defining two InterfaceRouteTables with
different names in the same namespace automatically creates an
InterfaceRouteTable for the primary and secondary interface of that
pod.
Configure Static Routes for a Virtual Network with a NAD
You can also specify static route properties in a network attachment definition (NAD)
object. After the NAD is reconciled or applied, a RouteTable is created and
the resulting VN object references that RouteTable. The following is an
example of a NAD with static route information defined:
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
name: vn-route
namespace: static-route
labels:
vn: vn-route
annotations:
juniper.net/networks: '{
"ipamV4Subnet": "108.108.2.0/24"
"routeTableReferences": '[{"name": "vn-rt", "namespace": "static-route"}]'
}'
spec:
config: '{
"cniVersion": "0.3.1",
"name": "vn-route",
"type": "contrail-k8s-cni"
}'Multiple Static Routes on Pod Interfaces
Using InterfaceRouteTable, you can associate multiple static routes to a
single pod interface (VMI). This means that that VMI object has multiple next hop
destinations, depending on the IP prefix. You can specify multiple
InterfaceRouteTable references using cluster service version (CSV) syntax
or JSON syntax annotations.
You must reference an InterfaceRouteTable in a "namespace/name" format.
In the following example, static-route is the namespace and
to-right and to-zone-1 are the
InterfaceRouteTable objects, or next hop destination for the
left-vn VMI.
The following example is a Deployment with multiple
InterfaceRouteTable references:
apiVersion: apps/v1
kind: Deployment
metadata:
name: forwarder
namespace: static-route
labels:
app: forwarder
spec:
replicas: 3
selector:
matchLabels:
app: forwarder
template:
metadata:
labels:
app: forwarder
annotations:
k8s.v1.cni.cncf.io/networks: |
[
{
"name": "left-vn",
"namespace": "static-route",
"cni-args": {
"core.juniper.net/interface-route-table": "static-route/to-right,static-route/to-zone-1"
}
},
{
"name": "right-vn",
"namespace": "static-route",
"cni-args": {
"core.juniper.net/interface-route-table": "static-route/to-left"
}
},
{
"name": "zone-1",
"namespace": "static-route",
"cni-args": {
"core.juniper.net/interface-route-table": "static-route/to-left"
}
},
{
"name": "zone-2",
"namespace": "static-route",
"cni-args": {
"core.juniper.net/interface-route-table": "static-route/to-left"
}
}
]
spec:
containers:
- name: praqma
image: <repository>:<tag>
securityContext:
capabilities:
add:
- NET_ADMIN
privileged: trueThe following example is a pod manifest with multiple InterfaceRouteTable
references using JSON syntax:
apiVersion: v1
kind: Pod
metadata:
name: irt-right
namespace: static-route
annotations:
k8s.v1.cni.cncf.io/networks: |
[{
"name": "right-vn",
"namespace": "static-route",
"cni-args": {
"core.juniper.net/interface-route-table": "[{\"namespace\": \"static-route\", "\name\": \"to-left\"}, {\"namespace\": \"static-route\", \"name\": \"to-zone-1\"}]"
}
}]
spec:
containers:
- name: praqma
image: <image-repository>:<tag>
securityContext:
capabilities:
add:
- NET_ADMIN
privileged: trueYou must use backward slashes in JSON syntax. Backward slashes are required to encode a JSON string inside another JSON string.
Troubleshooting RouteTable and InterfaceRouteTable
Dataplane Verification
-
In the vRouter introspect, verify that the VRF of the VN shows a row with a matching static route prefix specified in the RT using the following steps:
-
Access vRouter introspect at https://<vrouter_ip>:8085/Snh_VrfListReq
-
Verify that the VRF is associated with the VN.
-
Navigate to the ucindex column in the VRF unicast
RouteTable. -
Verify that the table contains a row with the correct static route prefix.
-
- In the introspect, verify that the next hop properties of the VN are accurate and as
desired. In the introspect, the next hop column for the prefix should contain the
following:
-
The next hop interface name must be a valid tap interface.
-
The
labelmust be a positive integer. -
The
resolvedvalue must betrue. -
The
route-type:value must beInterfaceStaticRoute.
-
Config Plane Verification
-
Verify the state of the
RouteTableandInterfaceRouteTableobjects.-
Check the status of the reconciler for the
InterfaceRouteTableobject.kubectl get interfaceroutetable -n
-
Check the status of the reconciler for the
RouteTableobject.kubectl get routetable -n
-
-
Verify the
RouteTablereference in the associated VN. Verify theInterfaceRouteTablereference in the associated VMI.-
Check the status of the reconciler for the VMI. You should see the
InterfaceRouteTablein the VMI with an associated universally unique identifier (UUID) the Contrail FQ (meta info such asapiversion,kind,namespace,name) name.kubectl get vmi -n -oyaml | grep -i interfaceRouteTable
kubectl get vn -n -oyaml | grep -i routeTable
-