Customize Cloud-Native Router Configuration using Node Annotations
Read this topic to understand how to customize Cloud-Native Router configuration using node annotations and custom configuration template.
Node Annotations
Starting with Juniper Cloud-Native Router (JCNR) Release 23.2, Cloud-Native Router supports customizing the Cloud-Native Router configurations using node annotations when deployed in L3 mode. Node annotations are key-value pairs. The key-value pairs are used to render the cRPD configuration via a go template. The configured template must be available in theJuniper_Cloud_Native_Router_release_number/helmchart/charts/jcnr-cni/files/
directory for the configuration to be applied to the cRPD pods.You must apply the node annotations before installing Cloud-Native Router to create cRPD pods with custom configuration. The cRPD pod must be deleted and respawned should you wish to apply the annotations any time after Cloud-Native Router installation. cRPD customization via node annotations is optional.
Configuration Example
Sample node annotation
and template files are available under
Juniper_Cloud_Native_Router_<release-number>/helmchart/cRPD_examples
directory.
You define the key-value pair for each worker node of your
cluster. An example of the node-annotations.yaml file is provided
below:
apiVersion: v1
kind: Node
metadata:
name: 5d8s1-node1 ---> Node name
annotations:
jcnr.juniper.net/params: '{
"isoLoopbackAddr": "49.0004.1000.0000.0001.00", ---> Key value pairs
"IPv4LoopbackAddr": "110.1.1.2",
"srIPv4NodeIndex": "2000",
"srIPv6NodeIndex": "3000",
"BGPIPv4Neighbor": "110.1.1.254",
"BGPLocalAsn": "64512"
}'
---
apiVersion: v1
kind: Node
metadata:
name: 5d8s1-node2
annotations:
jcnr.juniper.net/params: '{
"isoLoopbackAddr": "49.0004.1000.0000.0000.00",
"IPv4LoopbackAddr": "110.1.1.3",
"srIPv4NodeIndex": "2001",
"srIPv6NodeIndex": "3002",
"BGPIPv4Neighbor": "110.1.2.254",
"BGPLocalAsn": "64512"
}'The
key-value pairs you define in the annotations is used to render the cRPD
configuration via a go template. An example of the
jcnr-cni-custom-config.tmpl template file is provided
below:
apply-groups [custom];
groups {
custom {
interfaces {
lo0 {
unit 0 {
{{if .Node.isoLoopbackAddr}} ---> key is replaced by value, if available for the node
family iso {
address {{.Node.isoLoopbackAddr}};
}
{{end}}
family inet {
address {{.Node.IPv4LoopbackAddr}};
}
}
}
}
routing-options {
router-id {{.Node.IPv4LoopbackAddr}}
route-distinguisher-id {{.Node.IPv4LoopbackAddr}}
}
protocols {
isis {
interface all;
{{if and .Env.SRGB_START_LABEL .Env.SRGB_INDEX_RANGE}} ---> conditional statement based on environment variables
source-packet-routing {
srgb start-label {{.Env.SRGB_START_LABEL}} index-range {{.Env.SRGB_INDEX_RANGE}};
node-segment {
{{if .Node.srIPv4NodeIndex}}
ipv4-index {{.Node.srIPv4NodeIndex}};
{{end}}
{{if .Node.srIPv6NodeIndex}}
ipv6-index {{.Node.srIPv6NodeIndex}};
{{end}}
}
}
{{end}}
level 1 disable;
}
ldp {
interface all;
}
mpls {
interface all;
}
bgp {
interface all;
}
}
policy-options {
# policy to signal dynamic UDP tunnel attributes to BGP routes
policy-statement udp-export {
then community add udp;
}
community udp members encapsulation:0L:13;
}
protocols {
bgp {
group jcnrbgp1 {
type internal;
local-address {{.Node.IPv4LoopbackAddr}};
local-as {{.Node.BGPLocalAsn}};
neighbor {{.Node.BGPIPv4Neighbor}};
family inet-vpn {
unicast;
}
family inet6-vpn {
unicast;
}
}
}
}
routing-options {
dynamic-tunnels {
dyn-tunnels {
source-address {{.Node.IPv4LoopbackAddr}};
udp;
destination-networks {{.Node.BGPIPv4Neighbor}}/32;
}
}
}
}
}You can define additional cRPD configuration hierarchies in the template. The
values to be rendered from the annotations defined in the
node-annotations.yaml must be defined as
{{.Node.key}}. Any environment
variables, such as variables defined in values.yaml, must be
defined as {{.Env.variable_name}}.
Once the template is configured, you must copy the
jcnr-cni-custom-config.tmpl file to the
Juniper_Cloud_Native_Router_release_number/helmchart/charts/jcnr-cni/files/
directory.
# cp Juniper_Cloud_Native_Router_release_number/helmchart/cRPD_examples/jcnr-cni-custom-config.tmpl Juniper_Cloud_Native_Router_release_number/helmchart/charts/jcnr-cni/files/ cp: overwrite 'Juniper_Cloud_Native_Router_release_number/helmchart/charts/jcnr-cni/files/jcnr-cni-custom-config.tmpl'? yes #
# kubectl apply -f node-annotation.yaml node/5d8s1-node1 configured node/5d8s1-node2 configured
Follow
the JCNR
installation instructions to deploy the cloud-native router components,
including the cRPD. Once the installation completes, access the cRPD shell and issue the show configuration | display
set command in the cli mode to view the custom
configuration you
applied.
root@jcnr-01> show configuration
## Last commit: 2023-06-23 08:30:42 EDT by root
version 20230608.143922_builder.r1342735;
groups {
base { /* OMITTED */ };
custom {
interfaces {
lo0 {
unit 0 {
family inet {
address 110.1.1.2/32;
}
family iso {
address 49.0004.1000.0000.0001.00;
}
}
}
}
policy-options {
# policy to signal dynamic UDP tunnel attributes to BGP routes
policy-statement udp-export {
then {
community add udp;
}
}
community udp members encapsulation:0L:13;
}
routing-options {
route-distinguisher-id 110.1.1.2;
router-id 110.1.1.2;
dynamic-tunnels {
dyn-tunnels {
source-address 110.1.1.2;
udp;
destination-networks {
110.1.1.254/32;
}
}
}
}
protocols {
bgp {
group jcnrbgp1 {
type internal;
local-address 110.1.1.2;
family inet-vpn {
unicast;
}
family inet6-vpn {
unicast;
}
local-as 64512;
neighbor 110.1.1.254;
}
}
isis {
interface all;
source-packet-routing {
srgb start-label 400000 index-range 4000;
node-segment {
ipv4-index 2000;
ipv6-index 3000;
}
}
level 1 disable;
}
ldp {
interface all;
}
mpls {
interface all;
}
}
}
cni { /* OMITTED */ };
internal { /* OMITTED */ };
}
apply-groups [ custom base internal ]; Troubleshooting
The cRPD pod continues to remain in Pending state if invalid
configuration is rendered and applied via the go template. The rendered
configuration is saved in /etc/crpd directory on the Cloud-Native Router host as
juniper.conf.master. You can apply the rendered configuration
manually to a running cRPD pod to validate the configuration and identify issues.
For an AWS EKS deployment you can find the rendered config within the cRPD pod in
the /config directory.