Customize Cloud-Native Router Configuration
Read this topic to understand how to customize Cloud-Native Router configuration using a ConfigMap.
Cloud-Native Router ConfigMap
Starting with Juniper Cloud-Native Router (JCNR) Release 23.3, Cloud-Native Router supports customizing configuration using a ConfigMap when deployed in L3 mode. In cloud-based deployments, in the event of a node failure, the Cloud-Native Router pods may be spawned on newer or different nodes. A ConfigMap decouples the configuration parameters from node names and is based on node labels instead. This enables the Cloud-Native Router CNI deployer to consume the configuration parameters, apply them to the cRPD configuration template and render the configuration, as long as a matching label is available for the node.A ConfigMap is an API object to store data in key-values
pairs. A ConfigMap defines per node variables that are consumed by nodes matching
the label. The key-value pairs are used to render the configuration via a go
template. The configured template must be available in the
Juniper_Cloud_Native_Router_release_number/helmchart/charts/jcnr-cni/files/
directory for the configuration to be applied to the cRPD pods.
You must apply the ConfigMap before installing Cloud-Native Router to create cRPD pods with custom configuration. The cRPD pod must be deleted and respawned should you wish to apply the configuration parameters any time after Cloud-Native Router installation. The configuration parameters are applied by default to any newly spawned cRPD pods. The Cloud-Native Router customization via ConfigMap is optional.
Cloud-Native Router also supports customization via node annotations for backward compatibility with previous releases. Considering that node annotations are coupled with node names, it is highly recommended to customize Cloud-Native Router via ConfigMaps, specifically for cloud deployments. Refer to Customize JCNR Configuration using node annotations for more information.
Configuration Example
Sample ConfigMap and template files are available under
Juniper_Cloud_Native_Router_<release-number>/helmchart/cRPD_examples
directory.
You define the key-value pair for different node labels in your cluster. An example
of the jcnr-params-configmap.yaml file is provided below:
apiVersion: v1
kind: ConfigMap
metadata:
name: jcnr-params
namespace: jcnr
data:
jcnr1: |
{
"isoLoopbackAddr": "49.0004.1000.0000.0001.00",
"IPv4LoopbackAddr": "110.1.1.2",
"srIPv4NodeIndex": "2000",
"srIPv6NodeIndex": "3000",
"BGPIPv4Neighbor": "110.1.1.254",
"BGPLocalAsn": "64512"
}
jcnr2: |
{
"isoLoopbackAddr": "49.0004.1000.0000.0000.00",
"IPv4LoopbackAddr": "110.1.1.3",
"srIPv4NodeIndex": "2001",
"srIPv6NodeIndex": "3001",
"BGPIPv4Neighbor": "110.1.2.254",
"BGPLocalAsn": "64512"
}The key-value pairs you define in the annotations is used to render the cRPD
configuration via a go template. An example of the
jcnr-cni-custom-config-cm.tmpl template file is provided
below:
apply-groups [custom];
groups {
custom {
interfaces {
lo0 {
unit 0 {
{{if .Params.isoLoopbackAddr}}
family iso {
address {{.Params.isoLoopbackAddr}};
}
{{end}}
family inet {
address {{.Params.IPv4LoopbackAddr}};
}
}
}
}
routing-options {
router-id {{.Params.IPv4LoopbackAddr}}
route-distinguisher-id {{.Params.IPv4LoopbackAddr}}
}
protocols {
isis {
interface all;
{{if and .Env.SRGB_START_LABEL .Env.SRGB_INDEX_RANGE}}
source-packet-routing {
srgb start-label {{.Env.SRGB_START_LABEL}} index-range {{.Env.SRGB_INDEX_RANGE}};
node-segment {
{{if .Params.srIPv4NodeIndex}}
ipv4-index {{.Params.srIPv4NodeIndex}};
{{end}}
{{if .Params.srIPv6NodeIndex}}
ipv6-index {{.Params.srIPv6NodeIndex}};
{{end}}
}
}
{{end}}
level 1 disable;
}
ldp {
interface all;
}
mpls {
interface all;
}
}
policy-options {
# policy to signal dynamic UDP tunnel attributes to BGP routes
policy-statement udp-export {
then community add udp;
}
community udp members encapsulation:0L:13;
}
protocols {
bgp {
group jcnrbgp1 {
type internal;
local-address {{.Params.IPv4LoopbackAddr}};
local-as {{.Params.BGPLocalAsn}};
neighbor {{.Params.BGPIPv4Neighbor}};
family inet-vpn {
unicast;
}
family inet6-vpn {
unicast;
}
}
}
}
routing-options {
dynamic-tunnels {
dyn-tunnels {
source-address {{.Params.IPv4LoopbackAddr}};
udp;
destination-networks {{.Params.BGPIPv4Neighbor}}/32;
}
}
}
}
}You can define additional cRPD configuration hierarchies in the template. The
values to be rendered from the ConfigMap defined in the
jcnr-params-configmap.yaml must be defined as
{{.Params.var-name}}. Any
environment variables, such as variables defined in
values.yaml, must be defined as
{{.Env.variable_name}}.
Complete the following steps to apply the customizations.
-
Label each node based on the keys used in the ConfigMap.
kubectl label nodes <node_name1> jcnr.juniper.net/params-profile=jcnr1 kubectl label nodes <node_name2> jcnr.juniper.net/params-profile=jcnr2
- Apply the ConfigMap to the cluster nodes using the command provided
below:
# kubectl apply -f jcnr-params-configmap.yaml configmap/jcnr-params created
- Once the template is configured, you must copy the
jcnr-cni-custom-config.tmplfile to theJuniper_Cloud_Native_Router_release_number/helmchart/charts/jcnr-cni/files/directory.# cp Juniper_Cloud_Native_Router_release_number/helmchart/cRPD_examples/jcnr-cni-custom-config-cm.tmpl Juniper_Cloud_Native_Router_release_number/helmchart/charts/jcnr-cni/files/ #
- Deploy the cloud-native router components, including the cRPD. Once the
installation completes, access the cRPD
CLIAccess cRPD CLI and
issue the
show configuration | display setcommand in theclimode to view the custom configuration you applied.root@jcnr-01> show configuration ## Last commit: 2023-06-23 08:30:42 EDT by root version 20230608.143922_builder.r1342735; groups { base { /* OMITTED */ }; custom { interfaces { lo0 { unit 0 { family inet { address 110.1.1.2/32; } family iso { address 49.0004.1000.0000.0001.00; } } } } policy-options { # policy to signal dynamic UDP tunnel attributes to BGP routes policy-statement udp-export { then { community add udp; } } community udp members encapsulation:0L:13; } routing-options { route-distinguisher-id 110.1.1.2; router-id 110.1.1.2; dynamic-tunnels { dyn-tunnels { source-address 110.1.1.2; udp; destination-networks { 110.1.1.254/32; } } } } protocols { bgp { group jcnrbgp1 { type internal; local-address 110.1.1.2; family inet-vpn { unicast; } family inet6-vpn { unicast; } local-as 64512; neighbor 110.1.1.254; } } isis { interface all; source-packet-routing { srgb start-label 400000 index-range 4000; node-segment { ipv4-index 2000; ipv6-index 3000; } } level 1 disable; } ldp { interface all; } mpls { interface all; } } } cni { /* OMITTED */ }; internal { /* OMITTED */ }; } apply-groups [ custom base internal ];
Modifying the ConfigMap
If you wish to change the ConfigMap any time after Cloud-Native Router installation, you must delete the cRPD pod and respawn it using the following command:kubectl patch sts kube-crpd-worker-sts -n jcnr -p '{"spec":{"template":{"metadata":{"annotations":{"configmap-reload/timestamp":"'$(date +%s)'"}}}}}'Troubleshooting
The cRPD pod continues to restart in CrashLoopBackOff state if
invalid configuration is rendered and applied via the go template. The rendered
configuration is saved in /config directory on the Cloud-Native Router host as
juniper.conf.master. You can apply the rendered configuration
manually to a running cRPD pod to validate the configuration and identify issues.
For an AWS EKS deployment you can find the rendered config within the cRPD pod in
the /config directory.