Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
ON THIS PAGE
 

System Requirements for OpenShift Deployment

Read this section to understand the system, resource, port, and licensing requirements for installing Juniper Cloud-Native Router on the Red Hat OpenShift Container Platform (OCP).

Minimum Host System Requirements for OCP

Table 1 and Table 2 list the host system requirements for installing Cloud-Native Router on OCP.

Table 1: Minimum Host System Requirements for OCP
Component Value/Version Notes
CPU Intel x86 The tested CPU is Intel(R) Xeon(R) Silver 4314 CPU @ 2.40GHz 64 core
Host OS RHCOS 4.18, 4.19, 4.20  
Kernel Version RedHat Enterprise Linux (RHEL): 9.4, 9.6  
NIC See Table 2.
IAVF driver Version 4.5.3.1  
ICE_COMMS Version 1.3.35.0  
ICE Version 1.9.11.9 ICE driver is used only with the Intel E810 NIC
i40e Version 2.18.9 i40e driver is used only with the Intel XL710 NIC
OCP Version 4.18, 4.19, 4.20  
OVN-Kubernetes CNI    
Multus Version 3.8  
OpenShift SR-IOV Network Operator (openshift-sriov-network-operator) Required when using NAD-based SR-IOV interface configuration

The OpenShift SR-IOV Network Operator manages SR-IOV network devices and generates network attachment definitions (NADs) for JCNR fabric interfaces. Install the operator from the OpenShift OperatorHub before deploying JCNR with NAD-based interface configuration.

Note: This prerequisite applies only when you configure JCNR fabric interfaces using the networkDetails section of values.yaml. It is not required for standard fabricInterface-based deployments.
Helm 3.12.x  
Container-RT crio 1.25x, 1.33.x The tested versions are crio 1.25x (OCP 4.18, 4.19), crio 1.33.x (OCP 4.20, K8s 1.33.9).
Note:

The component versions listed in this table are expected to work with JCNR, but not every version or combination is tested in every release.
Table 2: Supported NICs (DPDK)
Controller Family NIC Firmware Driver COMMS DDP
Intel Ethernet 700 Series

Intel XL710

 9.52 2.26.8 –
 

Intel X710-DA4 (4 x 10G)

9.52 2.26.8 –
 

Intel XXV710-DA2 (2 x 25G)

 9.52 2.26.8 –
 

Intel X722

 6.50 2.26.8 –
Intel Ethernet 800 Series

Intel E810 CPK

 4.6 1.15.4 –
 

Intel E810 CVL

 4.6 1.15.4 1.3.46
 

Intel E810-C (4 x 25G)

 4.6 1.15.4 1.3.46
 

Intel E810-C (2 x 100G)

 4.6 1.15.4 1.3.46
 

Intel E810-CQDA2

 4.6 1.15.4 1.3.46
 

Intel E810-XXV

 4.6 1.15.4 1.3.46
Nvidia Mellanox1,2

Mellanox ConnectX-6

 26.43.1014 24.10-0.7.0.0 –
 

Mellanox ConnectX-7

 28.43.1014 24.10-0.7.0.0 –

1When using Nvidia Mellanox NICs, ensure your interface names do not exceed 11 characters in length.

2When using Nvidia Mellanox NICs, follow the interface naming procedure in Interface Naming for Mellanox NICs.

Resource Requirements for OCP

Table 3 lists the resource requirements for installing Cloud-Native Router on OCP.

Table 3: Resource Requirements for OCP
Resource Value Usage Notes
Data plane forwarding cores 1 core (1P + 1S)  
Service/Control Cores 0  
UIO Driver VFIO-PCI To enable, follow the steps below:

Create a Butane config file, 100-worker-vfiopci.bu, binding the PCI device to the VFIO driver.

variant: openshift
version: 4.8.0
metadata:
  name: 100-worker-vfiopci
  labels:
    machineconfiguration.openshift.io/role: worker 
storage:
  files:
  - path: /etc/modprobe.d/vfio.conf
    mode: 0644
    overwrite: true
    contents:
      inline: |
        options vfio-pci ids=10de:1eb8 
  - path: /etc/modules-load.d/vfio-pci.conf 
    mode: 0644
    overwrite: true
    contents:
      inline: vfio-pci

Create and apply the machine config:

$ butane 100-worker-vfiopci.bu -o 100-worker-vfiopci.yaml
$ oc apply -f 100-worker-vfiopci.yaml
Hugepages (1G) 6 Gi For 1G hugepages, configure hugepages on the worker nodes using the following commands:
oc create -f hugepages-tuned-boottime.yaml 
# cat hugepages-tuned-boottime.yaml  
apiVersion: tuned.openshift.io/v1 
kind: Tuned 
metadata: 
  name: hugepages  
  namespace: openshift-cluster-node-tuning-operator 
spec: 
  profile:  
  - data: | 
      [main] 
      summary=Boot time configuration for hugepages 
      include=openshift-node 
      [bootloader] 
      cmdline_openshift_node_hugepages=hugepagesz=1G hugepages=6 
    name: openshift-node-hugepages 
  recommend: 
  - machineConfigLabels:  
      machineconfiguration.openshift.io/role: "worker-hp" 
    priority: 30 
    profile: openshift-node-hugepages 
oc create -f hugepages-mcp.yaml  
# cat hugepages-mcp.yaml  
apiVersion: machineconfiguration.openshift.io/v1 
kind: MachineConfigPool 
metadata: 
  name: worker-hp 
  labels: 
    worker-hp: "" 
spec: 
  machineConfigSelector: 
    matchExpressions: 
      - {key: machineconfiguration.openshift.io/role, operator: In, values: [worker,worker-hp]} 
  nodeSelector: 
    matchLabels: 
      node-role.kubernetes.io/worker-hp: ""

For 1G hugepages, see Configure the Number of Huge Pages Available on a Node.
Hugepages (2MB) 3 Gi For 2MB hugepages, configure hugepages on the worker nodes using the following commands:
oc create -f hugepages-2m-tuned-boottime.yaml 
# cat hugepages-2m-tuned-boottime.yaml
  apiVersion: tuned.openshift.io/v1
  kind: Tuned
  metadata:
    name: hugepages-2m
    namespace: openshift-cluster-node-tuning-operator
  spec:
    profile:
    - data: |
        [main]
        summary=Boot time configuration for 2MB hugepages
        include=openshift-node
        [bootloader]
        cmdline_openshift_node_hugepages=hugepagesz=2M hugepages=1536
      name: openshift-node-hugepages-2m
    recommend:
    - machineConfigLabels:
        machineconfiguration.openshift.io/role: "worker-hp"
      priority: 30
      profile: openshift-node-hugepages-2m
oc create -f hugepages-2m-mcp.yaml  
# cat hugepages-2m-mcp.yaml
  apiVersion: machineconfiguration.openshift.io/v1
  kind: MachineConfigPool
  metadata:
    name: worker-hp
    labels:
      worker-hp: ""
  spec:
    machineConfigSelector:
      matchExpressions:
      - {key: machineconfiguration.openshift.io/role, operator: In,
         values: [worker,worker-hp]}
    nodeSelector:
      matchLabels:
        node-role.kubernetes.io/worker-hp: ""

For 2MB hugepages, see Configure 2MB Hugepages for Low Memory Deployments.
Cloud-Native Router Controller cores .5  
Cloud-Native Router vRouter Agent cores .5  

Miscellaneous Requirements for OCP

Table 4 lists additional requirements for installing Cloud-Native Router on OCP.

Table 4: Miscellaneous Requirements for OCP

Requirement

Example

Enable the host with SR-IOV and VT-d in the system's BIOS.

Depends on BIOS.

Enable VLAN driver at system boot.

Configure /etc/modules-load.d/vlan.conf as follows:

cat /etc/modules-load.d/vlan.conf
8021q

Reboot and verify by executing the command:

lsmod | grep 8021q

Enable VFIO-PCI driver at system boot.

Configure /etc/modules-load.d/vfio.conf as follows:

cat /etc/modules-load.d/vfio.conf
vfio
vfio-pci

Reboot and verify by executing the command:

lsmod | grep vfio

Set IOMMU and IOMMU-PT.

 Create a MachineConfig object that sets IOMMU and IOMMU-PT:
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: worker 
  name: 100-worker-iommu 
spec:
  config:
    ignition:
      version: 3.2.0
  kernelArguments:
      - intel_iommu=on iommmu=pt
$ oc create -f 100-worker-kernel-arg-iommu.yaml

Disable spoofcheck on VFs allocated to JCNR.

Note:

Applicable for L2 deployments only.

 		ip link set <interfacename> vf 1 spoofcheck off.

Set trust on VFs allocated to JCNR.

Note:

Applicable for L2 deployments only.

 		ip link set <interfacename> vf 1 trust on

Additional kernel modules need to be loaded on the host before deploying Cloud-Native Router in L3 mode. These modules are usually available in linux-modules-extra or kernel-modules-extra packages.

Note:

Applicable for L3 deployments only.

Create a /etc/modules-load.d/crpd.conf file and add the following kernel modules to it:

tun
ipip
ip_tunnel
ip6_tunnel
mpls_router
mpls_iptunnel
vrf
vxlan

(Optional) Enable kernel-based forwarding on the Linux host. This step is only required if you are using DHCP relay.

 
ip fou add port 6635 ipproto 137

Exclude Cloud-Native Router interfaces from NetworkManager control.

NetworkManager is a tool in some operating systems to make the management of network interfaces easier. NetworkManager may make the operation and configuration of the default interfaces easier. However, it can interfere with Kubernetes management and create problems.

To avoid NetworkManager from interfering with Cloud-Native Router interface configuration, exclude Cloud-Native Router interfaces from NetworkManager control. Here's an example on how to do this in some Linux distributions:

  1. Create the /etc/NetworkManager/conf.d/crpd.conf file and list the interfaces that you don't want NetworkManager to manage.

    For example:

    [keyfile]
unmanaged-devices+=interface-name:enp*;interface-name:ens*;interface-name:__crpd;interface-name:__jcnr
    where enp*, ens*, __crpd, and __jcnr refer to your Cloud-Native Router interfaces.
    Note: enp* indicates all interfaces starting with enp. For specific interface names, provide a comma-separated list.
  2. Restart the NetworkManager service:
    sudo systemctl restart NetworkManager

Configure the IPv6 address policy.

  1. Edit the /etc/sysctl.conf file on the host and paste the following content in it:

    net.ipv6.conf.default.addr_gen_mode=0
net.ipv6.conf.all.addr_gen_mode=0
net.ipv6.conf.default.autoconf=0
net.ipv6.conf.all.autoconf=0
  2. Run the command sysctl -p /etc/sysctl.conf to load the new sysctl.conf values on the host.

Create the bond interface manually.

Note:

Applicable for L3 deployments only.

 
ifconfig ens2f0 down
ifconfig ens2f1 down
ip link add bond0 type bond mode 802.3ad
ip link set ens2f0 master bond0
ip link set ens2f1 master bond0
ifconfig ens2f0 up ; ifconfig ens2f1 up; ifconfig bond0 up

Verify the core_pattern value is set on the host before deploying JCNR.

 
sysctl kernel.core_pattern
kernel.core_pattern = |/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h %e

You can update the core_pattern in /etc/sysctl.conf. For example:

kernel.core_pattern=/var/crash/core_%e_%p_%i_%s_%h_%t.gz
Set adequate send and receive buffer sizes.

  1. Edit the /etc/sysctl.conf file on the host and paste the following content in it:

    net.core.rmem_default=67108864
net.core.rmem_max=67108864
net.core.wmem_default=67108864
net.core.wmem_max=6710886
  2. Run the command sysctl -p /etc/sysctl.conf to load the new sysctl.conf values on the host.

Port Requirements

Juniper Cloud-Native Router listens on certain TCP and UDP ports. This section lists the port requirements for the cloud-native router.

Table 5: Cloud-Native Router Listening Ports
Protocol Port Description
TCP 8085 vRouter introspect—Used to gain internal statistical information about vRouter
TCP 8070 Telemetry Information—Used to see telemetry data from the Cloud-Native Router vRouter
TCP 8072 Telemetry Information—Used to see telemetry data from Cloud-Native Router control plane
TCP 8075, 8076 Telemetry Information—Used for gNMI requests
TCP 9091 vRouter agent REST server—Used for communication between the vRouter agent and the CNI. Default port is 9091. Configure agentRestServerPort in values.yaml to use a different port if 9091 is already in use on the host.
TCP 9092 vRouter DPDK health check—Cloud-Native Router checks to ensure the vRouter DPDK is running. Default port is 9092. Configure dpdkHealthCheckPort in values.yaml to use a different port if 9092 is already in use on the host.
TCP 50052 gRPC port—Cloud-Native Router listens on both IPv4 and IPv6
TCP 8081 Cloud-Native Router Deployer Port
TCP 24 cRPD SSH
TCP 830 cRPD NETCONF
TCP 666 rpd
TCP 1883 Mosquito mqtt—Publish/subscribe messaging utility
TCP 9500 agentd on cRPD
TCP 21883 na-mqttd

TCP

50053

Default gNMI port that listens to the client subscription request
TCP 51051 jsd on cRPD
UDP 50055 Syslog-NG

Interface Naming for Mellanox NICs

When deploying Mellanox NICs in an OpenShift cluster, a conflict can arise between how OCP and Cloud-Native Router use interface names on those NICs. This might prevent your cluster from coming up.

Prior to installing Cloud-Native Router, either disable predictable interface naming or rename the Cloud-Native Router interfaces. The Cloud-Native Router interfaces are the interfaces that you want Cloud-Native Router to control.

Option 1: Disable predictable interface naming

  1. Before you start, ensure you have console access to the node.

  2. Edit /etc/default/grub and append net.ifnames=0 to GRUB_CMDLINE_LINUX_DEFAULT.

  3. Update grub.

  4. Reboot the node.

  5. Log back into the node. You might have to do this through the console if the network interfaces don't come back up.

  6. List the interfaces and take note of the names of the non-Cloud-Native Router and Cloud-Native Router interfaces.

  7. For all the non-Cloud-Native Router interfaces, update NetworkManager (or your network renderer) with the new interface names and restart NetworkManager.

  8. Repeat on all the nodes where you're installing the Cloud-Native Router vRouter.

Note:

Remember to update the fabric interfaces in your Cloud-Native Router installation helm chart with the new names of the Cloud-Native Router interfaces (or use subnets).

Option 2: Rename the Cloud-Native Router interfaces

  1. Create a /etc/udev/rules.d/00-persistent-net.rules file to contain the rules.

  2. Add the following line to the file:

    where <mac_address> is the MAC address of the interface you're renaming and <new_ifname> is the new name you want to assign to the interface (for example, jcnr-eth1).

  3. Add a corresponding line for each interface you're renaming. (You're renaming all the interfaces that Cloud-Native Router controls.)

  4. Reboot the node.

  5. Repeat on all the nodes where you're installing the Cloud-Native Router vRouter.

Note:

Remember to update the fabric interfaces in your Cloud-Native Router installation helm chart with the new names of the Cloud-Native Router interfaces (or use subnets).

Download Options

See Cloud-Native Router Software Download Packages.

Cloud-Native Router Licensing

See Manage Cloud-Native Router Licenses.