Preparing for Junos Node Slicing Setup

To set up Junos node slicing, you must directly connect a pair of external x86 servers to the MX Series router. Besides the management port for the Linux host, each server also requires two additional ports for providing management connectivity for the JDM and the GNF VMs, respectively, and two ports for connecting to the MX Series router.

Figure 1 shows how an MX2020 router is connected to a pair of x86 external servers.

Figure 1: MX2020 Router—External x86 Server Connectivity

According to the example in Figure 1, em1, em2, and em3 on the x86 servers are the ports that are used for the management of the Linux host, the JDM and the GNFs, respectively. p3p1 and p3p2 on each server are the two 10-Gbps ports that are connected to the Control Boards of the MX Series router.

For more information on the XGE ports of the MX Series router Control Board (CB) mentioned in Figure 1, see:

• SCBE2-MX Description (for MX960 and MX480)

  Note:

  The XGE port numbers are not labeled on the SCBE2. On a vertically oriented SCBE2, the upper port is XGE-0 and the lower port is XGE-1. On a horizontally oriented SCBE2, the left port is XGE-0 and the right port is XGE-1.

• REMX2K-X8-64G and REMX2K-X8-64G-LT CB-RE Description (for MX2010 and MX2020)

For Junos node slicing, the BIOS of the x86 server CPUs should be set up such that:

• Hyperthreading is disabled.

• The CPU cores always run at their rated frequency.

• The CPU cores are set to reduce jitter by limiting C-state use.

To find the rated frequency of the CPU cores on the server, run the Linux host command lscpu, and check the value for the field Model name. See the following example:

To find the frequency at which the CPU cores are currently running, run the Linux host command grep MHz /proc/cpuinfo and check the value for each CPU core.

On a server that has the BIOS set to operate the CPU cores at their rated frequency, the observed values for the CPU cores will all match the rated frequency (or be very close to it), as shown in the following example.

On a server that does not have the BIOS set to operate the CPU cores at their rated frequency, the observed values for the CPU cores do not match the rated frequency, and the values could also vary with time (you can check this by rerunning the command).

To set the x86 server BIOS system profile to operate the CPU cores at their rated frequency, reduce jitter, and disable hyperthreading, consult the server manufacturer, because these settings vary with server model and BIOS versions.

Typical BIOS system profile settings to achieve this include:

• Logical processor: set to Disabled.

• CPU power management: set to Maximum performance.

• Memory frequency: set to Maximum performance.

• Turbo boost: set to Disabled.

• C-states and C1E state: set to Disabled.

• Energy efficient policy: set to Performance.

• Monitor/Mwait: set to Disabled.

A custom BIOS system profile might be required to set these values.

In Junos node slicing, each GNF VM is assigned dedicated CPU cores. This assignment is managed by Juniper Device Manager (JDM). On each x86 server, JDM requires that all CPU cores other than CPU cores 0 and 1 be reserved for Junos node slicing – and in effect, that these cores be isolated from other applications. CPU cores 2 and 3 are dedicated for GNF virtual disk and network I/O. CPU cores 4 and above are available for assignment to GNF VMs. To reserve these CPU cores, you must set the isolcpus parameter in the Linux GRUB configuration as described in the following procedure:

For x86 servers running Red Hat Enterprise Linux (RHEL) 9, perform the following steps:

1. Determine the number of CPU cores on the x86 server. Ensure that hyperthreading has already been disabled, as described in x86 Server CPU BIOS Settings. You can use the Linux command lscpu to find the total number of CPU cores, as shown in the following example:

   Here, there are 24 cores (12 x 2). The CPU cores are numbered as core 0 to core 23.

2. As per this example, the isolcpus parameter must be set to ’isolcpus=4-23’ (isolate all CPU cores other than cores 0, 1, 2, and 3 for use by the GNF VMs). The isolcpus parameter is set to ’isolcpus=4-23’ because of the following:

   • On each x86 server, JDM requires that all CPU cores other than CPU cores 0 and 1 be reserved for Junos node slicing.

   • CPU cores 2 and 3 are dedicated for GNF virtual disk and network I/O.

   Note:

   Previously, the isolcpus parameter 'isolcpus=2-23' was used. This has now been updated to 'isolcpus=4-23'. For more information, see KB35301.

   To set the isolcpus parameter in the Linux GRUB configuration file, follow the procedure described in the section Isolating CPUs from the process scheduler in this Red Hat document. A summary of the section is as follows:

   1. Edit the Linux GRUB file /etc/default/grub to append the isolcpus parameter to the variable GRUB_CMDLINE_LINUX, as shown in the following example:

   2. Run the Linux shell command grub2-mkconfig to generate the updated GRUB file as shown below:

      If you are using legacy BIOS, issue the following command:

      If you are using UEFI, issue the following command:

   3. Reboot the x86 server.

   4. Verify that the isolcpus parameter has now been set, by checking the output of the Linux command cat /proc/cmdline, as shown in the following example:

For x86 servers running Ubuntu 20.04, perform the following steps:

1. Determine the number of CPU cores on the x86 server. Ensure that hyperthreading has already been disabled, as described in x86 Server CPU BIOS Settings. You can use the Linux command lscpu to find the total number of CPU cores.

2. Edit the /etc/default/grub file to append the isolcpus parameter to the variable GRUB_CMDLINE_LINUX_DEFAULT, as shown in the following example:

3. To update the changes, run update-grub.

4. Reboot the server.

5. Verify that the isolcpus parameter has now been set, by checking the output of the Linux command cat /proc/cmdline.

If you are using Intel X710 NIC, ensure that you have the latest driver (2.4.10 or later) installed on the x86 servers, and that X710 NIC firmware version is 18.5.17 or later.

You need to first identify the X710 NIC interface on the servers. For example, this could be p3p1.

You can check the NIC driver version by running the Linux command ethtool -i interface. See the following example:

Refer to the Intel support page for instructions on updating the driver.

You need the following packages for building the driver:

• For RedHat:

  • kernel-devel

  • Development Tools

• For Ubuntu:

  • make

  • gcc

If you are using RedHat, run the following commands to install the packages:

If you are using Ubuntu, run the following commands to install the packages:

The x86 servers must have Red Hat Enterprise Linux (RHEL) 9 or Ubuntu 20.04 LTS installed.

For RHEL 9, install the following additional packages, which can be downloaded from the Red Hat Customer Portal.

• python-psutil-1.2.1-1.el7.x86_64.rpm

• net-snmp-5.7.2-24.el7.x86_64.rpm

• net-snmp-libs-5.7.2-24.el7.x86_64.rpm

• libvirt-snmp-0.0.3-5.el7.x86_64.rpm

Only for Junos OS Releases 17.4R1 and earlier, and for 18.1R1, if you are running RHEL 9, also install the following additional package:

• libstdc++-4.8.5-11.el7.i686.rpm

For Ubuntu 20.04, install the following packages:

• python-psutil

Only for Junos OS Releases 17.4R1 and earlier, and for 18.1R1, if you are running Ubuntu, also install the following additional package:

• libstdc++6:i386

Complete the following steps before you start installing the JDM:

• Ensure that the MX Series router is connected to the x86 servers as described in Connecting the Servers and the Router.

• Power on the two x86 servers and both the Routing Engines on the MX Series router.

• Identify the Linux host management port on both the x86 servers. For example, em1.

• Identify the ports to be assigned for the JDM and the GNF management ports. For example, em2 and em3.

• Identify the two 10-Gbps ports that are connected to the Control Boards on the MX Series router. For example, p3p1 and p3p2.