vRR on KVM supports single-root I/O Virtualization (SR-IOV)
interface types. Single root I/O virtualization (SR-IOV) allows a physical function to appear
as multiple, separate vNICs. SR-IOV allows a device, such as a network adapter to have
separate access to its resources among various hardware functions. If you have a physical NIC
that supports SR-IOV, you can attach SR-IOV-enabled vNICs or virtual functions (VFs) to the
vRR instances to improve performance.
- BIOS requirement to enable SR-IOV; Ensure that Intel VT-d or AMD IOMMU are enabled in
the systems BIOS settings
- SR-IOV on VRR for KVM requires one of the following Intel NIC drivers:
- Intel X710 or XL710 using 40G ports and i40e driver
- Intel X520 or X540 using 10G ports and ixgbe driver
- Junos OS Release:
- Starting with Junos OS Release 17.4 onwards, support for 40G ports with Intel
X710/XL710 NICs is available on vRR.
- Starting with Junos OS Release 20.4R3, support for 10G ports with Intel X520/X540
NICs is available on vRR.
-
Look for NICs (devices) available on your host using the lshw -businfo -c
network command.
- In the output below: 10G NICs are enp2s0f0 and enp2s0f1
- 40G NICs are enp4s0f0 and enp4s0f1
root@vrr-host: lshw -businfo -c network
Bus info Device Class Description
========================================================
pci@0000:02:00.0 enp2s0f0 network 82599ES 10-Gigabit SFI/SFP+ Network Connection
pci@0000:02:00.1 enp2s0f1 network 82599ES 10-Gigabit SFI/SFP+ Network Connection
pci@0000:04:00.0 enp4s0f0 network Ethernet Controller XL710 for 40GbE QSFP+
pci@0000:04:00.1 enp4s0f1 network Ethernet Controller XL710 for 40GbE QSFP+
pci@0000:04:02.0 enp4s2 network Illegal Vendor ID
pci@0000:07:00.0 enp7s0f0 network I350 Gigabit Network Connection
pci@0000:07:00.1 enp7s0f1 network I350 Gigabit Network Connection
-
Remove any VMs on the host using the device that you plan to configure SR-IOV on.
-
Check the maximum number of VFs supported on the device (for example enp2s0f0 and
enp4s0f0)
cat /sys/class/net/ enp4s0f0/device/sriov_totalvfs 63
-
Configure the number of VFs you want on the device
echo 2 > /sys/class/net/enp4s0f0/device/sriov_numvfs (as a root user)
-
Verify if the VFs are created using the lspci command. You
should see Virtual Functions corresponding to the Ethernet Controller of the device.
lspci | grep Ethernet
04:00.0 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02) - PF1
04:00.1 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02) - PF2
04:02.0 Ethernet controller: Intel Corporation XL710/X710 Virtual Function (rev 02) - VF1
04:0a.0 Ethernet controller: Intel Corporation XL710/X710 Virtual Function (rev 02) - VF2
-
Replace the existing interface stanza for the network device to obtain a new XML.
Ensure to keep the management device entry as it is.
<interface type='hostdev' managed='yes'>
<mac address='30:b6:4f:60:f2:82'/> - the same as in a non-SR-IOV XML or pick a valid mac
<driver name='kvm'/>
<source>
<address type='pci' domain='0x0000' bus='0x04’ slot='0x00’ function='0x0'/> - get this value from lspci output for the VF - column 1 domain:bus:slot:function (domain is 0x0000 by default).
</source>
<alias name='hostdev0'/>
<address type='pci' domain='0x0000' bus='0x01' slot='0x01’ function='0x0'/> - keep the same as in the existing XML
</interface>
-
Bring up the VRR instance with new SRIOV based interfaces
-
Finally, to check if the VF devices that are created in Junos vRR, perform grep for
“renaming” in the boot messages
root@device> show system boot-messages | grep renaming
re_vrr_ifd_rename: renaming ixv* -> em* - 10G (ixv*)
re_vrr_ifd_rename: renaming ixlv* -> em* - 40G (ixlv*)
re_vrr_ifd_rename: renaming vtnet* -> em* - Management
