ON THIS PAGE
RDMA over Converged Ethernet version 2
This topic briefly describes the RDMA over Converged Ethernet version 2 (ROCEv2) feature and its configuration on the QFX5220, QFX5230, QFX5240, and QFX5241 Switches.
Overview
Juniper Networks supports ROCEv2 primarily on its QFX Series Switches including QFX5220, QFX5230, QFX5240, and QFX5241 for building high performance lossless Ethernet fabrics for AI/ML workloads and storage. This feature is ideal for boosting data center efficiency, reducing overall complexity, and increasing data delivery performance. ROCEv2 traffic flow can coexist with other types of traffic flows in the network. ROCEv2-based Ethernet fabric uses UDP protocol whereas traditional Ethernet fabric uses TCP. Explicit congestion notification (ECN) and priority-based flow control (PFC) are some of the benefits that ROCEv2 offers over other Ethernet traffic flows. Thus, RDMA traffic is given priority over other traffic flows to read and write data to the server.
In the following section, you will learn to configure and verify ROCEv2 configuration on your Juniper QFX Series Switches. The configuration includes configuring ECN and PFC. For more information about data center quantized congestion notification (DCQCN), see Understanding DCQCN
Configure ROCEv2
To enable DCQCN, configure both ECN and PFC for a traffic flow.
Configure classifiers for ROCEv2 traffic and for Congestion Notification Packets (CNP).
set class-of-service classifiers dscp mydscp forwarding-class CNP loss-priority FC-DCSP code-points DSCP-CNP set class-of-service classifiers dscp mydscp forwarding-class NO-LOSS loss-priority FC-NO-LOSS code-points DSCP-NO-LOSS set interfaces interface-name unit unit classifiers dscp mydscp
Configure ECN on the egress port for a lossless flow.
set class-of-service drop-profiles dp1 interpolate fill-level Min-Fill-Percentage drop-probability Min-Drop-Prob set class-of-service drop-profiles dp1 interpolate fill-level Max-Fill-Percentage drop-probability Max-Drop-Prob set class-of-service schedulers s1 drop-profile-map loss-priority loss|high protocol ip|non-ip drop-profile dp1 set class-of-service schedulers s1 explicit-congestion-notification set class-of-service schedulers s2-cnp transmit-rate percent cnp-rate-percent set class-of-service schedulers s2-cnp priority priority set class-of-service scheduler-maps sm1 forwarding-class NO-LOSS scheduler s1 set class-of-service scheduler-maps sm1 forwarding-class CNP scheduler s2-cnp set class-of-service interfaces interface-name scheduler-map sm1
Configure PFC on the ingress port for the same lossless flow.
set class-of-service classifiers dscp LOSSLESS-DSCP code-point DSCP-Code-Point forwarding-class NO-LOSS loss-priority loss-priority set class-of-service interfaces interface-name unit unit classifiers dscp LOSSLESS-DSCP set class-of-service rewrite-rules ieee-802.1 LOSSLESS-PCP forwarding-class NO-LOSS loss-priority loss-priority code-point PCP-Code-Point set class-of-service interfaces interface-name unit unit rewrite-rules ieee-802.1 PCP-Name set protocols dcbx interface interface-name set protocols dcbx interface interface-name priority-flow-control priority pfc-priority
Configure the shared buffers.
set class-of-service shared-buffer ingress buffer-partition lossless percent ingress-lossless-percent set class-of-service shared-buffer ingress buffer-partition lossless dynamic-threshold ingress-lossless-dynamic-threshold set class-of-service shared-buffer ingress buffer-partition lossy percent ingress-lossy-percent set class-of-service shared-buffer ingress buffer-partition lossless-headroom percent ingress-lossless-headroom-percent set class-of-service shared-buffer egress buffer-partition lossless percent engress-lossless-percent set class-of-service shared-buffer egress buffer-partition lossy percent engress-lossy-percent
Note:You must follow these rules to commit the configuration on platforms running Junos OS Evolved:
You must configure all three of the ingress partitions.
The sum of the ingress shared buffer configuration for all partitions must be 100 percent.
For lossy and lossless buffer partitions both the ingress and egress buffer-partition percentages should be equal.
Setting
dynamic-thresholdfor the lossless ingress buffer partition is optional. ECN uses this option for the threshold calculation on lossless queues. If you don't configure this option,dynamic-thresholduses its default value of 7.Configure forwarding classes and assign queues.
set class-of-service forwarding-classes class CNP queue-num queue-num set class-of-service forwarding-classes class NO-LOSS queue-num queue-num set class-of-service forwarding-classes class NO-LOSS no-loss set class-of-service forwarding-classes class NO-LOSS pfc-priority pfc-priority
Verify your configuration.
show class-of-service drop-congestion-notification show class-of-service drop-congestion-notification interface show class-of-service forwarding-classes show class-of-service interfaces
Commit your configuration.
[edit class-of-service] user@host# commit