Customize PFC X-ON Threshold and Per-Queue Alpha Values

When you configure a congestion notification profile on an ingress port, lossless traffic is mapped to lossless priority groups. You configure these priority groups with priority-based flow control (PFC) X-OFF and X-ON thresholds. In case of congestion at an egress port, these priority groups ensure that the ingress port generates the PFC frames toward the peer based on the configured thresholds. When the shared occupancy or receive buffer on an ingress priority group reaches its PFC X-OFF limit, the corresponding priority group transmits the PFC pause frame to the egress peer. The peer temporarily stops transmitting packets to give the device time to resolve the traffic congestion.

The X-ON threshold is a buffer limit that is shared by the priority group. When the buffer usage on the ingress priority group drops below this PFC X-ON limit, the priority group sends a PFC message to the peer so it can resume packet transmission. Make sure the device has enough time to resolve the congestion. You must also ensure that traffic is not paused for long enough to cause disruption to your network. To optimize the downtime during a PFC pause storm, adjust the X-ON threshold through the congestion notification profile (CNP).

You can globally adjust the limit of buffers that each queue can consume from the shared buffer pool. The shared buffer pool is based on a dynamic threshold setting called the alpha value. You can configure a scheduler with different dynamic buffer threshold values for different queues, thereby controlling the shared buffer access by individual queues.

There are some additional considerations to keep in mind when configuring this feature with a high XON offset.

As part of the PFC feature, the hardware supports a PFC refresh functionality. When a priority group experiences congestion and the current buffer utilization exceeds the PFC XOFF threshold, the device sends a PFC XOFF frame to the peer device. If the buffer utilization does not fall back to the PFC XON threshold within the default PFC refresh time, the port generates a new PFC XOFF refresh frame and sends it to the peer device. For a 100G port, the default refresh time is 262 microseconds. This is why multiple PFC XOFF frames may be observed before a PFC XON frame is sent.

This behavior is expected for priority groups with a higher PFC XON offset. However, the PFC refresh timer operates on a per-port basis. Therefore, when the per-port PFC refresh timer expires, the port triggers PFC refresh XOFF frames for all priority groups that are in the XOFF state at that time. The hardware cannot distinguish which priority group's refresh timer has expired. As a result, even for a priority group with the default XON offset, the device might send multiple refresh XOFF frames continuously. This behavior is due to the expiration of the port-level PFC refresh timer. The many XOFF frames could cause the peer device to detect a PFC storm for that priority group. This could activate the PFC watchdog.

We recommend that if you set a very high XON offset for any priority group on a port, you should configure the peer device with a longer PFC watchdog detection timer. For example, if you set a PFC XON offset value of 10,000 for a priority group, the peer device should have a PFC watchdog detection timer of at least 10 milliseconds.