CoS on Virtual Chassis Fabric (VCF) EX4300 Leaf Devices (Mixed Mode)

VCF CoS in Mixed Mode with an EX4300 Leaf Device

In mixed mode, if all of the leaf devices are QFX5100, QFX3500, and QFX3600 switches, the full QFX Series CoS feature set is available, including data center bridging (DCB) features such as enhanced transmission selection (ETS, IEEE 802.1Qaz), priority-based flow control (PFC, IEEE 802.1Qbb), and Data Center Bridging Exchange Protocol (DCBX, an extension of LLDP, IEEE 802.1AB).

However, the EX4300 leaf device does not support DCB standards (ETS, PFC, DCBX). The lack of support for DCB standards means that the EX4300 leaf device does not support lossless transport. So a VCF that includes an EX4300 as a leaf device does not support lossless storage traffic such as Fibre Channel over Ethernet (FCoE).

In addition, a VCF with an EX4300 leaf device either does not support or has limited support for some other CoS features that the QFX Series switches support, including some buffer configuration features, some packet rewrite features, and Ethernet PAUSE (IEEE 802.3X).

Table 1 summarizes the CoS support on a VCF in mixed mode with one or more EX4300 leaf devices.

In addition to the CoS limitations shown in Table 1, using wild cards in a LAG configuration is not supported in mixed mode with one or more EX4300 leaf devices.

Scheduling on an EX4300 VCF Leaf Device

Because the EX4300 leaf device does not support ETS, the VCF translates the ETS scheduling configuration into the port scheduling configuration that the EX4300 device supports. The QFX5100 spine device uses two-tier ETS scheduling, as described in detail in Understanding CoS Hierarchical Port Scheduling (ETS).

Briefly, ETS allocates port bandwidth into forwarding class sets (priority groups) and forwarding classes (priorities) in a hierarchical manner. Each forwarding class set consists of individual forwarding classes, with each forwarding class mapped to an output queue.

Port bandwidth (minimum guaranteed bandwidth and maximum bandwidth) is allocated to each forwarding class set. Forwarding class set bandwidth is in turn allocated to the forwarding classes in the forwarding class set. If a forwarding class does not use its bandwidth allocation, other forwarding classes within the same forwarding class set can share the unused bandwidth. If the forwarding classes in a forwarding class set do not use the bandwidth allocated to that forwarding class set, other forwarding class sets on the port can share the unused bandwidth. (This is how ETS increases port bandwidth utilization, by sharing unused bandwidth among forwarding classes and forwarding class sets.)

However, the EX4300 leaf device supports port scheduling, not ETS. Port scheduling is a “flat” scheduling method that allocates bandwidth directly to forwarding classes in a non-hierarchical manner.

The VCF translates the two tiers of the ETS scheduling configuration (forwarding class sets and forwarding classes) into a single port scheduling configuration as follows:

• The bandwidth allocated to a forwarding class set is divided equally among the forwarding classes in the forwarding class set. (Traffic control profiles schedule bandwidth allocation to forwarding class sets.) The minimum guaranteed bandwidth (guaranteed-rate) and maximum bandwidth limit (shaping-rate) of the forwarding class set determine the guaranteed minimum bandwidth and the maximum bandwidth the forwarding classes receive, unless those values are different in the forwarding class scheduler configuration.

• If there is an explicit forwarding class bandwidth scheduler configuration, it overrides the forwarding class set configuration. Bandwidth scheduling values that are not explicitly configured in a forwarding class scheduler use the values from the forwarding class set (the traffic control profile configuration). Forwarding class schedulers control the minimum guaranteed bandwidth (transmit-rate), the maximum bandwidth (shaping-rate), and the priority (priority) for each forwarding class (output queue). Because the priority value is not configured at the forwarding class set level, the priority configured in the forwarding class scheduler is always used.

The following two scenarios illustrate how a VCF translates an ETS configuration into a port scheduling configuration:

Scenario 1

A forwarding class set named fc-set-1 has a configured guaranteed minimum bandwidth (guaranteed-rate) of 4G, and a configured maximum bandwidth (shaping-rate) of 5G.

Forwarding class set fc-set-1 consists of two forwarding classes, named fc-1 and fc-2:

• Forwarding class fc-1 has a guaranteed minimum bandwidth (transmit-rate) of 2.5G. There is no configured maximum bandwidth (shaping-rate).

• Forwarding class fc-2 has a guaranteed minimum bandwidth (transmit-rate) of 1.5G. There is no configured maximum bandwidth (shaping-rate).

On the EX4300 leaf device, the ETS configuration above is translated approximately to the following port scheduling configuration:

• Guaranteed minimum bandwidth—Because guaranteed minimum bandwidth has been explicitly configured in the forwarding class scheduler, forwarding class fc-1 receives a transmit rate of 2.5G and forwarding class fc-2 receives a transmit rate of 1.5G.

  Note:

  If there had been no forwarding class scheduler transmit-rate configuration, then the forwarding class set minimum guaranteed bandwidth of 4G would have been split evenly between the forwarding classes, with each forwarding class receiving a minimum guaranteed bandwidth rate of 2G.

• Maximum bandwidth—Because there is no explicit maximum bandwidth (shaping-rate configuration for the forwarding classes, the forwarding classes that belong to the forwarding class set receive an equal share of the maximum bandwidth configured at the forwarding class set level in the traffic control profile. Because the forwarding class set maximum bandwidth is 5G, forwarding classes fc-1 and fc-2 each receive a maximum bandwidth of 2.5G.

In this scenario, the minimum guaranteed bandwidth and the maximum bandwidth configured at the forwarding class set hierarchy level are achieved on the forwarding classes that belong to the forwarding class set. (This does not always happen, as Scenario 2 shows.) However, unused bandwidth is not shared the same way. For example, if forwarding class fc-1 experienced a burst of traffic at 3.5G, it would be limited to a maximum of 2.5G and traffic would be dropped. Using ETS, if forwarding class fc-2 was not using its allocated maximum bandwidth, then fc-1 could use (share) that unused bandwidth. But flat port scheduling does not share the unused bandwidth.

Scenario 2

A forwarding class set named fc-set-2 has a configured guaranteed minimum bandwidth (guaranteed-rate) of 6G, and a configured maximum bandwidth (shaping-rate) of 9G.

Forwarding class set fc-set-2 consists of three forwarding classes, named fc-3, fc-4, and fc-5:

• Forwarding class fc-3 has a guaranteed minimum bandwidth (transmit-rate) of 1G. There is no configured maximum bandwidth (shaping-rate).

• Forwarding class fc-4 has a maximum bandwidth (shaping-rate) of 2G. There is no configured guaranteed minimum bandwidth (transmit-rate).

• Forwarding class fc-5 has a guaranteed minimum bandwidth (transmit-rate) of 3G. There is no configured maximum bandwidth (shaping-rate).

On the EX4300 leaf device, the ETS configuration above is translated approximately to the following port scheduling configuration:

• Guaranteed minimum bandwidth—Two forwarding classes (fc-3 and fc-5) have an explicitly configured transmit rate, and one forwarding class (fc-4) does not. Forwarding classes fc-3 and fc-5 receive the minimum guaranteed bandwidth configured in their schedulers, so forwarding class fc-3 receives 1G guaranteed minimum bandwidth and forwarding class fc-5 receives 3G guaranteed minimum bandwidth.

  Forwarding class fc-4 does not have an explicitly configured transmit rate, so the port derives the minimum guaranteed bandwidth from the forwarding class set guaranteed rate. Forwarding class set fc-set-2 has a minimum guaranteed bandwidth (guaranteed-rate) of 6G, and there are three forwarding classes in the forwarding class set. Forwarding class fc-4 receives an equal share (one third) of the forwarding class set minimum guaranteed bandwidth. So forwarding class fc-4 is allocated a guaranteed minimum bandwidth (transmit-rate) of 2G (6G divided by 3 forwarding classes = 2G).

• Maximum bandwidth—Forwarding class fc-4 has an explicitly configured shaping rate, and forwarding classes fc-3 and fc-5 do not. Forwarding class fc-4 receives the maximum bandwidth configured in its scheduler, so forwarding class fc-4 receives a maximum bandwidth of 2G.

  Forwarding classes fc-3 and fc-5 do not have explicitly configured shaping rates, so the port derives the maximum bandwidth from the forwarding class set shaping rate. Forwarding class set fc-set-2 has a maximum bandwidth (shaping-rate) of 9G, and there are three forwarding classes in the forwarding class set. Forwarding classes fc-3 and fc-5 each receive an equal share (one third) of the forwarding class set shaping rate. So forwarding classes fc-3 and fc-5 are allocated a maximum bandwidth of 3G each (9G divided by 3 forwarding classes = 3G).

  Forwarding class fc-4 receives less maximum bandwidth than forwarding classes fc-3 and fc-5 because the explicitly configured shaping rate for forwarding class fc-4 is only 2G, and the explicit forwarding class configuration overrides the forwarding class set configuration.