Scheduler Node Scaling on the Trio MPC/MIC Overview
The Trio MPC/MIC hardware supports multiple levels of scheduler nodes. In per-unit-scheduling mode, each logical interface (unit) can have 4 or 8 queues and has a dedicated level 3 scheduler node. Scheduler nodes can be a one of four levels: the queue itself (level 4), the logical interface or unit (level 3), the interface set or virtual LAN (VLAN) collection (level 2), or the physical interface or port (level 1). For more information about hierarchical scheduling levels, see Configuring Hierarchical Schedulers for CoS.
The maximum number of scheduler nodes supported by the Trio MPCs/MICs at each level varies:
- Level 1 (port): 128
- Level 2 (interface set): 8,000 (16,000 for the 40G Trio)
- Level 3 (unit): 32,000 (64,000 for the 40G Trio)
- Level 4 (queues): 256,000
In per-unit scheduling mode, each logical interface unit has its own dedicated level 3 node and all logical interface units share a common level 2 node (one per port). This scheduling mode is shown in Figure 21.
Figure 21: Trio MPC/MIC Per-unit Scheduler Node Scaling
In this case, in per-unit scheduling mode, the level 2 node is a dummy node. The maximum number of logical interfaces that can have a CoS profile is 32,000 (64,000 for the 40G Trio), each with 4 or 8 queues.
In hierarchical scheduling mode, when an interface set has a CoS profile configured and one of its child nodes also has a CoS profile configured, the interface set is considered to be an internal node and is mapped to a level 2 node. This also involves one additional level 3 node in order to map any remaining traffic from child logical interfaces that do not have their own CoS profiles. This case is shown in Figure 22.
Figure 22: Trio MPC/MIC Hierarchical Scheduling Node Scaling
In this case, the maximum number of interface sets for remaining traffic is 8,000 (16,000 for the 40G Trio), each with 4 or 8 queues. The maximum number of logical interfaces that can have their own CoS profiles is 24,000 (48,000 for the 40G Trio), each with 4 or 8 queues.
When an interface set has a CoS profile but none of its child logical interfaces has a CoS profile, then the interface set is considered to be a leaf node and will have one level 2 and one level 3 node. The level 3 node is for the interface set’s remaining traffic. This case is shown in Figure 23.
Figure 23: Hierarchical Scheduler Leaf Node
In this case, the maximum number of interface sets is 8,000 (16,000 for the 40G Trio), each with 4 or 8 queues.
When a logical interface is not part of any interface set but has its own CoS profile, then it will have its own dedicated level 2 and level 3 scheduling nodes. This case is shown in Figure 24.
Figure 24: Profiles Without Logical Interface Sets
In this case, the maximum number of logical interfaces that can have CoS profiles is 8,000 (16,000 for the 40G Trio), each with 4 or 8 queues.
The logical interfaces share a common level 2 node (one per port). In hierarchical-scheduling mode, a set of logical interfaces, each with 4 or 8 queues, has a level 2 CoS profile and one of its logical interface children has a level 3 CoS profile. To better control system resources in hierarchical-scheduling mode, you can limit the number of hierarchical levels in the scheduling hierarchy to two. In this case, all logical interfaces and interface sets with CoS profiles share a single (dummy) level 2 node, so the maximum number of logical interfaces with CoS profiles is increased (the interface sets must be at level 3). To configure scheduler node scaling, include the hierarchical-schedulers statement with the maximum-hierarchy-levels option at the [edit interfaces xe-fpc/pic/port] hierarchy level. The only supported value is 2.
When this statement is configured so that all logical interfaces and interface sets with a CoS profile share a single dummy level 2 node, the maximum number of logical interface that can have a CoS profile is increased to 32,000 (64,000 for the 40G Trio), each with 4 or 8 queues.
 | Note: Level 3 interface sets are supported by the maximum-hierarchy-levels option, but level 2 interface sets are not supported. If you configure level 2 interface sets with the maximum-hierarchy-levels option, you generate PFE errors. |
The Trio MPCs/MICs also support the q-pic-large-buffer statement at the [edit chassis fpc fpc-number pic pic-number] hierarchy level. By default, 500 ms worth of buffer is supported when the delay buffer rate is less than 1 Gbps. By default, 100 ms worth of buffer is supported when the delay buffer rate is 1 Gbps or more. The maximum supported value for the delay buffer is 256 MB and the minimum value is 4 KB.
However, due to the limited number of drop profiles supported (255) and the large range of supported speeds, there can be differences between the user-configured value and the observed hardware value.
All tunnel interfaces have 100-ms buffers. The hugh-buffer-temporal statement is not supported.
The Trio MPCs/MICs take all Layer 1 and Layer 2 overhead bytes into account for all levels of the hierarchy, including preamble, interpacket gaps, frame check sequence, and cyclical redundancy check. Queue statistics also take these overheads into account when displaying byte statistics.
You can control how much overhead to count on an MPC/MIC with the traffic-manager statement and options. By default, no overhead is added to egress shaping statistics. You can configure the system to adjust the number of bytes to add to a packet to determine shaped session packet length by adding more bytes (up to 255) of overhead.
This example adds 12 bytes of overhead to the egress shaping statistics:
The Trio MPC/MIC does not support the excess-bandwidth-sharing statement. You can use the excess-rate statement in scheduler maps and traffic control profiles instead.
Input queuing is not supported on the Trio MPCs/MICs.
In contrast to the IQE and IQ2E PICs, the Trio MPCs/MICs scale down the guaranteed rate at the logical interface in oversubscribed PIR mode for the per-unit scheduler. For example, if there are two logical interface units with a shaping rate of 1 Gbps each on a 1-Gbps port (which is, therefore, oversubscribed 2 to 1), then the guaranteed rate on each unit is scaled down to 500 Mbps (scaled down by 2). This makes sure that the guaranteed rates are never oversubscribed.
With hierarchical schedulers in oversubscribed PIR mode, the guaranteed rate for every logical interface unit is set to zero. This means that the queue transmit rates are always oversubscribed.
Because in hierarchical oversubscribed PIR mode the queue transmit rates are always oversubscribed, the following are true:
- If the queue transmit rate is set as a percentage, then the guaranteed rate of the queue is set to zero; but the excess rate (weight) of the queue is set correctly.
- If the queue transmit rate is set as an absolute value and if the queue has guaranteed high or medium priority, then traffic up to the queue’s transmit rate is sent at that priority level. However, for guaranteed low traffic, that traffic is demoted to the excess low region. This means that best-effort traffic well within the queue’s transmit rate gets a lower priority than out-of-profile excess high traffic. This differs from the IQE and IQ2E PICs.
- If the queue transmit rate is set as a remainder, then it gets a guaranteed rate according to the amount left over from the logical interface unit’s shaping rate. As before, if the queue has guaranteed high or medium priority, then traffic up to the queue’s transmit rate is sent at that priority level. However, for guaranteed low traffic, that traffic is demoted to the excess low region. This means that best-effort traffic well within the queue’s transmit rate gets a lower priority than out-of-profile excess high traffic. This differs from the IQE and IQ2E PICs.
