Load Balancing for AI-ML Data Centers

Significant load balancing challenges arise when AI-ML data centers process elephant flows. If elephant flows are not load-balanced properly across the network, they are likely to cause traffic congestion. When traffic congestion does occur, ineffective load balancing can compound the problem by inadvertently directing traffic to already congested links. Junos OS Evolved offers several types of load-balancing configurations that are optimized for the challenges of elephant flows.

As the network administrator, you can configure three main types of load balancing on your network:

• Static load balancing (SLB)—In SLB, you configure certain types of traffic to always use certain links. SLB is the most basic type of load balancing.

• Dynamic load balancing (DLB)—DLB dynamically chooses the link for a traffic flow based on the size of the traffic queue and the local link bandwidth utilization. DLB also checks the health of a link before rerouting traffic. DLB is more effective at avoiding traffic congestion than SLB.

  DLB has several modes and types that allow for customization, including:

  • Selective DLB—Selectively enable DLB for certain per-packet scenarios and use SLB for others.

  • Flowlet mode—In flowlet mode, DLB tracks the status of flows using an inactivity timer. When the inactivity timer expires for a particular flow, DLB rechecks whether that link is still optimal for that flow. If the link is no longer optimal, DLB selects a new egress link.

  • Reactive path rebalancing—Use this enhancement to DLB to move the traffic to a better quality link even when flowlet mode is enabled.

• Global load balancing (GLB)—GLB is an improvement on DLB. While DLB takes into account only the local link bandwidth utilization, GLB has visibility into the bandwidth utilization of links at the next-to-next-hop (NNH) level. GLB can reroute traffic flows to avoid traffic congestion farther out in the network than what DLB can detect.

You can use these different load balancing techniques in parallel within your AI-ML data center fabric.

Table 1: Dynamic Load Balancing

Topic	Use Case
Dynamic Load Balancing (DLB) Overview	Learn why DLB is particularly useful for handling elephant flows.
Selective Dynamic Load Balancing (DLB)	Selective DLB is useful when an elephant flow encounters links that are too small for the entire data flow. In this scenario, selective DLB can calculate the optimal use of the links' available bandwidth in the data center fabric.
Customize Egress Port Link Quality Metrics for DLB	You (the network administrator) can customize the way DLB assigns quality metrics of egress ports so that DLB selects the optimal link.
Configure Flowset Table Size in DLB Flowlet Mode	In environments such as AI-ML data centers that must handle large numbers of flows, it is particularly useful for DLB to use a larger flowset table size. When each ECMP group can accommodate a large number of flows, DLB achieves better flow distribution across the ECMP member links.
Reactive Path Rebalancing	Reactive path rebalancing addresses limitations of DLB by enabling the user to move the traffic to a better-quality link even when flowlet mode is enabled.

Table 2: GLB

Topic	Use Case
Global Load Balancing (GLB)	In large-scale AI/ML data center deployments, GLB is used in multi-stage Clos topologies to support increasing numbers of devices and GPUs. These topologies introduce additional path diversity, allowing GLB to make more effective load-balancing decisions across multiple network layers.
Configure GLB on 3-CLOS IP Fabric with Multilinks	In a Clos network, congestion on the first two next hops impacts the load balancing decisions of the local node and the previous hop nodes and triggers global load balancing (GLB). GLB on 3-Stage Clos topologies with multilink between spine and top-of-rack switches mitigates congestion.

Table 3: Other Load Balancing Features

Topic	Use Case
Queue-Pair Hashing for RDMA Flows	You can enhance load balancing on your AI-ML training network by including the destination queue pair in the hash calculation. This is called queue-pair hashing.
BGP Deterministic Path Forwarding in a CLOS Network	BGP DPF selects a path based on the specified logical fabric to ensure optimum link utilization.
Weighted Packet Spray	Weighted Packet Spray (WPS) is a load balancing technique that distributes traffic unevenly across multiple links based on their configured link bandwidth values.