Resilient Hashing on LAGs and ECMP groups

 

Resilient hashing helps minimize the flow remapping across equal cost multipath (ECMP) groups and LAGs in a load-balanced system. The topics below discuss the working, usage and configuring of resilient hashing on link aggregation groups (LAGs) and ECMP groups.

Understanding the Use of Resilient Hashing to Minimize Flow Remapping in LAGs/ECMP Groups

You use resilient hashing to minimize flow remapping across members of a LAG/ECMP group in a load-balanced system. You can configure resilient hashing in link aggregation groups (LAGs) and in equal cost multipath (ECMP) groups.

Why You Might Want to Use Resilient Hashing and How It Works with Static Hashing

Resilient hashing works in conjunction with the default static hashing algorithm. When members are added to or deleted from a LAG/ECMP group, the static hashing algorithm might remap destination paths. With resilient hashing, the chances of a flow being remapped are minimal if its path is unaffected by the LAG/ECMP group's member change. When a flow is affected by a member change, the Packet Forwarding Engine rebalances the flow by reprogramming the flow set table.

Resilient hashing thus provides the following benefits:

  • Minimizes traffic-distribution imbalances among members of a LAG/ECMP group when members are added to or deleted from the group.

  • Minimizes the impact on flows bound to unaffected members when a new member is added or an existing member is deleted from the group.

In normal hash-based load balancing, with the static hashing algorithm used alone, flows are assigned to members through the mathematical mod (%) operation. Any increase or decrease in the number of group members results in a complete remapping of flows to member IDs, as shown in the following example:

  • Member ID = Hash (key) mod (number of members in group)

  • Example:

    • Hash (key) = 10

    • 10 mod 5 = 0 (member with ID 0 is selected for flow)

    • 10 mod 4 = 2 (member with ID 2 is selected for the same flow when the number of members is decreased by 1)

Resilient hashing minimizes the destination path remapping when a member in the LAG/ECMP group is added or deleted.

When the flow is affected by a member change in the group, resilient hashing rebalances the flow by reprogramming the flow set table.

Table 1: Destination Path Results for Static Hashing and for Resilient Hashing When Members Are Added to or Deleted from LAGs

LAG/ECMP Group size

Normal (Static) Hashing Result

Resilient Hashing Result

Notes

4

Hash(10) % 4 = 2

Flow is assigned to member ID 2.

Flow is assigned to one of four group members based on flow set table entries.

Original LAG/ECMP group size is 4.

3

Hash(10) % 3 = 1

Flow is assigned to member ID 1.

Flow is assigned to same member as in the previous case.

Delete one member from original LAG/ECMP group. LAG/ECMP group size is 3.

5

Hash(10) % 5 = 0

Flow is assigned to member ID 0.

There is minimal redistribution of flows from other members to this newly added member.

Add one member to original LAG group. LAG/ECMP group size is 5.

Limitations and Caveats for Resilient Hashing

Notice the following limitation and caveats for the resilient hashing feature:

  • Resilient hashing applies only to unicast traffic.

  • Resilient hashing supports a maximum of 1024 LAGs, with each group having a maximum of 256 members.

  • Resilient hashing does not guarantee that traffic distribution is even across all group members—it depends on the traffic pattern and on the organization of the resilient hashing flow set table in hardware. Resilient hashing minimizes remapping of flows to destination links when members are added to or deleted from the group.

  • If resilient hashing is enabled on a LAG or ECMP group and if set forwarding-options enhanced-hash-key with one of the options hash-mode, inet, inet6, or layer2 is used, some flows might change destination links, because the new hash parameters might generate new hash indexes for the flows, and hence the new destination links.

  • Resilient hashing is not supported on Virtual Chassis port (VCP) links.

  • LAG-based resilient hashing is not supported on QFX5200 and QFX5210 switches. ECMP-based resilient hashing is supported on those switches.

Resilient Hashing on LAGs

A LAG combines Ethernet interfaces (members) to form a logical point-to-point link that increases bandwidth, provides reliability, and allows load balancing. Resilient hashing minimizes destination remapping behavior when a new member is added or deleted from the LAG.

A resilient hashing configuration on LAGs is per-aggregated-Ethernet-interface–based.

Resilient Hashing on ECMP

An ECMP group for a route contains multiple next-hop equal cost addresses for the same destination in the routing table. (Routes of equal cost have the same preference and metric values.)

Junos OS uses the static hashing algorithm to choose one of the next-hop addresses in the ECMP group to install in the forwarding table. Resilient hashing enhances ECMPs by minimizing destination remapping behavior when a new member is added or deleted from the ECMP group.

A resilient hashing configuration on ECMP is global—it applies to all ECMP groups.

Configuring Resilient Hashing for LAGs/ECMP Groups

You use resilient hashing to minimize flow remapping across members of a LAG/ECMP group in a load-balanced system. You can configure resilient hashing in link aggregation groups (LAGs) and in equal cost multipath (ECMP) sets.

This topic includes:

  1. Configuring Resilient Hashing on LAGs

  2. Configuring Resilient Hashing on ECMP Groups

Configuring Resilient Hashing on LAGs

Note

LAG-based resilient hashing is not supported on QFX5200 and QFX5210 switches. ECMP-based resilient hashing is supported on those switches.

To enable resilient hashing for a LAG:

  • Configure resilient hashing on the aggregated Ethernet interface:
    [edit interfaces]

    user@switch# set aex aggregated-ether-options resilient-hash

Configuring Resilient Hashing on ECMP Groups

To enable resilient hashing for ECMP groups:

  • Configure resilient hashing for ECMP:
    [edit forwarding-options]

    user@switch# set enhanced-hash-key ecmp-resilient-hash
Note

When resilient hashing is added or removed, the traffic distribution across all members of an ECMP group for a given flow are reprogrammed and, as a result, some flows might be remapped to new ECMP group members.