Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Understanding Aggregated Ethernet Interfaces and LACP for Switches

 

IEEE 802.3ad link aggregation enables you to group Ethernet interfaces to form a single link layer interface, also known as a link aggregation group (LAG) or bundle.

Aggregating multiple links between physical interfaces creates a single logical point-to-point trunk link or a LAG. The LAG balances traffic across the member links within an aggregated Ethernet bundle and effectively increases the uplink bandwidth. Another advantage of link aggregation is increased availability, because the LAG is composed of multiple member links. If one member link fails, the LAG continues to carry traffic over the remaining links.

Note

On QFX5100, EX4600, QFX10002 standalone switches, and on a QFX5100 Virtual Chassis and EX4600 Virtual Chassis, you can configure a mixed rate of link speeds for the aggregated Ethernet bundle. Only link speeds of 40G and 10G are supported. Load balancing will not work if you configure link speeds that are not supported.

Note

The QFX5200 switches do not support mixed rate aggregated Ethernet bundles.

Link Aggregation Control Protocol (LACP) is a subcomponent of the IEEE 802.3ad standard and is used as a discovery protocol.

Note

To ensure load balancing across the aggregated Ethernet (AE) interfaces on a redundant server Node group, the members of the AE must be equally distributed across the redundant server Node group.

Note

During a network Node group switchover, traffic might be dropped for a few seconds.

Link Aggregation Group

You configure a LAG by specifying the link number as a physical device and then associating a set of interfaces (ports) with the link. All the interfaces must have the same speed and be in full-duplex mode. Juniper Networks Junos operating system (Junos OS) for EX Series Ethernet Switches assigns a unique ID and port priority to each interface. The ID and priority are not configurable.

The number of interfaces that can be grouped into a LAG and the total number of LAGs supported on a switch varies according to switch model. Table 1 lists the EX Series switches and the maximum number of interfaces per LAG and the maximum number of LAGs they support.

Note

For Junos OS Evolved, there is no restriction on the maximum number of AE interfaces in a mixed-rate AE bundle. Because all child logical interfaces belong to same AE physical interface and share the same selector, using much less load balance memory, mixed-rate AE interface configurations should go through even if they exceed 64 logical interfaces.

Table 1: Maximum Interfaces per LAG and Maximum LAGs per Switch

Switch

Maximum Interfaces per LAG

Maximum LAGs

EX2200

8

32

EX2300

8

128

EX3200

8

32

EX3300 and EX3300 Virtual Chassis

8

32

EX3400

16

128

EX4200 and EX4200 Virtual Chassis

8

111

EX4300 and EX4300 Virtual Chassis

16

128

EX4500, EX4500 Virtual Chassis, EX4550, and EX4550 Virtual Chassis

8

111

EX4600

32

128

EX6200

8

111

EX8200

12

255

EX8200 Virtual Chassis

12

239

EX9200

64

150

To create a LAG:

  1. Create a logical aggregated Ethernet interface.

  2. Define the parameters associated with the logical aggregated Ethernet interface, such as a logical unit, interface properties, and Link Aggregation Control Protocol (LACP).

  3. Define the member links to be contained within the aggregated Ethernet interface—for example, two 10-Gigabit Ethernet interfaces.

  4. Configure LACP for link detection.

Keep in mind these hardware and software guidelines:

  • For Junos OS Evolved, when a new interface is added as a member to the aggregated Ethernet bundle, a link flap event is generated. When you add an interface to the bundle, the physical interface is deleted as a regular interface and then added back as a member. During this time, the details of the physical interface are lost.

  • Up to 32 Ethernet interfaces can be grouped to form a LAG on a redundant server Node group, a server Node group, and a network Node group on a QFabric system. Up to 48 LAGs are supported on redundant server Node groups and server Node groups on a QFabric system, and up to 128 LAGs are supported on network Node groups on a QFabric system. You can configure LAGs across Node devices in redundant server Node groups, server Node groups, and network Node groups.

    Note

    If you try to commit a configuration containing more than 32 Ethernet interfaces in a LAG, you will receive an error message saying that the group limit of 32 has been exceeded, and the configuration checkout has failed.

  • Up to 64 Ethernet interfaces can be grouped to form a LAG, and up to 448 LAGs are supported on QFX3500, QFX3600, EX4600, and OCX Series switches, and up to 1,000 LAGs are supported on QFX5100, QFX5200, QFX5110, QFX10002, QFX10008, and QFX10016 switches.

    Note

    If you try to commit a configuration containing more than 64 Ethernet interfaces in a LAG, you will receive an error message saying that the group limit of 64 has been exceeded, and the configuration checkout has failed.

  • Up to 64 Ethernet interfaces can be grouped to form a LAG, and In a Junos Fusion, up to 1,000 LAGs are supported on QFX10002 switches acting as aggregation devices.

  • The LAG must be configured on both sides of the link.

  • The interfaces on either side of the link must be set to the same speed and be in full-duplex mode.

    Note

    Junos OS assigns a unique ID and port priority to each port. The ID and priority are not configurable.

  • QFabric systems support a special LAG called an FCoE LAG, which enables you to transport FCoE traffic and regular Ethernet traffic (traffic that is not FCoE traffic) across the same link aggregation bundle. Standard LAGs use a hashing algorithm to determine which physical link in the LAG is used for a transmission, so communication between two devices might use different physical links in the LAG for different transmissions. An FCoE LAG ensures that FCoE traffic uses the same physical link in the LAG for requests and replies in order to preserve the virtual point-to-point link between the FCoE device converged network adapter (CNA) and the FC SAN switch across a QFabric system Node device. An FCoE LAG does not provide load balancing or link redundancy for FCoE traffic. However, regular Ethernet traffic uses the standard hashing algorithm and receives the usual LAG benefits of load balancing and link redundancy in an FCoE LAG. See Understanding FCoE LAGs for more information.

Link Aggregation Control Protocol (LACP)

LACP is one method of bundling several physical interfaces to form one logical aggregated Ethernet interface. By default, Ethernet links do not exchange LACP protocol data units (PDUs), which contain information about the state of the link. You can configure Ethernet links to actively transmit LACP PDUs, or you can configure the links to passively transmit them, sending out LACP PDUs only when the Ethernet link receives them from the remote end. The LACP mode can be active or passive. The transmitting link is known as the actor, and the receiving link is known as the partner. If the actor and partner are both in passive mode, they do not exchange LACP packets, and the aggregated Ethernet links do not come up. If either the actor or partner is active, they do exchange LACP packets. By default, LACP is in passive mode on aggregated Ethernet interfaces. To initiate transmission of LACP packets and response to LACP packets, you must enable LACP active mode. You can configure both VLAN-tagged and untagged aggregated Ethernet interfaces without LACP enabled. LACP is defined in IEEE 802.3ad, Aggregation of Multiple Link Segments.

LACP was designed to achieve the following:

  • Automatic addition and deletion of individual links to the LAG without user intervention.

  • Link monitoring to check whether both ends of the bundle are connected to the correct group.

In a scenario where a dual-homed server is deployed with a switch, the network interface cards form a LAG with the switch. During a server upgrade, the server might not be able to exchange LACP PDUs. In such a situation, you can configure an interface to be in the up state even if no PDUs are exchanged. Use the force-up statement to configure an interface when the peer has limited LACP capability. The interface selects the associated LAG by default, whether the switch and peer are both in active or passive mode. When PDUs are not received, the partner is considered to be working in the passive mode. Therefore, LACP PDU transmissions are controlled by the transmitting link.

If the remote end of the LAG link is a security device, LACP might not be supported because security devices require a deterministic configuration. In this case, do not configure LACP. All links in the LAG are permanently operational unless the switch detects a link failure within the Ethernet physical layer or data link layers.

When LACP is configured, it detects misconfigurations on the local end or the remote end of the link. Thus, LACP can help prevent communication failure:

  • When LACP is not enabled, a local LAG might attempt to transmit packets to a remote single interface, which causes the communication to fail.

  • When LACP is enabled, a local LAG cannot transmit packets unless a LAG with LACP is also configured on the remote end of the link.