Aggregated Ethernet Interfaces Overview
Learn about aggregated Ethernet interfaces, LACP, and LAG.
Overview
IEEE 802.3ad link aggregation enables you to group Ethernet interfaces to form a single link layer interface, also known as a link aggregation group (LAG) or bundle.
Aggregating multiple links between physical interfaces creates a single logical point-to-point trunk link or a LAG. The LAG balances traffic across the member links within an aggregated Ethernet bundle and effectively increases the uplink bandwidth. Another advantage of link aggregation is increased availability, because the LAG is composed of multiple member links. If one member link fails, the LAG continues to carry traffic over the remaining links.
You can configure a mixed rate of link speeds for the aggregated Ethernet bundle. Link speeds of 10GbE, 40GbE, and 100GbE are supported. Load balancing does not work if you configure link speeds that are not supported.
Use Aggregated Ethernet interface to confirm platform and release support for specific features.
You can configure port channel using different SFP models between two endpoints keeping the same bandwidth.
For example:
switch 1 gig0/1 (SFP-10G-SR-S) --------- MX 1 gig0/1 (SFP-10G-SR-S)
switch 1 gig0/2 (SFP-10G-LR-S) --------- MX 1 gig0/2 (SFP-10G-LR-S)
Link Aggregation Control Protocol (LACP) is a subcomponent of the IEEE 802.3ad standard and is used as a discovery protocol.
To ensure load balancing across the aggregated Ethernet (AE) interfaces on a redundant server Node group, the members of the AE must be equally distributed across the redundant server Node group.
During a network Node group switchover, traffic might be dropped for a few seconds.
Link Aggregation Group (LAG)
You configure a LAG by specifying the link number as a physical device and then associating a set of interfaces (ports) with the link. All the interfaces must have the same speed and be in full-duplex mode. Juniper Networks Junos operating system (Junos OS) for EX Series Ethernet Switches assigns a unique ID and port priority to each interface. The ID and priority are not configurable.
The number of interfaces that can be grouped into a LAG and the total number of LAGs supported on a switch varies according to switch model. Additional Platform Information lists the EX Series switches and the maximum number of interfaces per LAG and the maximum number of LAGs they support.
LAGs with member links of different interface types, for example, ge and mge are not supported on multirate switches.
For Junos OS Evolved, the software does not impose a limit on the maximum number of aex in a mixed-rate aggregated Ethernet bundle. All child logical interfaces belong to the same aggregated Ethernet physical interface and share the same selector. So, much less load-balance memory and mixed-rate aex configurations should go through even if number exceeds 64 logical interfaces.
On QFX Series switches, if you try to commit a configuration containing more than 64 Ethernet interfaces in a LAG, you receive an error message. The error message says that the group limit of 64 has been exceeded and the configuration checkout has failed.
To create a LAG:
-
Create a logical aggregated Ethernet interface.
-
Define the parameters associated with the logical aggregated Ethernet interface, such as a logical unit, interface properties, and Link Aggregation Control Protocol (LACP).
-
Define the member links to be contained within the aggregated Ethernet interface—for example, two 10-Gigabit Ethernet interfaces.
-
Configure LACP for link detection.
Keep in mind these hardware and software guidelines:
-
For Junos OS Evolved, when a new interface is added as a member to the aggregated Ethernet bundle, a link flap event is generated. When you add an interface to the bundle, the physical interface is deleted as a regular interface and then added back as a member. During this time, the details of the physical interface are lost.
-
Up to 32 Ethernet interfaces can be grouped to form a LAG on a redundant server Node group, a server Node group, and a network Node group on a QFabric system. Up to 48 LAGs are supported on redundant server Node groups and server Node groups on a QFabric system, and up to 128 LAGs are supported on network Node groups on a QFabric system. You can configure LAGs across Node devices in redundant server Node groups, server Node groups, and network Node groups.
On a Qfabric system, if you try to commit a configuration containing more than 32 Ethernet interfaces in a LAG, you will receive an error message saying that the group limit of 32 has been exceeded, and the configuration checkout has failed.
-
The LAG must be configured on both sides of the link.
-
The interfaces on either side of the link must be set to the same speed and be in full-duplex mode.
Junos OS assigns a unique ID and port priority to each port. The ID and priority are not configurable.
-
QFabric systems support a special LAG called an FCoE LAG, which enables you to transport FCoE traffic and regular Ethernet traffic (traffic that is not FCoE traffic) across the same link aggregation bundle. Standard LAGs use a hashing algorithm to determine the physical link used for transmission. Thus, communication between two devices might use different physical links in the LAG for different transmissions. An FCoE LAG ensures that FCoE traffic uses the same physical link in the LAG for requests and replies. This preserves the virtual point-to-point link between the FCoE device converged network adapter (CNA) and the FC SAN switch across a QFabric system Node device. An FCoE LAG does not provide load balancing or link redundancy for FCoE traffic. However, regular Ethernet traffic uses the standard hashing algorithm and receives the usual LAG benefits of load balancing and link redundancy in an FCoE LAG. See Understanding FCoE LAGs for more information.
Configure Aggregated Ethernet Interfaces
Learn how to configure aggregated Ethernet interfaces. Includes a sample configuration as well.
To configure an aex:
Check the following guidelines while configuring aggregated Ethernet interfaces:
In general, aggregated Ethernet bundles support the features available on all supported interfaces that can become a member link within the bundle. As an exception, GbE IQ features and some newer GbE features are not supported in aggregated Ethernet bundles.
GbE IQ and SFP interfaces can be member links, but IQ- and SFP-specific features are not supported on the aggregated Ethernet bundle even if all the member links individually support those features.
Before you commit an aggregated Ethernet configuration, ensure that link mode is not configured on any member interface of the aggregated Ethernet bundle; otherwise, the configuration commit check fails.
Sample Aggregated Ethernet Interfaces Configuration
Aggregated Ethernet interfaces can use interfaces from different FPCs, DPCs, or PICs. The following configuration is sufficient to get an aggregated Gigabit Ethernet interface up and running.
[edit chassis]
aggregated-devices {
ethernet {
device-count 15;
}
}
[edit interfaces]
ge-1/3/0 {
gigether-options {
802.3ad ae0;
}
}
ge-2/0/1 {
gigether-options {
802.3ad ae0;
}
}
ae0 {
aggregated-ether-options {
link-speed 1g;
minimum-links 1;
}
}
vlan-tagging;
unit 0 {
vlan-id 1;
family inet {
address 10.0.0.1/24;
}
}
unit 1 {
vlan-id 1024;
family inet {
address 10.0.0.2/24;
}
}
unit 2 {
vlan-id 1025;
family inet {
address 10.0.0.3/24;
}
}
unit 3 {
vlan-id 4094;
family inet {
address 10.0.0.4/24;
}
}
}
Example: Configure Link Aggregation Between a QFX Series Switches and an Aggregation Switch
A QFX Series product allows you to combine multiple Ethernet links into one logical interface for higher bandwidth and redundancy. The ports that are combined in this manner are referred to as a link aggregation group (LAG) or bundle. The number of Ethernet links you can combine into a LAG depends on your QFX Series product model. You can configure LAGs to connect a QFX Series product or an EX4600 switch to other switches, like aggregation switches, servers, or routers. This example describes how to configure LAGs to connect a QFX3500, QFX3600, EX4600, QFX5100, and QFX10002 switch to an aggregation switch.
Requirements
This example uses the following software and hardware components:
Junos OS Release 11.1 or later for the QFX3500 and QFX3600 switches, Junos OS 13.2 or later for the QFX5100 and EX4600 switch, and Junos OS Release 15.1X53-D10 or later for QFX10002 switches.
One QFX3500, QFX3600, EX4600, QFX5100, or QFX10002 switch.
Overview and Topology
In this example, the switch has one LAG comprising two 10-Gigabit Ethernet interfaces. This LAG is configured in port-mode trunk (or interface-mode trunk) so that the switch and the VLAN to which it has been assigned can send and receive traffic.
Configuring the Ethernet interfaces as LAGs has the following advantages:
If one physical port is lost for any reason (a cable is unplugged or a switch port fails), the logical port transparently continues to function over the remaining physical port.
Link Aggregation Control Protocol (LACP) can optionally be configured for link monitoring and automatic addition and deletion of individual links without user intervention.
If the remote end of the LAG link is a security device, LACP might not be supported because security devices require a deterministic configuration. In this case, do not configure LACP. All links in the LAG are permanently operational unless the switch detects a link failure within the Ethernet physical layer or data link layers.
The topology used in this example consists of one switch with a LAG configured between two of its 10-Gigabit Ethernet interfaces. The switch is connected to an aggregation switch.
Table 1 details the topology used in this configuration example.
| Hostname | Base Hardware | Trunk Port |
|---|---|---|
switch |
QFX3500, QFX3600, EX4600, QFX5100, or QFX10002 switch |
|
Configuration
To configure a LAG between two 10-Gigabit Ethernet interfaces.
Procedure
CLI Quick Configuration
To quickly configure a LAG between two 10-Gigabit Ethernet interfaces on a switch, copy the following commands and paste them into the switch terminal window:
To configure a LAG using Enhanced Layer 2 Software—for example, on the EX4600, QFX5100, or
QFX10002 switch—use the interface-mode statement
instead of the port-mode statement. For ELS details,
see Using the Enhanced Layer 2 Software
CLI.
[edit] set chassis aggregated-devices ethernet device-count 1 set interfaces ae0 aggregated-ether-options minimum-links 1 set interfaces ae0 aggregated-ether-options link-speed 10g set interfaces ae0 unit 0 family ethernet-switching vlan members green set interfaces xe-0/0/2 ether-options 802.3ad ae0 set interfaces xe-0/0/3 ether-options 802.3ad ae0 set interfaces ae0 unit 0 family ethernet-switching port-mode trunk set interfaces ae0 aggregated-ether-options lacp active set interfaces ae0 aggregated-ether-options lacp periodic fast
Step-by-Step Procedure
To configure a LAG between a QFX Series switch and an aggregation switch:
Specify the number of LAGs to be created on the switch:
[edit chassis] user@switch# set aggregated-devices ethernet device-count 1
Specify the number of links that need to be present for the
ae0LAG interface to beup:[edit interfaces] user@switch# set ae0 aggregated-ether-options minimum-links 1
Specify the media speed of the
ae0link:[edit interfaces] user@switch# set ae0 aggregated-ether-options link-speed 10g
Specify the members to be included within the aggregated Ethernet bundle:
[edit interfaces] user@switch# set interfaces xe-0/0/2 ether-options 802.3ad ae0 [edit interfaces] user@switch# set interfaces xe-0/0/3 ether-options 802.3ad ae0
Assign a port mode of trunk to the
ae0link:Note:To configur a LAG using Enhanced Layer 2 Software—for example, on the EX4600, QFX5100, or QFX10002 switch—use the
interface-modestatement instead of theport-modestatement. For ELS details, see Using the Enhanced Layer 2 Software CLI.[edit interfaces] user@switch# set ae0 unit 0 family ethernet-switching port-mode trunk
or
[edit interfaces] user@switch# set ae0 unit 0 family ethernet-switching interface-mode trunk
Assign the LAG to a VLAN:
[edit interfaces] user@switch# set ae0 unit 0 family ethernet-switching vlan members green vlan-id 200
(Optional): Designate one side of the LAG as active for LACP:
[edit interfaces] user@switch# set ae0 aggregated-ether-options lacp active
(Optional): Designate the interval and speed at which the interfaces send LACP packets:
[edit interfaces] user@switch# set ae0 aggregated-ether-options lacp periodic fast
Results
Display the results of the configuration on a QFX3500 or QFX3600 switch:
[edit]
chassis {
aggregated-devices {
ethernet {
device-count 1;
}
}
}
green {
vlan-id 200;
}
}
interfaces {
ae0 {
aggregated-ether-options {
link-speed 10g;
minimum-links 1;
}
unit 0 {
family ethernet-switching {
port-mode trunk;
vlan {
members green;
}
}
}
xe-0/0/2 {
ether-options {
802.3ad ae0;
}
}
xe-0/0/3 {
ether-options {
802.3ad ae0;
}
}
}
Verification
To verify that switching is operational and one LAG has been created, perform these tasks:
Verify That LAG ae0.0 Has Been Created
Purpose
Verify that LAG ae0.0
has been created on the switch.
Action
show interfaces ae0 terse
Interface Admin Link Proto Local Remote ae0 up up ae0.0 up up eth-switch
Meaning
The output confirms that the ae0.0
link is up and shows the family
and IP address assigned to this link.
Troubleshooting
Troubleshooting a LAG That Is Down
Problem
The show interfaces terse command shows
that the LAG is down.
Solution
Check the following:
Verify that there is no configuration mismatch.
Verify that all member ports are up.
Verify that a LAG is part of family ethernet switching (Layer 2 LAG) or family inet (Layer 3 LAG).
Verify that the LAG member is connected to the correct LAG at the other end.
Additional Platform Information
Use Link Aggregation Group to confirm platform and release support for specific features. Additional platforms might be supported.
| LAG Information | EX4100-F Virtual Chassis | EX4200 and EX4200 Virtual Chassis | EX4300 and EX4300 Virtual Chassis | EX4600 | EX4650 Virtual Chassis | EX9200 |
|---|---|---|---|---|---|---|
| EX6200 | EX4400 | |||||
| Maximum Interfaces per LAG | 8 | 8 | 16 | 32 | 64 | 64 |
| Maximum LAGs per Switch | 128 | 111 | 128 | 128 | 72 | 150 |
| LAG Information | QFX5100 | QFX5120 | QFX5130 | QFX5700 | QFX10002 | QFX10008 | |
|---|---|---|---|---|---|---|---|
| QFX5110 | QFX5200 | ||||||
| Maximum Interfaces per LAG | 64 | 64 | 64 | 128 | 64 | 64 | 64 |
| Maximum LAGs per Switch | 96 | 72 | 128 | 144 | 150 | 1000 | 1000 |