Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Redundancy and Multihoming Using MC-LAG

 

Multichassis link aggregation (MC-LAG) enables an MX Series 5G Universal Routing Platform to form a logical LAG interface with two or more other devices. MC-LAG provides additional benefits over traditional LAG in terms of node level redundancy, multihoming support, and a loop-free Layer 2 network without the need to run Spanning Tree Protocol (STP). MC-LAG can be configured for virtual private LAN service (VPLS) routing instances, circuit cross-connect (CCC) applications, and Layer 2 circuit encapsulation types.

The MC-LAG devices use Inter-Chassis Control Protocol (ICCP) to exchange the control information between two MC-LAG network devices.

On one end of the MC-LAG is an MC-LAG client device that has one or more physical links in a link aggregation group (LAG). This client device does not need to be aware of the MC-LAG configuration. On the other side of the MC-LAG are two MC-LAG network devices. Each of these network devices has one or more physical links connected to a single client device. The network devices coordinate with each other to ensure that data traffic is forwarded properly.

MC-LAG includes the following functionality:

  • Only single-active MC-LAG mode with multi-homed VPLS instance is supported.

  • MC-LAG operates only between two devices.

  • Layer 2 circuit functions are supported with ether-ccc and vlan-ccc encapsulations.

  • VPLS functions are supported with ether-vpls and vlan-vpls encapsulations.

Note

Ethernet connectivity fault management (CFM) specified in the IEEE 802.1ag standard for Operation, Administration, and Management (OAM) is not supported on MC-LAG interfaces.

To enable MC-LAG, include the mc-ae statement at the [edit interfaces aeX aggregated-ether-options] hierarchy level along with one of the following statements at the [edit interfaces aeX] hierarchy level: encapsulation-ethernet-bridge, encapsulation ethernet-ccc, encapsulation ethernet-vpls, or encapsulation-flexible-ethernet-services. You also need to configure the lacp, admin-key, and system-id statements at the [edit interfaces aeX aggregated-ether-options] hierarchy level:

Note

When you configure the prefer-status-control-active statement, you must also configure the status-control active statement. If you configure the status-control standby statement with the prefer-status-control-active statement, the system issues a warning.

To delete an MC-LAG interface from the configuration, issue the delete interfaces aeX aggregated-ether-options mc-ae command at the [edit] hierarchy level in configuration mode:

Perform the following steps on each switch that is hosting an MC-LAG:

  1. Specify the same multichassis aggregated Ethernet identification number for the MC-LAG that the aggregated Ethernet interface belongs to on each switch.
    [edit interfaces]

    user@host# set aeX aggregated-ether-options mc-ae mc-ae-id mc-ae-id

    For example:

    [edit interfaces]

    user@host# set ae1 aggregated-ether-options mc-ae mc-ae-id 3
  2. Specify a unique chassis ID for the MC-LAG that the aggregated Ethernet interface belongs to on each switch.
    [edit interfaces]

    user@host# set aeX aggregated-ether-options chassis-id chassis-id

    For example:

    [edit interfaces]

    user@host# set ae1 aggregated-ether-options mc-ae chassis-id 0
  3. Specify the mode of the MC-LAG that the aggregated Ethernet interface belongs to.Note

    Only active/active mode is supported for Reverse Layer 2 Gateway Protocol (R-L2GP) at this time.

    [edit interfaces]

    user@host# set aeX aggregated-ether-options mc-ae mode mode

    For example:

    [edit interfaces]

    user@host# set ae1 aggregated-ether-options mc-ae mode active-active
  4. Specify whether the aggregated Ethernet interface participating in the MC-LAG is primary or secondary. Primary is active, and secondary is standby.Note

    You must configure status control on both switches hosting the MC-LAG. If one switch is in active mode, the other must be in standby mode.

    [edit interfaces]

    user@host# set aeX aggregated-ether-options mc-ae status-control (active | standby)

    For example:

    [edit interfaces]

    user@host# set aeX aggregated-ether-options mc-ae status-control (active | standby)
  5. Configure the MC-LAG interface to improve Layer 2 and Layer 3 convergence time when a multichassis aggregated Ethernet link goes down or comes up in a bridge domain.
    [edit interfaces]

    user@host# set aeX aggregated-ether-options mc-ae enhanced-convergence
  6. Specify the same LACP system ID on each switch.
    [edit interfaces]

    user@host# set aeX aggregated-ether-options lacp system-id mac-address

    For example:

    [edit interfaces]

    user@host# set ae1 aggregated-ether-options lacp system-id 00:01:02:03:04:05
  7. Specify the same LACP administration key on each switch.
    [edit interfaces]

    user@host# set aeX aggregated-ether-options lacp admin-key number

    For example:

    [edit interfaces]

    user@host# set ae1 aggregated-ether-options lacp admin-key 3
  8. Configure ICCP by doing the following on each switch hosting the MC-LAG:
    1. Configure the local IP address to be used by all switches hosting the MC-LAG.
      [edit protocols]

      user@host# set iccp local-ip-addr local-ip-address

      For example:

      [edit protocols]

      user@host# set iccp local-ip-addr 10.3.3.1
    2. (Optional) Configure the IP address of the router and the time during which an ICCP connection must succeed between the routers hosting the MC-LAG.Note

      On QFX and EX Series switches, the default session establishment hold time is 300 seconds. However, the session establishment time must be at least 100 seconds higher than the init delay time. You can optionally update the session establishment time to be 340 seconds and the init delay time to be 240 seconds.

      [edit protocols]

      user@host# set iccp peer peer-ip-address session-establishment-hold-time seconds

      For example:

      [edit protocols]

      user@host# set iccp peer 10.3.3.2 session-establishment-hold-time 340
    3. (Optional) Configure the IP address to be used for backup liveness detection:Note

      By default, backup liveness detection is not enabled. Configure backup liveness detection if you require faster failover of data traffic loss during an MC-LAG peer reboot. Backup liveness detection helps achieve subsecond traffic loss during an MC-LAG peer reboot.

      [edit protocols]

      user@host# set iccp peer peer-ip-address backup-liveness-detection backup-peer-ip ip-address

      For example:

      [edit protocols]

      user@host# set iccp peer 10.3.3.2 backup-liveness-detection backup-peer-ip 10.207.64.232
    4. Configure the minimum interval at which the router must receive a reply from the other router with which it has established a Bidirectional Forwarding Detection (BFD) session.Note

      Configuring the minimum receive interval is required to enable BFD.

      [edit protocols]

      user@host# set iccp peer peer-ip-address liveness-detection minimum-receive-interval milliseconds

      For example:

      [edit protocols]

      user@host# set iccp peer 10.3.3.2 liveness-detection minimum-receive-interval 60
    5. Configure the minimum transmit interval during which a router must receive a reply from a router with which it has established a BFD session.
      [edit protocols]

      user@host# set iccp peer peer-ip-address liveness-detection transmit-interval minimum-interval milliseconds

      For example:

      [edit protocols]

      user@host# set iccp peer 10.3.3.2 liveness-detection transmit-interval minimum-interval 60
    6. Specify the switch service ID.

      The switch service ID is used to synchronize applications, IGMP, ARP, and MAC learning across MC-LAG members.

      [edit switch-options]
      user@host# set service-id number

      For example:

      [edit switch-options]
      user@host# set service-id 1
  9. Configure a multichassis protection link between the routers.
    [edit]

    user@host# set multi-chassis multi-chassis-protection peer-ip-address interface interface-name

    For example:

    [edit]

    user@host# set multi-chassis multi-chassis-protection 10.3.3.1 interface ae0
  10. Enable RSTP globally on all interfaces.
    [edit]

    user@host# set protocols rstp interface all mode point-to-point
  11. Disable RSTP on the interchassis control link protection link (ICL-PL) interfaces on both routers.
    [edit]

    user@host# set protocols rstp interface interface-name disable

    For example:

    [edit]

    user@host# set protocols rstp interface ae0.0 disable
  12. Configure the MC-LAG interfaces as edge ports on both routers.
    user@host# set protocols rstp interface interface-name edge

    For example:

    [edit]

    user@host# set protocols rstp interface ae1 edge
  13. Enable BPDU block on all interfaces except for the ICL-PL interfaces on both routers.
    [edit]

    user@host# set protocols rstp bpdu-block-on-edge

    For example:

    [edit]

    user@host# set protocols rstp bpdu-block-on-edge

Multichassis link aggregation groups (MC-LAGs) enable a client device to form a logical LAG interface between two MC-LAG peers (for example, EX9200 switches). An MC-LAG provides redundancy and load balancing between the two MC-LAG peers, multihoming support, and a loop-free Layer 2 network without running Spanning Tree Protocol (STP).

On one end of an MC-LAG, there is an MC-LAG client device, such as a server, that has one or more physical links in a link aggregation group (LAG). This client device does not need to have an MC-LAG configured. On the other side of MC-LAG, there are two MC-LAG peers. Each of the MC-LAG peers has one or more physical links connected to a single client device.

The MC-LAG peers use Inter-Chassis Control Protocol (ICCP) to exchange control information and coordinate with each other to ensure that data traffic is forwarded properly.

Note

An interface with an already configured IP address cannot form part of the aggregated Ethernet interface or multichassis aggregated Ethernet interface group.

Perform the following steps on each switch that hosts an MC-LAG:

  1. Specify the same multichassis aggregated Ethernet identification number for the MC-LAG that the aggregated Ethernet interface belongs to on each switch.
    [edit interfaces]

    user@switch# set aex aggregated-ether-options mc-ae mc-ae-id number

    For example:

    [edit interfaces]

    user@switch# set ae1 aggregated-ether-options mc-ae mc-ae-id 3
  2. Specify a unique chassis ID for the MC-LAG that the aggregated Ethernet interface belongs to on each switch.
    [edit interfaces]

    user@switch# set aex aggregated-ether-options mc-ae chassis-id number

    For example:

    [edit interfaces]

    user@switch# set ae1 aggregated-ether-options mc-ae chassis-id 0
  3. Specify the mode of the MC-LAG the aggregated Ethernet interface belongs to.
    [edit interfaces]

    user@switch# set aex aggregated-ether-options mc-ae mode mode

    For example:

    [edit interfaces]

    user@switch# set ae1 aggregated-ether-options mc-ae mode active-active
  4. Specify whether the aggregated Ethernet interface participating in the MC-LAG is primary or secondary.

    Primary is active, and secondary is standby.

    Note

    You must configure status control on both switches that host the MC-LAG. If one switch is in active mode, the other must be in standby mode.

    [edit interfaces]

    user@switch# set aex aggregated-ether-options mc-ae status-control (active | standby)

    For example:

    [edit interfaces]

    user@switch# set ae1 aggregated-ether-options mc-ae status-control active
    Note

    If you configure both nodes as prefer-status-control-active, you must also configure ICCP peering using the peer’s loopback address to make sure that the ICCP session does not go down because of physical link failures. Additionally, you must configure backup liveness detection on both of the MC-LAG nodes.

    Note

    On EX9200 switches, the prefer-status-control-active statement was added in Junos OS Release 13.2R1.

  5. Specify the init delay time.

    The init delay time specifies the number of seconds by which to delay bringing up the MC-LAG interface back to the up state when the MC-LAG peer is rebooted. By delaying the bring-up of the interface until after the protocol convergence, you can prevent packet loss during the recovery of failed links and devices.

    Note

    On QFX and EX Series switches, the default session establishment hold time is 300 seconds. However, the session establishment time must be at least 100 seconds higher than the init delay time. You can optionally update the session establishment time to be 340 seconds and the init delay time to be 240 seconds.

    [edit interfaces]

    user@switch# set aex aggregated-ether-options mc-ae init-delay-time seconds

    For example:

    [edit interfaces]

    user@switch# set ae0 aggregated-ether-options mc-ae init-delay-time 240
  6. Specify the same LACP system ID on each switch.
    [edit interfaces]

    user@switch# set aex aggregated-ether-options lacp system-id mac-address

    For example:

    [edit interfaces]

    user@switch# set ae1 aggregated-ether-options lacp system-id 00:01:02:03:04:05
  7. Specify the same LACP administration key on each switch.
    [edit interfaces]

    user@switch# set aex aggregated-ether-options lacp admin-key number

    For example:

    [edit interfaces]

    user@switch# set ae1 aggregated-ether-options lacp admin-key 3
  8. Configure ICCP by performing the following steps on each switch that hosts the MC-LAG:
    1. Configure the local IP address to be used by the switches that host the MC-LAG.
      [edit protocols]

      user@switch# set iccp local-ip-addr local-ip-address

      For example:

      [edit protocols]

      user@switch# set iccp local-ip-addr 10.3.3.1
    2. (Optional) Configure the IP address of the switch and the time during which an ICCP connection must be established between the switches that host the MC-LAG.Note

      On QFX and EX Series switches, the default session establishment hold time is 300 seconds. However, the session establishment time must be at least 100 seconds higher than the init delay time. You can optionally update the session establishment time to be 340 seconds and the init delay time to be 240 seconds.

      [edit protocols]

      user@switch# set iccp peer peer-ip-address session-establishment-hold-time seconds

      For example:

      [edit protocols]

      user@switch# set iccp peer 10.3.3.2 session-establishment-hold-time 340
    3. (Optional) Configure the backup-liveness-detection statement on the management interface (fxp0) only.

      We recommend that you configure the backup liveness detection feature to implement faster failover of data traffic during an MC-LAG peer reboot.

      Note

      On EX9200 switches, the backup-liveness-detection statement was added in Junos OS Release 13.2R1.

      Note

      By default, backup liveness detection is not enabled. Configure backup liveness detection if you require minimal traffic loss during a reboot. Backup liveness detection helps achieve sub-second traffic loss during an MC-LAG reboot.

      [edit protocols]

      user@switch# set iccp peer peer-ip-address backup-liveness-detection backup-peer-ip ip-address

      For example:

      [edit protocols]

      user@switch# set iccp peer 10.3.3.2 backup-liveness-detection backup-peer-ip 10.207.64.232
    4. Configure the minimum interval at which the switch must receive a reply from the other switch with which it has established a Bidirectional Forwarding Detection (BFD) session.Note

      Configuring the minimum receive interval is required to enable BFD. We recommend a minimum receive interval value of 1000 seconds.

      [edit protocols]

      user@switch# set iccp peer peer-ip-address liveness-detection minimum-receive-interval milliseconds

      For example:

      [edit protocols]

      user@switch# set iccp peer 10.3.3.2 liveness-detection minimum-receive-interval 1000
    5. Configure the minimum transmit interval during which a switch must receive a reply from a switch with which it has established a BFD session.
      [edit protocols]

      user@switch# set iccp peer peer-ip-address liveness-detection transmit-interval minimum-interval milliseconds

      For example:

      [edit protocols]

      user@switch# set iccp peer 10.3.3.2 liveness-detection transmit-interval minimum-interval 1000
  9. Specify the switch service ID.

    The switch service ID is used to synchronize applications, IGMP, ARP, and MAC learning across MC-LAG members.

    [edit switch-options]
    user@switch# set service-id number

    For example:

    [edit switch-options]
    user@switch# set service-id 1
  10. Configure a multichassis protection link between the switches.
    [edit multi-chassis]

    user@switch# set multi-chassis-protection peer-ip-address interface interface-name

    For example:

    [edit multi-chassis]

    user@switch# set multi-chassis-protection 10.3.3.1 interface ae0
Note

Multichassis link aggregation (MC-LAG) is supported on QFX3500 and QFX3600 standalone switches running the original CLI, and on QFX5100, QFX5200, EX4600, QFX10002, QFX10008, and QFX10016 switches running Enhanced Layer 2 Software.

Multichassis link aggregation groups (MC-LAGs) enable a client device to form a logical LAG interface between two switches. An MC-LAG provides redundancy and load balancing between the two switches, multihoming support, and a loop-free Layer 2 network without running Spanning Tree Protocol (STP).

The MC-LAG switches use Inter-Chassis Control Protocol (ICCP) to exchange the control information between two MC-LAG switches.

Note

The ICCP link should be physically separate (out of band) from the data plane traffic.

On one end of an MC-LAG is an MC-LAG client device, such as a server, that has one or more physical links in a link aggregation group (LAG). This client device does not need to detect the MC-LAG. On the other side of MC-LAG are two MC-LAG switches. Each of the switches has one or more physical links connected to a single client device. The switches coordinate with each other to ensure that data traffic is forwarded properly.

Note

An interface with an already configured IP address cannot form part of the aggregated Ethernet interface or multichassis aggregated Ethernet interface group.

Perform the following steps on each switch that is hosting an MC-LAG:

  1. Specify the same multichassis aggregated Ethernet identification number for the MC-LAG that the aggregated Ethernet interface belongs to on each switch.
    [edit interfaces]

    user@switch# set aeX aggregated-ether-options mc-ae mc-ae-id number

    For example:

    [edit interfaces]

    user@switch# set ae1 aggregated-ether-options mc-ae mc-ae-id 3
  2. Specify a unique chassis ID for the MC-LAG that the aggregated Ethernet interface belongs to on each switch.
    [edit interfaces]

    user@switch# set aeX aggregated-ether-options mc-ae chassis-id number

    For example:

    [edit interfaces]

    user@switch# set ae1 aggregated-ether-options mc-ae chassis-id 0
  3. Specify the mode of the MC-LAG the aggregated Ethernet interface belongs to.
    [edit interfaces]

    user@switch# set aeX aggregated-ether-options mc-ae mode mode

    For example:

    [edit interfaces]

    user@switch# set ae1 aggregated-ether-options mc-ae mode active-active
    Note

    By default, backup liveness detection is not enabled. Configure backup liveness detection if you require minimal traffic loss during a reboot. Backup liveness detection helps achieve sub-second traffic loss during an MC-LAG reboot.

  4. Specify whether the aggregated Ethernet interface participating in the MC-LAG is primary or secondary.

    Primary is active, and secondary is standby.

    Note

    You must configure status control on both switches hosting the MC-LAG. If one switch is in active mode, the other must be in standby mode.

    [edit interfaces]

    user@switch# set aeX aggregated-ether-options mc-ae status-control (active | standby)

    For example:

    [edit interfaces]

    user@switch# set ae1 aggregated-ether-options mc-ae status-control active
    Note

    If you configure both nodes as prefer-status-control-active, you must also configure ICCP peering using the peer’s loopback address to make sure that the ICCP session does not go down because of physical link failures. Additionally, you must configure backup liveness detection on both of the MC-LAG nodes.

  5. Specify the init delay time.

    The init delay time specifies the number of seconds by which to delay bringing up the MC-LAG interface back to the up state when the MC-LAG peer is rebooted. By delaying the bring-up of the interface until after the protocol convergence, you can prevent packet loss during the recovery of failed links and devices.

    Note

    On QFX and EX Series switches, the default session establishment hold time is 300 seconds. However, the session establishment time must be at least 100 seconds higher than the init delay time. You can optionally update the session establishment time to be 340 seconds and the init delay time to be 240 seconds.

    [edit interfaces]

    user@switch# set aex aggregated-ether-options mc-ae init-delay-time seconds

    For example:

    [edit interfaces]

    user@switch# set ae0 aggregated-ether-options mc-ae init-delay-time 240
  6. Specify the same LACP system ID on each switch.
    [edit interfaces]

    user@switch# set aeX aggregated-ether-options lacp system-id mac-address

    For example:

    [edit interfaces]

    user@switch# set ae1 aggregated-ether-options lacp system-id 00:01:02:03:04:05
  7. Specify the same LACP administration key on each switch.
    [edit interfaces]

    user@switch# set aeX aggregated-ether-options lacp admin-key number

    For example:

    [edit interfaces]

    user@switch# set ae1 aggregated-ether-options lacp admin-key 3
  8. Configure ICCP by doing the following on each switch hosting the MC-LAG:Note

    The ICCP link should be physically separate (out of band) from the data plane traffic.

    1. Configure the local IP address to be used by all switches hosting the MC-LAG.
      [edit protocols]

      user@switch# set iccp local-ip-addr local-ip-address

      For example:

      [edit protocols]

      user@switch# set iccp local-ip-addr 10.3.3.1
    2. (Optional) Configure the IP address of the switch and the time during which an ICCP connection must succeed between the switches hosting the MC-LAG.Note

      On QFX and EX Series switches, the default session establishment hold time is 300 seconds. However, the session establishment time must be at least 100 seconds higher than the init delay time. You can optionally update the session establishment time to be 340 seconds and the init delay time to be 240 seconds.

      [edit protocols]

      user@switch# set iccp peer peer-ip-address session-establishment-hold-time seconds

      For example:

      [edit protocols]

      user@switch# set iccp peer 10.3.3.2 session-establishment-hold-time 340
    3. (Optional) Configure the backup-liveness-detection statement on the management interface (fxp0) only.Note

      By default, backup liveness detection is not enabled. Configure backup liveness detection if you require minimal traffic loss during a reboot. Backup liveness detection helps achieve sub-second traffic loss during an MC-LAG reboot.

      [edit protocols]

      user@switch# set iccp peer peer-ip-address backup-liveness-detection backup-peer-ip ip-address

      For example:

      [edit protocols]

      user@switch# set iccp peer 10.3.3.2 backup-liveness-detection backup-peer-ip 10.207.64.232
    4. Configure the minimum interval at which the switch must receive a reply from the other switch with which it has established a Bidirectional Forwarding Detection (BFD) session.Note

      Configuring the minimum receive interval is required to enable BFD.

      [edit protocols]

      user@switch# set iccp peer peer-ip-address liveness-detection minimum-receive-interval milliseconds

      For example:

      [edit protocols]

      user@switch# set iccp peer 10.3.3.2 liveness-detection minimum-receive-interval 1000
    5. Configure the minimum transmit interval during which a switch must receive a reply from a switch with which it has established a BFD session.
      [edit protocols]

      user@switch# set iccp peer peer-ip-address liveness-detection transmit-interval minimum-interval milliseconds

      For example:

      [edit protocols]

      user@switch# set iccp peer 10.3.3.2 liveness-detection transmit-interval minimum-interval 1000
  9. Specify the switch service ID.

    The switch service ID is used to synchronize applications, IGMP, ARP, and MAC learning across MC-LAG members.

    [edit switch-options]
    user@switch# set service-id number

    For example:

    [edit switch-options]
    user@switch# set service-id 1
  10. Configure a multichassis protection link between the switches.
    [edit]

    user@switch# set multi-chassis multi-chassis-protection peer-ip-address interface interface-name

    For example:

    [edit]

    user@switch# set multi-chassis multi-chassis-protection 10.3.3.1 interface ae0
  11. If you are using ELS, configure the service-id on both switches.

    The service-id must be the same number on both switches.

    [edit switch-options]

    user@switch# set service-id number

    For example:

    [edit switch-options]

    user@switch# set service-id 10
  12. Configure the MC-LAG interfaces as edge ports on both switches.
    [edit protocols rstp]
    user@switch# set interface interface-name edge

    For example:

    [edit protocols rstp]

    user@switch# set interface ae1 edge
  13. Enable BPDU block on all interfaces except for the ICL-PL interfaces on both switches.
    [edit protocols rstp]

    user@switch# set bpdu-block-on-edge

In an MC-LAG network, an MC-LAG client link without Link Access Control Protocol (LACP) configuration remains down and cannot be accessed by the MC-LAG switches.

To ensure that the client device with limited LACP capability is up and accessible on the MC-LAG network, configure one of the aggregated Ethernet links or interfaces on a MC-LAG switch to be up by using the force-up statement at the appropriate hierarchy level on your device:

  • [edit interfaces interface-name aggregated-ether-options lacp]

  • [edit interfaces interface-name ether-options 802.3ad lacp]

You can configure the force-up feature on the MC-LAG switches in either active mode or standby mode. However, in order to prevent duplicate traffic and packet drops, you configure the force-up feature only on one aggregated Ethernet link of the MC-LAG switches . If multiple aggregated Ethernet links are up on the MC-LAG switches with force-up feature configured, then the device selects the link based on the LACP port ID and port priority. The port with the lowest priority is given preference. In case of two ports with the same priority, the one with the lowest port ID is given preference.

Note

The force-up option is not supported on QFX10002 switches.

Note

On the QFX5100 switch, you can configure the force-up feature in Link Aggregation Control Protocol (LACP) on the MC-LAG switches starting with Junos OS Release 14.1X53-D10.

Note
  • If LACP comes up partially in the MC-LAG network—that is, it comes up on one of the MC-LAG switches and does not comes up on other MC-LAG switches—the force-up feature is disabled.

Note

Multichassis link aggregation (MC-LAG) is supported on QFX3500 and QFX3600 standalone switches running the original CLI and QFX5100 , EX4600, and QFX10002 standalone switches running Enhanced Layer 2 Software. (This example has not been tested on all devices that support MC-LAG. See Feature Explorer for a full listing of devices that support MC-LAG.

This example shows how multichassis link aggregation groups (MC-LAGs) enable a client device to form a logical LAG interface between two switches to provide redundancy and load balancing between the two switches, multihoming support, and a loop-free Layer 2 network without running Spanning Tree Protocol (STP).

Requirements

This example uses the following hardware and software components:

  • Junos OS Release 12.2 or later for the QFX3500 and QFX3600 standalone switches, Junos OS Release 13.2X51-D10 or later for the QFX5100 standalone switches, Junos OS Release 13.2X51-D25 or later for EX4600 switches, or Junos OS Release 15.1X53-D10 or later for QFX10002 standalone switches.

  • Two QFX3500 or QFX3600 standalone switches, two QFX5100 standalone switches, two EX4600 switches, or two QFX10002 standalone switches.

Before you configure an MC-LAG, be sure that you understand how to:

Overview

In this example, you configure an MC-LAG across two switches, consisting of two aggregated Ethernet interfaces, an interchassis control link-protection link (ICL-PL), multichassis protection link for the ICL-PL, the Inter-Chassis Control Protocol for the peers hosting the MC-LAG, and Layer 3 connectivity between MC-LAG peers. Layer 3 connectivity is required for ICCP.

Topology

The topology used in this example consists of two switches hosting an MC-LAG. The two switches are connected to a server. Figure 1 shows the topology used in this example.

Figure 1: Configuring a Multichassis LAG Between Switch A and Switch B
Configuring a Multichassis LAG Between
Switch A and Switch B

Table 1 details the topology used in this configuration example.

Table 1: Components of the Topology for Configuring a Multichassis LAG Between Two Switches

HostnameBase HardwareMultichassis Link Aggregation Group

Switch A

Switch B

QFX3500 or QFX3600 standalone switch, or QFX5100 standalone switch

QFX3500 or QFX3600 standalone switch, or QFX5100 standalone switch

QFX3500, QFX3600, EX4600, QFX5100, or QFX10002 standalone switch

ae0 is configured as an aggregated Ethernet interface, and is used as an ICL-PL. The following interfaces are part of ae0: xe-0/0/12 and xe-0/0/13 Switch A and

xe-0/0/12 and xe-0/0/13 on Switch B.

ae1 is configured as an MC-LAG, and the following two interfaces are part of ae1:

xe-0/0/44 on Switch A and

xe-0/0/46 on Switch B.

.

Configuration

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them in a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

Note

This example shows how to configure MC-LAG using both the original CLI and Enhanced Layer 2 Software (ELS).

In ELS, there are three statements and one additional statement that are different from the original CLI:

  • The port-mode statement in the [edit interfaces interface-name unit number family ethernet-switching] hierarchy is not supported. Use the interface-mode statement instead.

  • The vlan statement in the [edit interfaces interface-name] hierarchy is not supported. Use the irb statement instead.

  • The vlan.logical-interface-number option in the [edit vlans vlan-name l3-interface] hierarchy is not supported. Use the irb.logical-interface-number option instead.

  • The service-id statement in the [edit switch-options] hierarchy is required in the ELS CLI.

Switch A—Original CLI

set chassis aggregated-devices ethernet device-count 2
set interfaces xe-0/0/12 ether-options 802.3ad ae0
set interfaces xe-0/0/13 ether-options 802.3ad ae0
set interfaces xe-0/0/44 ether-options 802.3ad ae1
set interfaces ae0 unit 0 family ethernet-switching port-mode trunk
set interfaces ae0 unit 0 family ethernet-switching vlan members v500
set interfaces ae0 unit 0 family ethernet-switching vlan members v100
set interfaces ae1 aggregated-ether-options lacp active
set interfaces ae1 aggregated-ether-options lacp system-id 00:01:02:03:04:05
set interfaces ae1 aggregated-ether-options lacp admin-key 3
set interfaces ae1 aggregated-ether-options mc-ae mc-ae-id 3
set interfaces ae1 aggregated-ether-options mc-ae chassis-id 0
set interfaces ae1 aggregated-ether-options mc-ae mode active-active
set interfaces ae1 aggregated-ether-options mc-ae status-control active
set interfaces ae1 aggregated-ether-options mc-ae init-delay-time 240
set interfaces ae1 unit 0 family ethernet-switching port-mode trunk
set interfaces ae1 unit 0 family ethernet-switching vlan members v100
set interfaces ae1 unit 0 family ethernet-switching vlan members v500
set interfaces vlan unit 500 family inet address 10.3.3.2/8
set vlans v100 vlan-id 100
set vlans v500 vlan-id 500
set vlans v100 l3-interface vlan.100
set vlans v500 l3-interface vlan.500
set protocols iccp local-ip-addr 10.3.3.2
set protocols iccp peer 10.3.3.1 session-establishment-hold-time 340
set protocols iccp peer 10.3.3.1 backup-liveness-detection backup-peer-ip 10.207.64.233
set protocols iccp peer 10.3.3.1 liveness-detection minimum-receive-interval 1000
set protocols iccp peer 10.3.3.1 liveness-detection transmit-interval minimum-interval 1000
set protocol rstp system-identifier 00:01:02:03:04:05
set protocols rstp interface ae0 disable
set protocols rstp interface ae1 edge
set protocols rstp interface all mode point-to-point
set protocols rstp bpdu-block-on-edge
set multi-chassis multi-chassis-protection 10.3.31 interface ae0
set switch-options service-id 10

Switch A—ELS

set chassis aggregated-devices ethernet device-count 2
set interfaces xe-0/0/12 ether-options 802.3ad ae0
set interfaces xe-0/0/13 ether-options 802.3ad ae0
set interfaces xe-0/0/44 ether-options 802.3ad ae1
set interfaces ae0 unit 0 family ethernet-switching interface-mode trunk
set interfaces ae0 unit 0 family ethernet-switching vlan members v500
set interfaces ae0 unit 0 family ethernet-switching vlan members v100
set interfaces ae1 aggregated-ether-options lacp active
set interfaces ae1 aggregated-ether-options lacp system-id 00:01:02:03:04:05
set interfaces ae1 aggregated-ether-options lacp admin-key 3
set interfaces ae1 aggregated-ether-options mc-ae mc-ae-id 3
set interfaces ae1 aggregated-ether-options mc-ae chassis-id 0
set interfaces ae1 aggregated-ether-options mc-ae mode active-active
set interfaces ae1 aggregated-ether-options mc-ae status-control active
set interfaces ae1 aggregated-ether-options mc-ae init-delay-time 240
set interfaces ae1 unit 0 family ethernet-switching interface-mode trunk
set interfaces ae1 unit 0 family ethernet-switching vlan members v100
set interfaces ae1 unit 0 family ethernet-switching vlan members v500
set interfaces irb unit 500 family inet address 10.3.3.2/8
set vlans v100 vlan-id 100
set vlans v500 vlan-id 500
set vlans v100 l3-interface irb.100
set vlans v500 l3-interface irb.500
set protocols iccp local-ip-addr 10.3.3.2
set protocols iccp peer 10.3.3.1 session-establishment-hold-time 340
set protocols iccp peer 10.3.3.1 backup-liveness-detection backup-peer-ip 10.207.64.233
set protocols iccp peer 10.3.3.1 liveness-detection minimum-receive-interval 1000
set protocols iccp peer 10.3.3.1 liveness-detection transmit-interval minimum-interval 1000
set protocol rstp system-identifier 00:01:02:03:04:05
set protocols rstp interface ae1 edge
set protocols rstp interface ae1 mode point-to-point
set protocols rstp bpdu-block-on-edge
set multi-chassis multi-chassis-protection 10.3.3.1 interface ae0
set switch-options service-id 10

Switch B—Original CLI

set chassis aggregated-devices ethernet device-count 2
set interfaces xe-0/0/12 ether-options 802.3ad ae0
set interfaces xe-0/0/13 ether-options 802.3ad ae0
set interfaces xe-0/0/46 ether-options 802.3ad ae1
set interfaces ae0 unit 0 family ethernet-switching port-mode trunk
set interfaces ae0 unit 0 family ethernet-switching vlan members v500
set interfaces ae0 unit 0 family ethernet-switching vlan members v100
set interfaces ae1 aggregated-ether-options lacp active
set interfaces ae1 aggregated-ether-options lacp system-id 00:01:02:03:04:05
set interfaces ae1 aggregated-ether-options lacp admin-key 3
set interfaces ae1 aggregated-ether-options mc-ae mc-ae-id 3
set interfaces ae1 aggregated-ether-options mc-ae chassis-id 1
set interfaces ae1 aggregated-ether-options mc-ae mode active-active
set interfaces ae1 aggregated-ether-options mc-ae status-control standby
set interfaces ae1 aggregated-ether-options mc-ae init-delay-time 240
set interfaces ae1 unit 0 family ethernet-switching port-mode trunk
set interfaces ae1 unit 0 family ethernet-switching vlan members v100
set interfaces ae1 unit 0 family ethernet-switching vlan members v500
set interfaces vlan unit 500 family inet address 10.3.3.1/8
set vlans v100 vlan-id 100
set vlans v500 vlan-id 500
set vlans v100 l3-interface vlan.100
set vlans v500 l3-interface vlan.500
set protocols iccp local-ip-addr 10.3.3.1
set protocols iccp peer 10.3.3.2 session-establishment-hold-time 340
set protocols iccp peer 10.3.3.2 backup-liveness-detection backup-peer-ip 10.207.64.234
set protocols iccp peer 10.3.3.2 liveness-detection minimum-receive-interval 1000
set protocols iccp peer 10.3.3.2 liveness-detection transmit-interval minimum-interval 1000
set protocol rstp system-identifier 00:01:02:03:04:05
set protocols rstp interface ae0 disable
set protocols rstp interface ae1 edge
set protocols rstp interface all mode point-to-point
set protocols rstp bpdu-block-on-edge
set multi-chassis multi-chassis-protection 10.3.3.2 interface ae0
set switch-options service-id 10

Switch B—ELS

set chassis aggregated-devices ethernet device-count 2
set interfaces xe-0/0/12 ether-options 802.3ad ae0
set interfaces xe-0/0/13 ether-options 802.3ad ae0
set interfaces xe-0/0/46 ether-options 802.3ad ae1
set interfaces ae0 unit 0 family ethernet-switching interface-mode trunk
set interfaces ae0 unit 0 family ethernet-switching vlan members v500
set interfaces ae0 unit 0 family ethernet-switching vlan members v100
set interfaces ae1 aggregated-ether-options lacp active
set interfaces ae1 aggregated-ether-options lacp system-id 00:01:02:03:04:05
set interfaces ae1 aggregated-ether-options lacp admin-key 3
set interfaces ae1 aggregated-ether-options mc-ae mc-ae-id 3
set interfaces ae1 aggregated-ether-options mc-ae chassis-id 1
set interfaces ae1 aggregated-ether-options mc-ae mode active-active
set interfaces ae1 aggregated-ether-options mc-ae status-control standby
set interfaces ae1 aggregated-ether-options mc-ae init-delay-time 240
set interfaces ae1 unit 0 family ethernet-switching interface-mode trunk
set interfaces ae1 unit 0 family ethernet-switching vlan members v100
set interfaces ae1 unit 0 family ethernet-switching vlan members v500
set interfaces irb unit 500 family inet address 10.3.3.1/8
set vlans v100 vlan-id 100
set vlans v500 vlan-id 500
set vlans v100 l3-interface irb.100
set vlans v500 l3-interface irb.500
set protocols iccp local-ip-addr 10.3.3.1
set protocols iccp peer 10.3.3.2 session-establishment-hold-time 340
set protocols iccp peer 10.3.3.2 backup-liveness-detection backup-peer-ip 10.207.64.234
set protocols iccp peer 10.3.3.2 liveness-detection minimum-receive-interval 1000
set protocols iccp peer 10.3.3.2 liveness-detection transmit-interval minimum-interval 1000
set protocol rstp system-identifier 00:01:02:03:04:05
set protocols rstp interface ae1 edge
set protocols rstp interface ae1 mode point-to-point
set protocols rstp bpdu-block-on-edge
set multi-chassis multi-chassis-protection 10.3.3.2 interface ae0
set switch-options service-id 10

Configuring MC-LAG on Two Switches

Step-by-Step Procedure

The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode.

To enable multichassis protection link between MC-LAG peers:

  1. Configure the number of LAGs on both Switch A and Switch B.
  2. Add member interfaces to the aggregated Ethernet interfaces on both Switch A and Switch B.

    Switch A and Switch B:

    Switch A:

    Switch B:

  3. Configure a trunk interface between Switch A and Switch B.Note

    The port-mode statement is not supported on Enhanced Layer 2 Software (ELS). If you are running ELS, use the interface-mode statement.

    Orignal CLI:

    or

    ELS:

  4. Configure a multichassis protection link between Switch A and Switch B.

    Switch A:

    Switch B:

Step-by-Step Procedure

To enable ICCP:

  1. Configure the local IP address to be in the ICCP connection on Switch A and Switch B.

    Switch A:

    Switch B:

  2. Configure the peer IP address and minimum receive interval for a BFD session for ICCP on Switch A and Switch B.Note

    Configure at least 1000 ms as the minimum receive interval.

    Switch A:

    Switch B:

  3. Configure the peer IP address and minimum transmit interval for BFD session for ICCP on Switch A and Switch B.Note

    Configure at least 1000 ms as the transmit interval minimum interval.

    Switch A:

    Switch B:

  4. (Optional) Configure the time during which an ICCP connection must succeed between MC-LAG peers on Switch A and Switch B.Note

    On QFX and EX Series switches, the default session establishment hold time is 300 seconds. However, the session establishment time must be at least 100 seconds higher than the init delay time. You can optionally update the session establishment time to be 340 seconds and the init delay time to be 240 seconds.

    Switch A:

    Switch B:

  5. (Optional) Configure the backup IP address to be used for backup liveness detection on both Switch A and Switch B.Note

    By default, backup liveness detection is not enabled. Configuring a backup IP address helps achieve sub-second traffic loss during an MC-LAG peer reboot.

    Switch A:

    Switch B:

  6. Configure Layer 3 connectivity between the MC-LAG peers on both Switch A and Switch B.

    Original CLI:

    Original CLI:

    ELS:

    ELS:

    Note

    The port-mode statement is not supported on Enhanced Layer 2 Software (ELS). If you are running ELS, use the interface-mode statement.

Step-by-Step Procedure

To enable the MC-LAG interface:

  1. Enable LACP on the MC-LAG interface on Switch A and Switch B.Note

    At least one end needs to be active. The other end can be either active or passive.

  2. Specify the same multichassis aggregated Ethernet identification number on both MC-LAG peers on Switch A and Switch B.
  3. Specify the same service ID on Switch A and Switch B.

    ELS:

  4. Specify a unique chassis ID for the MC-LAG on the MC-LAG peers on Switch A and Switch B.

    Switch A:

    Switch B:

  5. Specify the operating mode of the MC-LAG on both Switch A and Switch B.Note

    Only active-active mode is supported at this time.

  6. Specify the status control for MC-LAG on Switch A and Switch B.Note

    You must configure status control on both Switch A and Switch B hosting the MC-LAG. If one peer is in active mode, the other must be in standby mode.

    Switch A:

    Switch B:

  7. Specify the number of seconds by which the bring-up of the multichassis aggregated Ethernet interface should be deferred after you reboot Switch A and Switch B.Note

    The recommended value for maximum VLAN configuration (for example, 4,000 VLANS) is 240 seconds. If IGMP snooping is enabled on all of the VLANs, the recommended value is 420 seconds.

  8. Specify the same LACP system ID for the MC-LAG on Switch A and Switch B.
  9. Specify the same LACP administration key on both Switch A and Switch B.
  10. Enable VLANs on the MC-LAG on Switch A and Switch B.Note

    The port-mode statement is not supported on Enhanced Layer 2 Software (ELS). If you are running ELS, use the interface-mode statement.

    Original CLI:

    Original CLI:

    Original CLI:

    or

    ELS:

    ELS:

    ELS:

Step-by-Step Procedure

To enable R2LGP:

Note

To process R2LGP requests, the mc-ae mode must be set to active-active.

  1. Configure the RSTP system identifier on Switch A and Switch B:

    Configure the RSTP system identifier on Switch A and Switch B.

Step-by-Step Procedure

To enable RSTP:

The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode.

  1. Enable RSTP globally on all interfaces on Switch A and Switch B.Note

    The all option is not available on ELS, so you cannot issue this command on ELS.

    ELS:

  2. Disable RSTP on the ICL-PL interfaces on Switch A and Switch B:Note

    This command is not needed on ELS.

  3. Configure the MC-LAG interfaces as edge ports on Switch A and Switch B.Note

    The ae1 interface is a downstream interface. This is why RSTP and bpdu-block-on-edge need to be configured. MC LAG switches are usually configured as root bridge. When downstream switches send superior BPDUs to the MC LAG switches, the MC LAG interfaces will be set as blocked by the downstream switches. The default behavior for the original CLI is to drop superior BPDUs.

  4. Enable BPDU blocking on all interfaces except for the ICL-PL interfaces on Switch A and Switch B.Note

    The ae1 interface is a downstream interface. This is why RSTP and bpdu-block-on-edge need to be configured.

Results

From configuration mode, confirm your configuration by entering the show chassis, show interfaces, show protocols, show multi-chassis, show switch-options, and show vlans commands. If the output does not display the intended configuration, repeat the instructions in this example to correct the configuration.

Switch A—Original CLI

Switch A-ELS

Switch B—Original CLI

Switch B—ELS

Verification

Verify that the configuration is working properly.

Verifying That ICCP Is Working on Switch A

Purpose

Verify that ICCP is running on Switch A.

Action

[edit]

user@switch> show iccp

Meaning

This output shows that the TCP connection between the peers hosting the MC-LAG is up, liveness detection is up, and MCSNOOPD and ESWD client applications are running.

Verifying That ICCP Is Working on Switch B

Purpose

Verify that ICCP is running on Switch B.

Action

show iccp

[edit]

user@switch> show iccp

Meaning

This output shows that the TCP connection between the peers hosting the MC-LAG is up, liveness detection is up, and MCSNOOPD and ESWD client applications are running.

Verifying That LACP Is Active on Switch A

Purpose

Verify that LACP is active on Switch A.

Action

[edit]

user@switch> show lacp interfaces

Meaning

This output shows that Switch A is participating in LACP negotiation.

Verifying That LACP Is Active on Switch B

Purpose

Verify that LACP is active on Switch B

Action

[edit]

user@switch> show lacp interfaces

Meaning

This output shows that Switch B is participating in LACP negotiation.

Verifying That the multichassis aggregated Ethernet and ICL-PL Interfaces Are Up on Switch A

Purpose

Verify that the multichassis aggregated Ethernet and ICL-PL interfaces are up on Switch A.

Action

[edit]

user@switch> show interfaces mc-ae

Meaning

This output shows that the multichassis aggregated Ethernet interface on Switch A is up and active.

Verifying That the multichassis aggregated Ethernet and ICL-PL Interfaces Are Up on Switch B

Purpose

Verify that the multichassis aggregated Ethernet and ICL-PL interfaces are up on Switch B.

Action

[edit]

user@switch> show interfaces mc-ae

Meaning

This output shows that the multichassis aggregated Ethernet interface on Switch B is up and active.

Verifying That MAC Learning Is Occurring on Switch A

Purpose

Verify that MAC learning is working on Switch A.

Action

[edit]

user@switch> show ethernet-switching table

Meaning

The output shows four learned MAC addresses entries.

Verifying That MAC Learning Is Occurring on Switch B

Purpose

Verify that MAC learning is working on Switch B.

Action

[edit]

user@switch> show ethernet-switching table

Meaning

The output shows four learned MAC addresses entries.

Troubleshooting

Troubleshooting a LAG That Is Down

Problem

The show interfaces terse command shows that the MC-LAG is down.

Solution

Check the following:

  1. Verify that there is no configuration mismatch.

  2. Verify that all member ports are up.

  3. Verify that the MC-LAG is part of family Ethernet switching (Layer 2 LAG).

  4. Verify that the MC-LAG member is connected to the correct MC-LAG member at the other end.