TechLibrary

Table of Contents

Guide That Contains This Content

[+] Expand All

[-] Collapse All

Example: Configuring Multichassis Link Aggregation for Layer 3 Unicast using MAC Address Synchronization

There are two methods for enabling Layer 3 unicast functionality across a multichassis link aggregation group (MC-LAG). You can choose either to synchronize the MAC addresses for the Layer 3 interfaces of the switches participating in the MC-LAG , or you can configure Virtual Router Redundancy Protocol (VRRP). The procedure to configure MAC address synchronization is included in this example. For more information on configuring VRRP for use in a Layer 3 unicast MC-LAG, see Example: Configuring Multichassis Link Aggregation for Layer 3 Unicast Using VRRP.

Requirements

This example uses the following hardware and software components:

Junos OS Release 12.3 or later for the QFX Series
Two QFX3500 or QFX3600 switches

Before you configure an MC-LAG for Layer 3 unicast, be sure that you understand how to:

Configure aggregated Ethernet interfaces on a switch. See Example: Configuring Link Aggregation Between a QFX Series Product and an Aggregation Switch.
Configure the Link Aggregation Control Protocol (LACP) on aggregated Ethernet interfaces on a switch. See Example: Configuring Link Aggregation with LACP Between a QFX Series Product and an Aggregation Switch.
Configure a standard MC-LAG between switches. See Example: Configuring Multichassis Link Aggregation.

Overview

In this example, you configure an MC-LAG across two switches by including interfaces from both switches in an aggregated Ethernet interface (ae1). To support the MC-LAG, create a second aggregated Ethernet interface (ae0) for the interchassis control link-protection link (ICL-PL). Configure a multichassis protection link for the ICL-PL, Interchassis Control Protocol (ICCP) for the peers hosting the MC-LAG, and Layer 3 connectivity between MC-LAG peers.

Note: Layer 3 connectivity is required for ICCP.

To complete the configuration, configure MAC address synchronization between the peers and specify the same IP address on both Layer 3 interface members (also known as the routed VLAN interface [RVI] or the integrated routing and bridging (IRB) interface) in the MC-LAG VLAN.

Topology

The topology used in this example consists of two switches hosting an MC-LAG. The two switches are connected to a server. Figure 1 shows the topology of this example.

Figure 1: Configuring a Multichassis LAG Between Switch A and Switch B

Table 1 details the topology used in this configuration example.

Table 1: Components of the Topology for Configuring a Multichassis LAG Between Two Switches

Hostname	Base Hardware	Multichassis Link Aggregation Group
Switch A Switch B	QFX3500 switch or QFX3600 switch QFX3500 switch or QFX3600 switch	ae0 is configured as an aggregated Ethernet interface, and is used as an ICL-PL. The following interfaces are part of ae0: xe-0/0/12 and xe-0/0/13 on Switch A and xe-0/0/12 and xe-0/0/13 on Switch B. These interfaces are included in VLAN v500. ae1 is configured as an MC-LAG, and the following two interfaces are part of ae1: xe-0/0/44 on Switch A and xe-0/0/46 on Switch B. These interfaces are included in VLAN v100.

Hostname

Base Hardware

Multichassis Link Aggregation Group

Switch A

Switch B

QFX3500 switch or QFX3600 switch

ae0 is configured as an aggregated Ethernet interface, and is used as an ICL-PL. The following interfaces are part of ae0: xe-0/0/12 and xe-0/0/13 on Switch A and
xe-0/0/12 and xe-0/0/13 on Switch B. These interfaces are included in VLAN v500.

ae1 is configured as an MC-LAG, and the following two interfaces are part of ae1:
xe-0/0/44 on Switch A and
xe-0/0/46 on Switch B. These interfaces are included in VLAN v100.

Configuration

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them in a text file, remove any line breaks, change any details necessary to match your network configuration, and paste the commands into the CLI at the [edit] hierarchy level of Switch A.

set chassis aggregated-devices ethernet device-count
2

set interfaces xe-0/0/12 ether-options 802.3ad
ae0

set interfaces xe-0/0/13 ether-options 802.3ad
ae0

set interfaces xe-0/0/44 ether-options 802.3ad
ae1

set interfaces ae0 unit 0 family ethernet-switching
port-mode trunk

set interfaces ae0 unit 0 family ethernet-switching
vlan members v500

set
interfaces ae0 unit 0 family ethernet-switching vlan members v100

set interfaces ae1 aggregated-ether-options
lacp active

set interfaces ae1 aggregated-ether-options
lacp system-id 00:01:02:03:04:05

set interfaces ae1 aggregated-ether-options
lacp admin-key 3

set interfaces ae1 aggregated-ether-options
mc-ae mc-ae-id 3

set interfaces ae1 aggregated-ether-options
mc-ae chassis-id 0

set interfaces ae1 aggregated-ether-options
mc-ae mode active-active

set interfaces ae1 aggregated-ether-options
mc-ae status-control active

set ae1 aggregated-ether-options mc-ae init-delay-time
240

set interfaces ae1 unit 0 family ethernet-switching
port-mode trunk

set interfaces ae1 unit 0 family ethernet-switching
vlan members v100

set interfaces vlan unit 100 family inet address
100.1.1.10

set interfaces vlan unit 500 family inet address
3.3.3.2/24

set vlans v100 vlan-id 100

set vlans v100 l3-interface vlan.100

set vlans v100 mcae-mac-synchronize

set vlans v500 vlan-id 500

set vlans v500 l3-interface vlan.500

set protocols iccp local-ip-addr 3.3.3.2

set protocols iccp peer 3.3.3.1 session-establishment-hold-time
50

set protocols iccp peer 3.3.3.1 backup-liveness-detection
backup-peer-ip 10.207.64.233

set protocols iccp peer 3.3.3.1 liveness-detection
minimum-receive-interval 60

set protocols iccp peer 3.3.3.1 liveness-detection
transmit-interval minimum-interval 60

set protocols rstp interface ae0.0 disable

set protocols rstp interface ae1.0 edge

set protocols rstp interface all mode point-to-point

set protocols rstp bpdu-block-on-edge

set multi-chassis multi-chassis-protection 3.3.3.1
interface ae0

set chassis aggregated-devices ethernet device-count
2

set interfaces xe-0/0/12 ether-options 802.3ad
ae0

set interfaces xe-0/0/13 ether-options 802.3ad
ae0

set interfaces xe-0/0/46 ether-options 802.3ad
ae1

set interfaces ae0 unit 0 family ethernet-switching
port-mode trunk

set interfaces ae0 unit 0 family ethernet-switching
vlan members v500

set
interfaces ae0 unit 0 family ethernet-switching vlan members v100

set interfaces ae1 aggregated-ether-options
lacp active

set interfaces ae1 aggregated-ether-options
lacp system-id 00:01:02:03:04:05

set interfaces ae1 aggregated-ether-options
lacp admin-key 3

set interfaces ae1 aggregated-ether-options
mc-ae mc-ae-id 3

set interfaces ae1 aggregated-ether-options
mc-ae chassis-id 1

set interfaces ae1 aggregated-ether-options
mc-ae mode active-active

set interfaces ae1 aggregated-ether-options
mc-ae status-control standby

set ae1 aggregated-ether-options mc-ae init-delay-time
240

set interfaces ae1 unit 0 family ethernet-switching
port-mode trunk

set interfaces ae1 unit 0 family ethernet-switching
vlan members v100

set interfaces vlan unit 100 family inet address
100.1.1.10

set interfaces vlan unit 500 family inet address
3.3.3.1/24

set vlans v100 vlan-id 100

set vlans v100 l3-interface vlan.100

set vlans v100 mcae-mac-synchronize

set vlans v500 vlan-id 500

set vlans v500 l3-interface vlan.500

set protocols iccp local-ip-addr 3.3.3.1

set protocols iccp peer 3.3.3.2 session-establishment-hold-time
50

set protocols iccp peer 3.3.3.2 backup-liveness-detection
backup-peer-ip 10.207.64.234

set protocols iccp peer 3.3.3.2 liveness-detection
minimum-receive-interval 60

set protocols iccp peer 3.3.3.2 liveness-detection
transmit-interval minimum-interval 60

set protocols rstp interface ae0.0 disable

set protocols rstp interface ae1.0 edge

set protocols rstp interface all mode point-to-point

set protocols rstp bpdu-block-on-edge

set multi-chassis multi-chassis-protection 3.3.3.2
interface ae0

Configuring MC-LAG on Two Switches

Step-by-Step Procedure

To enable multichassis protection link between MC-LAG peers:

The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode in the CLI User Guide.

Configure the number of LAGs on both Switch A and Switch B.
[edit chassis] user@switch# set aggregated-devices ethernet device-count 2
Add member interfaces to the aggregated Ethernet interfaces on both Switch A and Switch B.
Switch A and Switch B:
[edit interfaces] user@switch# set xe-0/0/12 ether-options 802.3ad ae0
[edit interfaces] user@switch# set xe-0/0/13 ether-options 802.3ad ae0
Switch A
[edit interfaces] user@switch# set xe-0/0/44 ether-options 802.3ad ae1
Switch B
[edit interfaces] user@switch# set xe-0/0/46 ether-options 802.3ad ae1
Configure a trunk interface between Switch A and Switch B.
[edit interfaces] user@switch# set ae0 unit 0 family ethernet-switching port-mode trunk
Configure a multichassis protection link between Switch A and Switch B.
Switch A:
[edit] user@switch# set multi-chassis multi-chassis-protection 3.3.3.2 interface ae0
Switch B:
[edit] user@switch# set multi-chassis multi-chassis-protection 3.3.3.1 interface ae0

Step-by-Step Procedure

To enable ICCP:

Configure the local IP address to be in the ICCP connection on Switch A and Switch B.
Switch A:
[edit protocols] user@switch# set iccp local-ip-addr 3.3.3.2
Switch B:
[edit protocols] user@switch# set iccp local-ip-addr 3.3.3.1
Configure the peer IP address and minimum receive interval for a (BFD) session for ICCP on Switch A and Switch B.
Switch A:
[edit protocols] user@switch# set iccp peer 3.3.3.1 liveness-detection minimum-receive-interval 60
Switch B:
[edit protocols] user@switch# set iccp peer 3.3.3.2 liveness-detection minimum-receive-interval 60
Configure the peer IP address and minimum transmit interval for Bidirectional Forwarding Detection (BFD) session for ICCP on Switch A and Switch B.
Switch A:
[edit protocols] user@switch# set iccp peer 3.3.3.1 liveness-detection transmit-interval minimum-interval 60
Switch B:
[edit protocols] user@switch# set iccp peer 3.3.3.2 liveness-detection transmit-interval minimum-interval 60
(Optional) Configure the time during which an ICCP connection must succeed between MC-LAG peers on Switch A and Switch B.
Note: Configuring session establishment hold time helps in faster ICCP connection establishment. The recommended value is 50 seconds.
Switch A:
[edit protocols] user@switch# set iccp peer 3.3.3.1 session-establishment-hold-time 50
Switch B:
[edit protocols] user@switch# set iccp peer 3.3.3.2 session-establishment-hold-time 50
(Optional) Configure the backup IP address to be used for backup liveness detection on both Switch A and Switch B.
Note: By default, backup liveness detection is not enabled. Configuring a backup IP address helps achieve sub-second traffic loss during a MC-LAG peer reboot.
Switch A:
[edit protocols] user@switch# set iccp peer 3.3.3.1 backup-liveness-detection backup-peer-ip 10.207.64.233
Switch B:
[edit protocols] user@switch# set iccp peer 3.3.3.2 backup-liveness-detection backup-peer-ip 10.207.64.234
Configure Layer 3 connectivity across the ae0 ICCP link by adding a Layer 3 interface on both Switch A and Switch B.
[edit vlans] user@switch# set v500 vlan-id 500
[edit vlans] user@switch# set v500 l3-interface vlan.500
[edit interfaces] user@switch# set ae0 unit 0 family ethernet-switching port-mode trunk vlan members v500
[edit interfaces] user@switch# set ae0 unit 0 family ethernet-switching port-mode trunk vlan members v100

Step-by-Step Procedure

To enable the MC-LAG interface:

Enable LACP on the MC-LAG interface on Switch A and Switch B.
Note: At least one end needs to be active. The other end can be either active or passive.
[edit interfaces] user@switch# set ae1 aggregated-ether-options lacp active
Specify the same multichassis aggregated Ethernet identification number on both MC-LAG peers on Switch A and Switch B.
[edit interfaces] user@switch# set ae1 aggregated-ether-options mc-ae mc-ae-id 3
Specify a unique chassis ID for the MC-LAG on the MC-LAG peers on Switch A and Switch B.
Switch A:
[edit interfaces] user@switch# set ae1 aggregated-ether-options mc-ae chassis-id 0
Switch B:
[edit interfaces] user@switch# set ae1 aggregated-ether-options mc-ae chassis-id 1
Specify the operating mode of the MC-LAG on both Switch A and Switch B.
Note: Only active-active mode is supported at this time.
[edit interfaces] user@switch# set ae1 aggregated-ether-options mc-ae mode active-active
Specify the status control for MC-LAG on Switch A and Switch B.
Note: You must configure status control on both Switch A and Switch B hosting the MC-LAG. If one peer is in active mode, the other must be in standby mode.
Switch A:
[edit interfaces] user@switch# set ae1 aggregated-ether-options mc-ae status-control active
Switch B:
[edit interfaces] user@switch# set ae1 aggregated-ether-options mc-ae status-control standby
Specify the number of seconds by which the bring-up of the MC-AE interface should be deferred after you reboot Switch A and Switch B.
Note: The recommended value for maximum VLAN configuration (for example, 4,000 VLANS) is 240 seconds. If IGMP snooping is enabled on all of the VLANs, the recommended value is 420 seconds.
[edit interfaces] user@switch# set ae1 aggregated-ether-options mc-ae init-delay-time 240
Specify the same LACP system ID for the MC-LAG on Switch A and Switch B.
[edit interfaces] user@switch# set ae1 aggregated-ether-options lacp system-ID 00:01:02:03:04:05
Specify the same LACP administration key on both Switch A and Switch B.
[edit interfaces] user@switch# set ae1 aggregated-ether-options lacp admin-key 3
Enable a VLAN on the MC-LAG on Switch A and Switch B.
[edit interfaces] user@switch# set ae1 unit 0 family ethernet-switching port-mode trunk
[edit] user@switch# set vlans v100 vlan-id 100
[edit interfaces] user@switch# set ae1 unit 0 family ethernet-switching vlan members v100
Create a Layer 3 interface for the MC-LAG VLAN and assign the same IP address on both Switch A and Switch B.
[edit] user@switch# set vlans v100 l3-interface vlan.100
[edit interfaces] user@switch# set vlan unit 100 family inet address 100.1.1.10
Configure MAC address synchronization in the MC-LAG VLAN on both Switch A and Switch B.
[edit] user@switch# set vlans v100 mcae-mac-synchronize

Step-by-Step Procedure

To enable RSTP:

Enable RSTP globally on all interfaces on Switch A and Switch B.
[edit] user@switch# set protocols rstp interface all mode point-to-point
Disable RSTP on the ICL-PL interfaces on Switch A and Switch B:
[edit] user@switch# set protocols rstp interface ae0.0 disable
Configure the MC-LAG interfaces as edge ports on Switch A and Switch B.
Note: The ae1 interface is a downstream interface. This is why RSTP and bpdu-block-on-edge need to be configured.
[edit] user@switch# set protocols rstp interface ae1.0 edge
Enable BPDU blocking on all interfaces except for the ICL-PL interfaces on Switch A and Switch B.
Note: The ae1 interface is a downstream interface. This is why RSTP and bpdu-block-on-edge need to be configured.
[edit] user@switch# set protocols rstp bpdu-block-on-edge

Results

Display the results of the configuration on Switch A.

chassis {aggregated-devices {ethernet {device-count 2;}}}

interfaces {xe-0/0/12 {ether-options {802.3ad ae0;}}xe-0/0/13 {ether-options {802.3ad ae0;}}xe-0/0/44 {ether-options {802.3ad ae1;}}ae0 {unit 0 {family ethernet-switching {port-mode trunk;vlan {members v500;}}}}ae1 {aggregated-ether-options {lacp {active;system-id 00:01:02:03:04:05;admin-key 3;}mc-ae {mc-ae-id 3;chassis-id 0;mode active-active;status-control active;init-delay-time 240}}unit 0 {family ethernet-switching {port-mode trunk;vlan {members v100;}}}}vlan {unit 100 {family inet {address 100.1.1.10/32;}}unit 500 {family inet {address 3.3.3.2/24;}}}}

protocols {iccp {local-ip-addr 3.3.3.2;peer 3.3.3.1 {session-establishment-hold-time 50;backup-liveness-detection {backup-peer-ip 10.207.64.233;}liveness-detection {minimum-receive-interval 60;transmit-interval {minimum-interval 60;}}}}rstp {interface ae0.0 {disable;}interface ae1.0 {edge;}interface all {mode point-to-point;}bpdu-block-on-edge;}}

multi-chassis {multi-chassis-protection 3.3.3.1 {interface ae0;}}

vlans {v100 {vlan-id 100;l3-interface vlan.100;mcae-mac-synchronize;}v500 {vlan-id 500;l3-interface vlan.500;}}

Display the results of the configuration on Switch B.

chassis {aggregated-devices {ethernet {device-count 2;}}}

interfaces {xe-0/0/12 {ether-options {802.3ad ae0;}}xe-0/0/13 {ether-options {802.3ad ae0;}}xe-0/0/46 {ether-options {802.3ad ae1;}}ae0 {unit 0 {family ethernet-switching {port-mode trunk;vlan {members v500;}}}}ae1 {aggregated-ether-options {lacp {active;system-id 00:01:02:03:04:05;admin-key 3;}mc-ae {mc-ae-id 3;chassis-id 1;mode active-active;status-control standby;init-delay-time 240}}unit 0 {family ethernet-switching {port-mode trunk;vlan {members v100;}}}}vlan {unit 100 {family inet {address 100.1.1.10/32;}}unit 500 {family inet {address 3.3.3.1/24;}}}}

protocols {iccp {local-ip-addr 3.3.3.1;peer 3.3.3.2 {session-establishment-hold-time 50;backup-liveness-detection {backup-peer-ip 10.207.64.234;}liveness-detection {minimum-receive-interval 60;transmit-interval {minimum-interval 60;}}}}rstp {interface ae0.0 {disable;}interface ae1.0 {edge;}interface all {mode point-to-point;}bpdu-block-on-edge;}}

multi-chassis {multi-chassis-protection 3.3.3.2 {interface ae0;}}

vlans {v100 {vlan-id 100;l3-interface vlan.100;mcae-mac-synchronize;}v500 {vlan-id 500;l3-interface vlan.500;}}

Verification

To verify that the MC-LAG group has been created and is working properly, perform these tasks:

Verifying That ICCP Is Working on Switch A

Purpose

Verify that ICCP is running on Switch A.

Action

[edit]
user@switch# show iccp

Redundancy Group Information for peer 3.3.3.1
  TCP Connection       : Established
  Liveliness Detection : Up

 Client Application: MCSNOOPD

 Client Application: eswd

Meaning

This output shows that the TCP connection between the peers hosting the MC-LAG is up, liveness detection is up, and MCSNOOPD and ESWD client applications are running.

Verifying That ICCP Is Working on Switch B

Purpose

Verify that ICCP is running on Switch B.

Action

show iccp

[edit]
user@switch# show iccp

Redundancy Group Information for peer 3.3.3.2
  TCP Connection       : Established
  Liveliness Detection : Up

 Client Application: MCSNOOPD

 Client Application: eswd

Meaning

This output shows that the TCP connection between the peers hosting the MC-LAG is up, liveness detection is up, and MCSNOOPD and ESWD client applications are running.

Verifying That LACP Is Active on Switch A

Purpose

Verify that LACP is active on Switch A.

Action

[edit]
user@switch# show lacp
interfaces

Aggregated interface: ae1
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      xe-0/0/46      Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      xe-0/0/46    Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State
      xe-0/0/46                 Current   Fast periodic Collecting distributing

Meaning

This output shows that Switch A is participating in LACP negotiation.

Verifying That LACP Is Active on Switch B

Purpose

Verify that LACP is active on Switch B

Action

[edit]
user@switch# show lacp
interfaces

Aggregated interface: ae1
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      xe-0/0/44      Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      xe-0/0/44    Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State
      xe-0/0/44                 Current   Fast periodic Collecting distributing

Meaning

This output shows that Switch B is participating in LACP negotiation.

Verifying That the MC-AE and ICL-PL Interfaces Are Up on Switch A

Purpose

Verify that the MC-AE and ICL-PL interfaces are up on Switch A.

Action

[edit]
user@switch# show interfaces
mc-ae

Member Link                  : ae1
 Current State Machine's State: mcae active state
 Local Status                 : active
 Local State                  : up
 Peer Status                  : active
 Peer State                   : up
     Logical Interface        : ae1.0
     Topology Type            : bridge
     Local State              : up
     Peer State               : up
     Peer Ip/MCP/State        : 3.3.3.1 ae0.0 up

Meaning

This output shows that the MC-AE interface on Switch A is up and active.

Verifying That the MC-AE and ICL-PL Interfaces Are Up on Switch B

Purpose

Verify that the MC-AE and ICL-PL interfaces are up on Switch B.

Action

[edit]
user@switch# show interfaces
mc-ae

Member Link                  : ae1
 Current State Machine's State: mcae active state
 Local Status                 : active
 Local State                  : up
 Peer Status                  : active
 Peer State                   : up
     Logical Interface        : ae1.0
     Topology Type            : bridge
     Local State              : up
     Peer State               : up
     Peer Ip/MCP/State        : 3.3.3.2 ae0.0 up

Meaning

This output shows that the MC-AE interface on Switch B is up and active.

Verifying MAC Address Synchronization on Switch A and Switch B

Purpose

Confirm that MAC address synchronization is working on both Switch A and Switch B.

Action

[edit]
user@switch# show ethernet-switching
table vlan v100

Ethernet-switching table: 3 unicast entries
  VLAN	            MAC address       Type         Age Interfaces
  v100              *                 Flood          - All-members
  v100              84:18:88:df:35:36 Static         - Router
  v100              84:18:88:df:83:0a Static         - Router

Meaning

The output shows two static MAC addresses in VLAN v100. The addresses belong to the Layer 3 IRB/RVI interfaces of both Switch A and Switch B that you configured in the MC-LAG VLAN. Appearance of both addresses indicates that MAC address synchronization is working.

Troubleshooting

Troubleshooting a LAG That Is Down

Problem

The show interfaces terse command shows that the MC-LAG is down

Solution

Check the following:

Verify that there is no configuration mismatch.
Verify that all member ports are up.
Verify that the MC-LAG is part of family Ethernet switching (Layer 2 LAG).
Verify that the MC-LAG member is connected to the correct MC-LAG member at the other end.

TechLibrary

Related Documentation

Example: Configuring Multichassis Link Aggregation for Layer 3 Unicast using MAC Address Synchronization

Requirements

Overview

Topology

Configuration

CLI Quick Configuration

Configuring MC-LAG on Two Switches

Step-by-Step Procedure

Step-by-Step Procedure

Step-by-Step Procedure

Step-by-Step Procedure

Results

Verification

Verifying That ICCP Is Working on Switch A

Purpose

Action

Meaning

Verifying That ICCP Is Working on Switch B

Purpose

Action

Meaning

Verifying That LACP Is Active on Switch A

Purpose

Action

Meaning

Verifying That LACP Is Active on Switch B

Purpose

Action

Meaning

Verifying That the MC-AE and ICL-PL Interfaces Are Up on Switch A

Purpose

Action

Meaning

Verifying That the MC-AE and ICL-PL Interfaces Are Up on Switch B

Purpose

Action

Meaning

Verifying MAC Address Synchronization on Switch A and Switch B

Purpose

Action

Meaning

Troubleshooting

Troubleshooting a LAG That Is Down

Problem

Solution

Related Documentation

Published: 2015-03-16