Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

High Availability in Layer 3 Networks Using VRRP and MAC Address Synchronization for MC-LAG

 

Active-Active Bridging and VRRP over IRB Functionality Overview

Active-active bridging and VRRP over IRB support extends multichassis link aggregation group (MC-LAG) by adding the following functionality to MX Series routers and QFX Series switches:

  • Interchassis link (ICL) pseudowire interface or Ethernet interface (ICL-PL field) for active-active bridging

  • Active-active bridging

  • VRRP over IRB for active-active bridging

  • A single bridge domain not corresponding to two redundancy group IDs

How Active-Active Bridging over IRB Functionality Works

Active-Active bridging over IRB functionality uses the address resolution protocol (ARP) Active-Active MC-LAG.

Suppose one of the PE routers issues an ARP request and another PE router gets the response and, because of the aggregated Ethernet distribution logic, the ARP resolution is not successful. Junos OS uses ARP response packet snooping to perform active-active multichassis link aggregation group support, providing synchronization without the need to maintain any specific state.

Address Resolution Protocol Active-Active MC-LAG Support Methodology

Suppose one of the PE routers issues an ARP request and another PE router gets the response and, because of the aggregated Ethernet distribution logic, the ARP resolution is not successful. Junos OS uses ARP response packet snooping to perform active-active multichassis link aggregation group support, providing easy synchronization without the need to maintain any specific state.

Benefits of Active-Active Bridging and VRRP over IRB Functionality

Benefits of active-active bridging and VRRP over IRB functionality include:

  • An MC-LAG reduces operational expenses by providing active-active links with a LAG, eliminates the need for Spanning Tree Protocol (STP), and provides faster Layer 2 convergence upon link and device failures.

  • An MC-LAG adds node-level redundancy to the normal link-level redundancy that a LAG provides. An MC-LAG improves network resiliency, which reduces network down time as well as expenses.

  • In data centers, it is desirable for servers to have redundant connections to the network. You probably want active-active connections along with links from any server to at least two separate routers.

  • An MC-LAG allows you to bond two or more physical links into a logical link between two routers or between a server and a router, which improves network efficiency. An MC-LAG enables you to load-balance traffic on multiple physical links. If a link fails, the traffic can be forwarded through the other available link, and the logical aggregated link remains in the UP state.

Where Can I Use Active-Active Bridging and VRRP over IRB Functionality?

Active-active bridging and Virtual Router Redundancy Protocol (VRRP) over integrated routing and bridging (IRB) is supported on MX Series routers and QFX Series switches.

MC-LAG Functions in an Active-Active Bridging Domain

The following functions are supported for MC-LAG in an active-active bridging domain:

  • MC-LAG is supported only between two chassis, using an interchassis link (ICL) pseudowire interface or Ethernet interface (ICL-PL field) for active-active bridging, and active-active bridging VRRP over IRB for active-active bridging.

  • For VPLS networks, you can configure the aggregated Ethernet (aeX) interfaces on MC-LAG devices with the encapsulation ethernet-vpls statement to use Ethernet VPLS encapsulation on Ethernet interfaces that have VPLS enabled and that must accept packets carrying standard Tag Protocol ID (TPID) values or the encapsulation vlan-vpls statement to use Ethernet VLAN encapsulation on VPLS circuits.

  • Layer 2 circuit functionalities are supported with ethernet-ccc as the encapsulation mode.

  • Network topologies in a triangular and square pattern are supported. In a triangular network design, with equal-cost paths to all redundant nodes, slower, timer-based convergence can possibly be prevented. Instead of indirect neighbor or route loss detection using hellos and dead timers, you can identify the physical link loss and denote a path as unusable and reroute all traffic to the alternate equal-cost path. In a square network design, depending on the location of the failure, the routing protocol might converge to identify a new path to the subnet or the VLAN, causing the convergence of the network to be slower.

  • Interoperation of Link Aggregation Control Protocol (LACP) for MC-LAG devices is supported. LACP is one method of bundling several physical interfaces to form one logical interface. When LACP is enabled, the local and remote sides of the aggregated Ethernet links exchange protocol data units (PDUs), which contain information about the state of the link. You can configure Ethernet links to actively transmit PDUs, or you can configure the links to passively transmit them, sending out LACP PDUs only when the links receive the PDUs from another link. One side of the link must be configured as active for the link to be up.

  • Active-standby mode is supported using LACP. When an MC-LAG operates in the active-standby mode, one of the router’s ports only becomes active when failure is detected in the active links. In this mode, the provider edge (PE) routers perform an election to determine the active and standby routers.

  • Configuration of the pseudowire status type length variable (TLV) is supported. The pseudowire status TLV is used to communicate the status of a pseudowire back and forth between two PE routers. The pseudowire status negotiation process ensures that a PE router reverts back to the label withdraw method for pseudowire status if its remote PE router neighbor does not support the pseudowire status TLV.

  • The MC-LAG devices use Inter-Chassis Control Protocol (ICCP) to exchange the control information between two MC-LAG network devices.

Points to Remember When Configuring MC-LAG Active-Active Bridge Domains

Keep the following points in mind when you configure MC-LAG in an active-active bridging domain:

  • A single bridge domain cannot be associated with two redundancy groups. You cannot configure a bridge domain to contain logical interfaces from two different multichassis aggregated Ethernet interfaces and associate them with different redundancy group IDs by using the redundancy group group-id statement at the [edit interfaces aeX aggregated-ether-options] hierarchy level.

  • You must configure logical interfaces in a bridge domain from a single multichassis aggregated Ethernet interface and associate it with a redundancy group. You must configure a service ID by including the service-id vid statement at the [edit bridge-domains bd-name] hierarchy level for multichassis aggregated Ethernet interfaces if you configure logical interfaces on multichassis aggregated Ethernet interfaces that are part of the bridge domain.

More Data Traffic Forwarding Rules

In active-active bridging and VRRP over IRB topographies, network interfaces are categorized into three different interface types, as follows:

S-LinksSingle-homed link (S-Link) terminating on MC-LAG-N device or MC-LAG in active-standby mode. In Figure 4, interfaces ge-0/0/0.0 and ge-1/0/0.0 are S-Links.
MC-LinksMC-LAG links. In Figure 4, interface ae0.0 is the MC-Link.
ICLInterchassis link.

Based on incoming and outgoing interface types, some constraints are added to the Layer 2 forwarding rules for MC-LAG configurations, as described in the data traffic forwarding rules. Note that if only one of the MC-LAG member link is in the UP state, it is considered an S-Link.

The following data traffic forwarding rules apply:

  1. When an MC-LAG network receives a packet from a local MC-Link or S-Link, the packet is forwarded to other local interfaces, including S-Links and MC-Links based on the normal Layer 2 forwarding rules and on the configuration of the mesh-group and no-local-switching statements. If MC-Links and S-Links are in the same mesh group and their no-local-switching statements are enabled, the received packets are only forwarded upstream and not sent to MC-Links and S-Links.Note

    The functionality described in Rule 2 is not supported.

  2. The following circumstances determine whether or not an ICL receives a packet from a local MC-Link or S-Link:
    1. If the peer MC-LAG network device has S-Links or MC-LAGs that do not reside on the local MC-LAG network device

    2. Whether or not interfaces on two peering MC-LAG network devices are allowed to talk to each other only if both a. and b. are true. Traffic is always forwarded to the ICL.

  3. When an MC-LAG network receives a packet from the ICL, the packet is forwarded to all local S-Links and active MC-LAGs that do not exist in the MC-LAG network that the packet comes from.
  4. Note

    The topology shown in Figure 1 is not supported.

    In certain cases, for example the topology shown in Figure 1, there could be a loop caused by the ICL. To break the loop, one of the following mechanisms could be used:
    1. Run certain protocols, such as STP. In this case, whether packets received on one ICL are forwarded to other ICLs is determined by using Rule 3.

    2. Configure the ICL to be fully meshed among the MC-LAG network devices. In this case, traffic received on the ICL would not be forwarded to any other ICLs.

    In either case, duplicate packets could be forwarded to the MC-LAG clients. Consider the topology shown in Figure 1, where if network routing instance N1 receives a packet from ge-0/0/0.0, it could be flooded to ICL1 and ICL3.

    When receiving from ICL1 and ICL3, network routing instances N3 and N2 could flood the same packet to MCL2, as shown in Figure 1. To prevent this from happening, the ICL designated forwarder should be elected between MC-LAG peers, and traffic received on an ICL could be forwarded to the active-active MC-LAG client by the designated forwarder only.

  5. When received from an ICL, traffic should not be forwarded to the core-facing client link connection between two provider edge (PE) devices (MC-Link) if the peer chassis's (where the traffic is coming from) MC-Link is UP.

How to Configure MC-LAG Active-Active Bridge Domains

For a MC-LAG configured in an active-active bridge domain and with VRRP configured over an IRB interface, you must include the accept-data statement at the [edit interfaces interface-name unit logical-unit-number family inet address address vrrp-group group-id] hierarchy level to enable the router that functions as the master router to accept all packets destined for the virtual IP address.

On an MC-LAG, if you modify the source MAC address to be the virtual MAC address, you must specify the virtual IP address as the source IP address instead of the physical IP address. In such a case, the accept-data option is required for VRRP to prevent ARP from performing an incorrect mapping between IP and MAC addresses for customer edge (CE) devices. The accept-data attribute is needed for VRRP over IRB interfaces in MC-LAG to enable OSPF or other Layer 3 protocols and applications to work properly over multichassis aggregated Ethernet (mc-aeX) interfaces.

Note

On an MC-LAG, the unit number associated with aggregated Ethernet interfaces on provider edge router PE1 must match the unit number associated with aggregated Ethernet interfaces on provider edge router PE2. If the unit numbers differ, MAC address synchronization does not happen. As a result, the status of the MAC address on the remote provider edge router remains in a pending state.

If you are using the VRRP over IRB or RVI method to enable Layer 3 functionality, you must configure static ARP entries for the IRB or RVI interface of the remote MC-LAG peer to allow routing protocols to run over the IRB or RVI interfaces.

MAC Address Management

If an MC-LAG is configured to be active-active, upstream and downstream traffic could go through different MC-LAG network devices. Since the media access control (MAC) address is learned only on one of the MC-LAG network devices, the reverse direction's traffic could be going through the other MC-LAG network and be flooded unnecessarily. Also, a single-homed client's MAC address is only learned on the MC-LAG network device it is attached to. If a client attached to the peer MC-LAG network needs to communicate with that single-homed client, then traffic would be flooded on the peer MC-LAG network device. To avoid unnecessary flooding, whenever a MAC address is learned on one of the MC-LAG network devices, it gets replicated to the peer MC-LAG network device. The following conditions should be applied when MAC address replication is performed:

  • MAC addresses learned on an MC-LAG of one MC-LAG network device should be replicated as learned on the same MC-LAG of the peer MC-LAG network device.

  • MAC addresses learned on single-homed customer edge (CE) clients of one MC-LAG network device should be replicated as learned on the ICL-PL interface of the peer MC-LAG network device.

  • MAC addresses learned on MC-LAG VE clients of one MC-LAG network device should be replicated as learned on the corresponding VE interface of the peer MC-LAG network device.

  • MAC address learning on an ICL is disabled from the data path. It depends on software to install MAC addresses replicated through Inter-Chassis Control Protocol (ICCP).

MAC Aging

MAC aging support in Junos OS extends aggregated Ethernet logic for a specified MC-LAG. A MAC address in software is deleted until all Packet Forwarding Engines have deleted the MAC address. In the case of an MC-LAG, a remote provider edge is treated as a remote Packet Forwarding Engine and has a bit in the MAC data structure.

Layer 3 Routing

In general, when an MC-LAG is configured to provide Layer 3 routing functions to downstream clients, the MC-LAG network peers should be configured to provide the same gateway address to the downstream clients. To the upstream routers, the MC-LAG network peers could be viewed as either equal-cost multipath (ECMP) or two routes with different preference values.

Junos OS supports active-active MC-LAGs by using VRRP over IRB. Junos OS also supports active-active MC-LAGs by using IRB MAC address synchronization. You must configure IRB using the same IP address across MC-LAG peers. IRB MAC synchronization is supported on 32-bit interfaces and interoperates with earlier MPC and MIC releases.

To ensure that Layer 3 operates properly, instead of dropping the Layer 3 packet, the VRRP backup attempts to perform routing functions if the packet is received on an MC-LAG. A VRRP backup sends and responds to Address Resolution Protocol (ARP) requests.

For ARP, the same issue exists as with Layer 2 MAC addresses. Once ARP is learned, it must be replicated to the MC-LAG through ICCP. The peer must install an ARP route based on the ARP information received through ICCP.

For ARP aging, ARP requests on the MC-LAG peers can be aged out independently.

Topologies Supported for MC-LAG Active-Active Bridge Domains

The topologies shown in Figure 2 and Figure 3 are supported. These figures use the following abbreviations:

  • Aggregated Ethernet (AE)

  • Interchassis link (ICL)

  • Multichassis link (MCL)

Potential Problems When Configuring MC-LAG Active-Active Bridge Domains

When configured to be active-active, the client device load-balances the traffic to the peering MC-LAG network devices. In a bridging environment, this could potentially cause the following problems:

  • Traffic received on the MC-LAG from one MC-LAG network device could be looped back to the same MC-LAG on the other MC-LAG network device.

  • Duplicated packets could be received by the MC-LAG client device.

  • Traffic could be unnecessarily forwarded on the interchassis link.

To better illustrate the problems listed, consider Figure 4, where an MC-LAG device MCL1 and single-homed clients ge-0/0/0.0 and ge-1/0/0.0 are allowed to talk to each other through an ICL. These problems could occur:

Figure 4: MC-LAG Device and Single-Homed Client
MC-LAG Device
and Single-Homed Client
  • Traffic received on network routing instance N1 from MCL1 could be flooded to ICL to reach network routing instance N2. Once it reaches network routing instance N2, it could flood again to MCL1.

  • Traffic received on interface ge-0/0/0.0 could be flooded to MCL1 and ICL on network routing instance N1. Once network routing instance N2 receives such traffic from ICL, it could again be flooded to MCL1.

  • If interface ge-1/0/0.0 does not exist on network routing instance N2, traffic received from interface ge-0/0/0.0 or MCL1 on network routing instance N1 could be flooded to network routing instance N2 through ICL unnecessarily since interface ge-0/0/0.0 and MCL1 could reach each other through network routing instance N1.

Restrictions When Configuring MC-LAG Active-Active Bridge Domains

In an IPv6 network, you cannot configure an MC-LAG in an active-active bridge domain if you specified the vlan-id none statement at the [edit bridge-domain bd-name] hierarchy level. The vlan-id none statement that enables the removal of the incoming VLAN tags identifying a Layer 2 logical interface when packets are sent over VPLS pseudowires is not supported for IPv6 packets in an MC-LAG.

The following functionality is not supported for MC-LAG active-active bridge domains:

  • Virtual private LAN service (VPLS) within the core

  • Bridged core

  • Topology as described in Rule 4 of More Data Traffic Forwarding Rules

  • Routed multichassis aggregated Ethernet interface, where the VRRP backup router is used in the edge of the network

  • Track object, where in the case of an MC-LAG, the status of the uplinks from the provider edge can be monitored, and the MC-LAG can act on the status

  • Mixed mode (active-active MC-LAG is supported on MX Series routers with MPC or MIC interfaces only)

    All interfaces in the bridge domain that are multichassis aggregated Ethernet active-active must be on MPCs or MICs.

The topologies shown in Figure 5, Figure 6, and Figure 7 are not supported:

Figure 5: Interchassis Data Link Between Active-Active Nodes
Interchassis Data Link Between Active-Active
Nodes
Figure 6: Active-Active MC-LAG with Single MC-LAG
Active-Active MC-LAG with Single MC-LAG
Figure 7: Active-Active MC-LAG with Multiple Nodes on a Single Multichassis Link
Active-Active MC-LAG with Multiple Nodes
on a Single Multichassis Link
Note

A redundancy group cannot span more than two routers.

IGMP Snooping on Active-Active MC-LAG

IGMP Snooping on Active-Active MC-LAG

For multicast to work in an active-active MC-LAG scenario, the typical topology is as shown in Figure 8 and Figure 9 with interested receivers over S-links and MC-Links. Starting in Junos OS Release 11.2, support is extended for sources connected over the Layer 2 interface.

If an MC-LAG is configured to be active-active, reports from MC-LAG clients could reach any of the MC-LAG network device peers. Therefore, the IGMP snooping module needs to replicate the states such that the Layer 2 multicast route state on both peers are the same. Additionally for S-Link clients, snooping needs to replicate these joins to its snooping peer, which in the case of Layer 3 connected source, passes this information to the PIM on IRB to enable the designated router to pull traffic for these groups,

The ICL should be configured as a router facing interface. For the scenario where traffic arrives through a Layer 3 interface, it is a requirement to have PIM and IGMP enabled on the IRB interface configured on the MC-LAG network device peers.

With reference to Figure 8, either Device N1 or N2 becomes a designated router (for this example, N1 is the designated router). Router N1 therefore pulls the multicast traffic from the core. Once multicast data hits the network Device N1, the data is forwarded based on the snooping learned route.

For MC-Link clients, data is forwarded through N1. In the case of failover of the MC-Links, the data reaches the client through N2. For S-Link clients on N1, data would be forwarded through normal snooping routes.

For S-Link clients on N2, data is forwarded through the ICL interface. Layer 2 multicast routes on N1 do not show these groups unless there is interest for the same group over MC-Links or over S-Links on N1. For the IRB scenario, the IGMP membership and Layer 3 multicast route on N1 does however show these groups learned over the IRB interface.

Therefore, for a case where a specific group interest is only on the S-Link on N2, data arriving on N1 reaches N2 through the default route, and the Layer 2 multicast route on N2 has the S-Link in the outgoing interface list.

In Figure 9, MCL1 and MCL2 are on different devices, and the multicast source or IGMP querier is connected through MCL2. The data forwarding behavior seen is similar to that explained for multicast topology with source connected through Layer 3.

Note

IGMP snooping should not be configured in proxy mode. There should be no IGMP hosts or IGMP or PIM routers sitting on the ICL interface.

Up and Down Event Handling

The following conditions apply to up and down event handling:

  • If the Inter-Chassis Control Protocol (ICCP) connection is UP but the ICL interface goes DOWN, the router configured as the backup brings down all the multichassis aggregated Ethernet interfaces shared with the peer that is connected to ICL. This ensures that there are no loops in the network. Otherwise, both PEs become PIM-designated routers and, hence, forward multiple copies of the same packet to the customer edge.

  • If the ICCP connection is UP and the ICL comes UP, the router configured as the backup brings up the multichassis aggregated Ethernet interfaces shared with the peer.

  • If both the ICCP connection and the ICL are DOWN, the router configured as the backup brings up the multichassis aggregated Ethernet interfaces shared with the peer.

  • The Layer 2 address learning process (l2ald) does not store the information about a MAC address learned from a peer in the kernel. If l2ald restarts, and if the MAC address was not learned from the local multichassis aggregated Ethernet interface, l2ald clears the MAC addresses, which causes the router to flood the packets destined to this MAC address. This behavior is similar to that in a Routing Engine switchover. (Note that currently l2ald runs on a Routing Engine only when it is a master). Also, during the time l2ald is DOWN, ARP packets received from an ICCP peer are dropped. ARP retry takes care of this situation. This is the case with Routing Engine switchover, too.

  • If ICCP restarts, l2ald does not identify that a MAC address was learned from a peer and, if the MAC address was learned only from the peer, that MAC address is deleted, and the packets destined to this MAC address are flooded.

Inter-Chassis Control Protocol

Inter-Chassis Control Protocol (ICCP) is used to synchronize configurations, states, and data.

ICCP supports the following types of state information:

  • MC-LAG members and their operational states

  • Single-homed members and their operational states

ICCP supports the following application database synchronization parameters:

  • MAC addresses learned and to be aged

  • ARP information learned over IRB

Inter-Chassis Control Protocol Message

ICCP messages and attribute-value pairs (AVPs) are used for synchronizing MAC address and ARP information.

IGMP Snooping in MC-LAG Active-Active Mode

IGMP snooping in MC-LAG active-active mode is supported on MX240 routers, MX480 routers, MX960 routers and QFX Series switches.

The following topics are included:

IGMP Snooping in MC-LAG Active-Active Mode Functionality

Multichassis link aggregation group (MC-LAG) active-active mode and IGMP snooping in active-standby mode are supported. MC-LAG allows one device to form a logical LAG interface with two or more network devices. MC-LAG provides additional benefits including node level redundancy, multihoming, and a loop-free Layer 2 network without running Spanning Tree Protocol (STP). The following features are supported:

  • State synchronization between peers for IGMP snooping in a bridge domain with only Layer 2 interfaces

  • Qualified learning

  • Router-facing multichassis links

The following enhancements to active-active bridging and Virtual Router Redundancy Protocol (VRRP) over integrated routing and bridging (IRB) are supported:

  • MC-LAG support for IGMP snooping in a pure Layer 2 switch

  • MC-LAG support for IGMP snooping in bridge domains doing qualified learning

  • Support for MC-Links being router-facing interfaces

The following functions are not supported:

  • MC-LAG for VPLS instances

  • MC-Links trunk ports

  • Proxy mode for active-active

  • Adding interchassis links to outgoing interfaces on an as needed basis

    Interchassis links can be added to the outgoing interface list as router-facing interfaces.

Typically Supported Network Topology for IGMP Snooping with MC-LAG Active-Active Bridging

Figure 10 depicts a typical network topology over which IGMP snooping with MC-LAG active-active bridging is supported.

Figure 10: Typical Network Over Which Active-Active Is Supported
Typical
Network Over Which Active-Active Is Supported

Interfaces I3 and I4 are single-homed interfaces. The multichassis links ae0.0 and ae0.1 belong to the same bridge domain in both the chassis. Interfaces I3, ae0.0, and ae0.1 are in the same bridge domain in the secondary active (S-A) router. Interfaces I4, ae0.0, and ae0.1 are in the same bridge domain in the primary active (P-A) router. Interfaces I3, I4, ae0.0, and ae0.1 are in the same learning domain as is the interchassis link (ICL) connecting the two chassis.

The primary active router is the chassis in which integrated routing and bridging has become PIM-DR. The secondary active router is the chassis in which integrated routing and bridging is not PIM-DR. Router P-A is the chassis responsible for pulling traffic from the IP core. Hence, PIM-DR election is used to avoid duplication of data traffic.

Learning domains are described in Qualified Learning.

For the IGMP speakers (hosts and routers) in the learning domain, P-A and S-A together should appear as one device with interfaces I4, I3, ae0.0, and ae0.1.

No duplicate control packets should be sent on multichassis links, meaning the control packet should be sent through only one link.

Control Plane State Updates Triggered by Packets Received on Remote Chassis

Following are the control plane state updates that are triggered by the packets received on remote chassis:

  • The membership state in Layer 3 multicast routing is updated as a result of reports learned on remote legs of multichassis links and S-links attached to the remote chassis.

  • The membership state and routing entry in snooping are updated when reports are received on the remote legs of a multichassis link.

Note
  • When reports are received on S-links attached to the remote chassis, the membership state or routing entry in snooping is not updated.

  • When synchronizing multicast snooping state between PE routers, timers, such as the Group Membership Timeout timer, are not synchronized. When the synch notification is received, the remote PE router receiving the notification starts or restarts the relevant timer.

  • The list of <s,g>s for which the state is maintained is the same in both the chassis under snooping as long as the outgoing interface lists involve only multichassis links.

Data Forwarding

This discussion assumes integrated routing and bridging on Router P-A is the PIM-DR. It pulls the traffic from sources in the core. Traffic might also come on Layer 2 interfaces in the bridge domain. For hosts directly connected to the P-A chassis, there is no change in the way data is delivered.

For delivering traffic to hosts connected to S-A (which is the non-DR) on the single-homed link like I3, we rely on the interchassis link. The traffic that hits P-A is sent over ICL to S-A to be delivered to the links that have reported interests in s,g and the links that are router-facing.

When the ae0 leg in P-A goes down, the hosts connected to the multichassis link receive traffic through ICL. In S-A, traffic received on ICL is sent to multichassis links in the outgoing interface list for which the ae counterpart in P-A is down.

Pure Layer 2 Topology Without Integrated Routing and Bridging

Figure 11 shows that the chassis connecting to the PIM-DR is the primary active (P-A) router and the other is the secondary active (S-A) router.

Figure 11: Layer 2 Configuration Without Integrated Routing and Bridging
Layer 2 Configuration
Without Integrated Routing and Bridging

Qualified Learning

In this topology, interfaces I1, I2, I3, I4, I5, I6, I7, I8, I9, and I10 are single-homed interfaces. The multichassis links ae0.0 and ae0.1 belong to the same bridge domain in both the chassis. Interfaces I10, I1,I7,I3,I5, ae0.0 and ae0.1 are in same bridge domain, bd1 in P-A. Interfaces I9, I2, I8, I4, I6, ae0.0, and ae0.1 are in same bridge domain, bd1 in S-A.

This discussion assumes the following configuration:

  • In P-A and S-A, qualified learning is ON in bd1.

  • Interfaces I1, I2, I3, ae0.0, and I4 belong to vlan1, learning domain ld1.

  • Interfaces I7, I8, I5, ae0.1, and I6 belong to vlan2, learning domain ld2.

  • Interfaces I9 and I10 belong to vlan3, learning domain ld3.

For the IGMP speakers (hosts and routers) in the same learning domain ld1, P-A and S-A linked should appear to be one switch.

For the IGMP speakers (hosts and routers) in the same learning domain ld2, P-A and S-A linked should appear to be one switch.

Since there are no multichassis links in learning domain ld3, for the IGMP speakers (hosts and routers) in learning domain ld3, P-A and S-A will not appear to be one switch.

This discussion assumes interchassis link ICL1 corresponds to learning domain ld1 and interchassis link ICL2 corresponds to learning domain ld2.

Control packet flow is supported, with the exception of passing information to IRB.

Data Forwarding with Qualified Learning

This discussion assumes one learning domain (LD), ld1, and further assumes that interface I1 on Router P-A is connected to the PIM-DR in the learning domain and pulls the traffic from sources in the core.

For delivering traffic to hosts connected to Router S-A (which is the non-DR) on the single-homed link like I2, I4 (belonging to ld1), we rely on ICL1. The traffic that hits Router P-A on interface I1 is sent over interchassis link ICL1 to Router S-A to be delivered to the links that have reported interests in s,g or the links that are router-facing in learning domain ld1.

When the interface ae0 leg in Router P-A goes down, the hosts connected to the multichassis link receive traffic from interface I1 using the interchassis link ICL1. In Router S-A, traffic received on interchassis link ICL1 is sent to multichassis links in the outgoing interface list for which the aggregated Ethernet counterpart in Router P-A is down.

It is further assumed that interface I9 in Router S-A belongs to the learning domain ld3 with interests in s,g, and that interface I10 in learning domain ld3 in Router P-A receives traffic for s,g. Interface I9 does not receive data in this topology because there are no multichassis links (in a-a mode) and hence no interchassis link in learning domain ld3.

Static Groups on Single-Homed Interfaces

For multichassis links, the static group configuration should exist on both legs, and synchronization with the other chassis is not required.

Synchronization of the static groups on single-homed interfaces between the chassis is not supported. However, the addition of logical interfaces to the default outgoing interface list supports traffic delivery to the interface within a static configuration.

Router-Facing Interfaces as Multichassis Links

IGMP queries could arrive on either leg of the multichassis links, but in both peers, the multichassis link should be considered as router-facing.

Reports should exit only once from the multichassis link, that is, from only one leg.

The following MC-LAG support for IGMP snooping in IRB is provided:

  • Non-proxy snooping

  • Logical interfaces must be outgoing interfaces for all routes including the default route

  • IGMP snooping in a pure Layer 2 switch

  • IGMP snooping in bridge domains doing qualified learning

  • Router-facing interface MC-Links

The following features are not supported:

  • Proxy mode for active-active

  • MC-LAG support for VPLS instances

  • Trunk ports as multichassis links

  • Adding logical interfaces to outgoing interfaces on an as need basis.

    However, logical interfaces are always added as a router-facing interface to the outgoing interface list.

Example: Configuring IGMP Snooping in MC-LAG Active-Active Mode

This example shows how to configure Internet Group Management Protocol (IGMP) snooping for uninterrupted traffic flow with a multichassis link aggregation group (MC-LAG) in an active-active scenario.

Requirements

This example uses the following hardware and software components:

Note

This example also applies to QFX10002 and QFX10008 switches.

  • Four Juniper Networks MX Series routers

  • Junos OS Release 11.2 or later running on all four routers

Before you begin, make sure that Protocol Independent Multicast (PIM) and Internet Group Management Protocol (IGMP) are running on all interfaces that will receive multicast packets. IGMP is automatically enabled on all IPv4 interfaces on which you configure PIM.

Overview

When links are aggregated, the links can be treated as if they were a single link. Link aggregation increases bandwidth, provides graceful degradation as failure occurs, and increases availability. MC-LAG provides redundant Layer 2 access connectivity at the node level. This enables two or more systems to share a common LAG endpoint. The multiple endpoints present a single logical chassis to the start point, and the start node does not need to be aware that MC-LAG is being used.

In this example, the CE router is not aware that its aggregated Ethernet links are connected to two separate PE devices. The two PE devices each have a LAG connected to the CE device. The configured mode is active-active, meaning that both PE routers’ LAG ports are active and carrying traffic at the same time.

Note

The other possible mode is active-standby, in which one of the router’s ports only becomes active when failure is detected in the active links. In active-standby mode, the PE routers perform an election to determine the active and standby routers.

From the perspective of the CE device, all four ports belonging to a LAG are connected to a single service provider device. Because the configured mode is active-active, all four ports are active, and the CE device load-balances the traffic to the peering PE devices. On the PE routers, a regular LAG is configured facing the CE device.

Inter-Chassis Control Protocol (ICCP) messages are sent between the two PE devices. These messages exchange MC-LAG configuration parameters and ensure that both chassis use the correct Link Aggregation Control Protocol (LACP) parameters when talking to the CE device.

The interchassis link-protection link (ICL) provides redundancy when a link failure occurs on one of the active links. The ICL-PL between the MC-LAG peering devices relays traffic that would otherwise be dropped due to a link failure.

Topology Diagram

Figure 12 shows the topology used in this example.

Figure 12: IGMP Snooping in MC-LAG Active-Active Mode on MX Series Routers
 IGMP
Snooping in MC-LAG Active-Active Mode on MX Series Routers

Configuring the PE Routers

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them in a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

Router PE1

Router PE2

Configuring the PE1 Router

Step-by-Step Procedure

The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode.

To configure Router PE1:

  1. Specify the number of aggregated Ethernet interfaces to be created.

  2. Specify the members to be included within the aggregated Ethernet bundles.

  3. Configure the interfaces that connect to multicast senders or receivers, the ICL interfaces, and the ICCP interfaces.

  4. Configure parameters on the aggregated Ethernet bundles.

  5. Configure LACP on the aggregated Ethernet bundles.

  6. Configure the MC-LAG interfaces.

    The multichassis aggregated Ethernet identification number (mc-ae-id) specifies which link aggregation group the aggregated Ethernet interface belongs to. The ae0 interfaces on Router PE1 and Router PE2 are configured with mc-ae-id 5. The ae1 interfaces on Router PE1 and Router PE2 are configured with mc-ae-id 10.

    The redundancy-group 10 statement is used by ICCP to associate multiple chassis that perform similar redundancy functions and to establish a communication channel so that applications on peering chassis can send messages to each other. The ae0 and ae1 interfaces on Router PE1 and Router PE2 are configured with the same redundancy group, redundancy-group 10.

    The chassis-id statement is used by LACP for calculating the port number of the MC-LAG's physical member links. Router PE1 uses chassid-id 1 to identify both its ae0 and ae1 interfaces. Router PE2 uses chassis-id 0 to identify both its ae0 and ae1 interfaces.

    The mode statement indicates whether an MC-LAG is in active-standby mode or active-active mode. Chassis that are in the same group must be in the same mode.

  7. Configure a domain that includes the set of logical ports.

    The ports within a bridge domain share the same flooding or broadcast characteristics in order to perform Layer 2 bridging.

    The bridge-level service-id statement is required to link related bridge domains across peers (in this case Router PE1 and Router PE2), and should be configured with the same value.

  8. At the global level and also in the bridge domain, replicate IGMP join and leave messages from the active link to the standby link of a dual-link MC-LAG interface, to enable faster recovery of membership information after failover.

  9. (Optional) Suppress MC-LAG reports to optimize the syncing of the ICCP messages. By default, every IGMP packet received on the MC-AE interface is replicated to the peer. If multiple hosts behind the CE router send reports for the same group, all the packets are synced even though only a single report is used for building the IGMP snooping state on the peer. Also all subsequent refreshes sent in response to the IGMP queries are also synced to this peer. This requires significant CPU cycles on both peers which send and receive these reports over ICCP. Starting with Junos OS Release 16.1, you can configure the suppress-report statement at the [edit multicast-snooping-options multichassis-lag-replicate-state] hierarchy level to optimize the syncing of the ICCP messages.

    Optimizing the syncing of ICCP messages ensures that the message exchanges using ICCP between the peers is more efficient. This also improves scaling by ensuring that the membership state is present only at the receiving PE.

    Note
    • Because the IGMP reports/leaves sent between the MC-LAG peers are suppressed, IGMP snooping statistics will not be the same on both peers. Total statistics will be the sum of the IGMP reports received on both MCLAG peers.

    • When MC-LAG reports are suppressed, the MCSNOOPD client application will not receive the source IP address (host information).

  10. Configure multicast snooping for the MC-LAG interfaces.

    Note

    Starting with Junos OS Release 16.1, you can selectively add ICL to preserve ICL bandwidth. To do this, you must not configure the ICL as an multicast-router-interface as specified in this step. Instead, you must configure the enhanced-ip statement.

    When you configure to selectively add ICL, control packets are directly sent from PFE to RPD. Therefore, if an IRB interface is attached to a bridge-domain, the proxy functionality in the L2 domain will not be effective because MCSNOOPD only proxies to external routers connected to the physical interfaces. In such scenarios, you can enable proxy to IRB. To do this, configure the irb statement at the [edit protocols igmp-snooping proxy] hierarchy level.

  11. Configure ICCP parameters.

  12. Configure the service ID at the global level.

    You must configure the same unique network-wide configuration for a service in the set of PE routers providing the service. This service ID is required if the multichassis aggregated Ethernet interfaces are part of a bridge domain.

Results

From configuration mode, confirm your configuration by entering the show bridge-domains, show chassis, show interfaces, show multicast-snooping-options, show protocols, and show switch-options commands. If the output does not display the intended configuration, repeat the instructions in this example to correct the configuration.

If you are done configuring the device, enter commit from configuration mode.

Repeat the procedure for Router PE2, using the appropriate interface names and addresses.

Configuring the CE Device

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them in a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

Device CE

Configuring the CE Device

Step-by-Step Procedure

The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode.

To configure the CE device:

  1. Specify the number of aggregated Ethernet interfaces to be created.

  2. Specify the members to be included within the aggregated Ethernet bundle.

  3. Configure an interface that connects to multicast senders or receivers.

  4. Configure parameters on the aggregated Ethernet bundle.

  5. Configure LACP on the aggregated Ethernet bundle.

    The active statement initiates transmission of LACP packets.

    For the system-priority statement, a smaller value indicates a higher priority. The device with the lower system priority value determines which links between LACP partner devices are active and which are in standby mode for each LACP group. The device on the controlling end of the link uses port priorities to determine which ports are bundled into the aggregated bundle and which ports are put in standby mode. Port priorities on the other device (the noncontrolling end of the link) are ignored.

  6. Configure a domain that includes the set of logical ports.

    The ports within a bridge domain share the same flooding or broadcast characteristics in order to perform Layer 2 bridging.

Results

From configuration mode, confirm your configuration by entering the show bridge-domains, show chassis, and show interfaces commands. If the output does not display the intended configuration, repeat the instructions in this example to correct the configuration.

If you are done configuring the device, enter commit from configuration mode.

Configuring the Provider Router

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them in a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

Router P

Configuring the P Router

Step-by-Step Procedure

The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode .

To configure Router P:

  1. Specify the number of aggregated Ethernet interfaces to be created.

  2. Specify the members to be included within the aggregated Ethernet bundle.

  3. Configure an interface that connects to multicast senders or receivers.

  4. Configure parameters on the aggregated Ethernet bundle.

  5. Configure LACP on the aggregated Ethernet bundle.

  6. Configure a domain that includes the set of logical ports.

Results

From configuration mode, confirm your configuration by entering the show bridge-domains, show chassis, and show interfaces commands. If the output does not display the intended configuration, repeat the instructions in this example to correct the configuration.

If you are done configuring the device, enter commit from configuration mode.

Verification

Verify that the configuration is working properly by running the following commands:

There are two methods for enabling Layer 3 multicast functionality across a multichassis link aggregation group (MC-LAG). You can choose either to configure Virtual Router Redundancy Protocol (VRRP) over the integrated routing and bridging (IRB) interface or to synchronize the MAC addresses for the Layer 3 interfaces of the switches participating in the MC-LAG. This provides redundancy and load balancing between the two MC-LAG peers. The procedure to configure VRRP for use in a Layer 3 multicast MC-LAG is included in this example.

Requirements

This example uses the following hardware and software components:

  • Two EX9200 switches

  • Junos OS Release 13.2R1 or later

Before you configure an MC-LAG for Layer 3 multicast using VRRP, be sure that you understand how to:

Overview

In this example, you configure two MC-LAGs between two switches, consisting of two aggregated Ethernet interfaces (ae1 and ae2). To support the MC-LAG, you create a third aggregated Ethernet interface (ae0) for the interchassis link (ICL). You also configure a multichassis protection link for the ICL, Inter-Chassis Control Protocol (ICCP) for the peers hosting the MC-LAG, and Layer 3 connectivity between MC-LAG peers.

Note

Layer 3 connectivity is required for ICCP.

To complete the MC-LAG configuration, enable VRRP by completing the following tasks for each MC-LAG:

  1. Create an IRB interface.

  2. Create a VRRP group and assign a virtual IP address that is shared between each switch in the VRRP group.

  3. Enable a member of a VRRP group to accept all packets destined for the virtual IP address if it is the master in the VRRP group.

  4. Configure Layer 3 connectivity between the VRRP groups.

Topology

The topology used in this example consists of two switches hosting two MC-LAGs—ae1 and ae2. The two switches are connected to a multicast source (Server 1) over MC-LAG ae1, and a multicast receiver (Server 2) over MC-LAG ae2. Figure 13 shows the topology for this example.

Figure 13: Configuring Two MC-LAGs Between Switch A and Switch B
Configuring Two MC-LAGs Between
Switch A and Switch B

Table 1 details the topology used in this configuration example.

Table 1: Components of the Topology for Configuring Two MC-LAGs Between Switch A and Switch B

HostnameBase HardwareLink Aggregation Groups

Switch A

Switch B

EX9200 switch

  • ae0 is configured as an aggregated Ethernet interface, and is used as an ICL. The following two interfaces are part of ae0:

    xe-0/0/2 and xe-0/0/3 on Switch A and

    xe-0/0/2 and xe-0/0/3 on Switch B.

  • ae1 is configured as an MC-LAG for the multicast source (Server 1), and the following two interfaces are part of ae1:

    xe-0/0/4 on Switch A and

    xe-0/0/6 on Switch B.

  • ae2 is configured as an MC-LAG for the multicast receiver (Server 2), and the following two interfaces are part of ae2:

    xe-0/0/5 on Switch A and

    xe-0/0/7 on Switch B.

Configuration

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them in a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

Switch A

Switch B

Configuring MC-LAG for Layer 3 Multicast Using VRRP on Two Switches

Step-by-Step Procedure

The following procedure requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode.

To enable a multichassis protection link between MC-LAG peers:

  1. Configure the number of LAGs on both Switch A and Switch B.

    Switch A and Switch B:

  2. Add member interfaces to the aggregated Ethernet interfaces on both Switch A and Switch B.

    Switch A and Switch B:

    Switch A:

    Switch B:

  3. Configure ae0 as the trunk interface between Switch A and Switch B.

    Switch A and Switch B:

  4. Configure ae0 as the multichassis protection link between Switch A and Switch B.

    Switch A:

    Switch B:

Step-by-Step Procedure

To enable ICCP:

  1. Configure the local IP address to be in the ICCP connection on Switch A and Switch B.

    Switch A:

    Switch B:

  2. Configure the peer IP address, minimum receive interval, and minimum transmit interval for a Bidirectional Forwarding Detection (BFD) session for ICCP on Switch A and Switch B.Note

    Configuring the minimum receive interval is required to enable BFD. We recommend a minimum receive interval value of 60 seconds.

    Switch A:

    Switch B:

  3. (Optional) Configure the time during which an ICCP connection must be established between MC-LAG peers on Switch A and Switch B.Note

    On QFX and EX Series switches, the default session establishment hold time is 300 seconds. However, the session establishment time must be at least 100 seconds higher than the init delay time. You can optionally update the session establishment time to be 340 seconds and the init delay time to be 240 seconds.

    Switch A:

    Switch B:

  4. (Optional) Configure the backup-liveness-detection statement on the management interface (fxp0) only.

    We recommend that you configure the backup liveness detection feature to implement faster failover of data traffic during an MC-LAG peer reboot.

    Note

    The backup-liveness-detection statement is supported starting in Junos OS Release 13.2R1.

    Switch A:

    Switch B:

  5. Configure Layer 3 connectivity between the MC-LAG peers on both Switch A and Switch B.

    Switch A and Switch B:

    Switch A:

    Switch B:

Step-by-Step Procedure

To enable the ae1 and ae2 MC-LAG interfaces:

  1. Enable LACP on the MC-LAG interfaces on Switch A and Switch B.Note

    At least one end needs to be active. The other end can be either active or passive.

  2. Specify the same multichassis aggregated Ethernet identification number for each MC-LAG peer on Switch A and Switch B.
  3. Specify a unique chassis ID for the MC-LAG on the MC-LAG peers on Switch A and Switch B.

    Switch A:

    Switch B:

  4. Specify the operating mode of the MC-LAGs on both Switch A and Switch B.
  5. Specify the status control for the MC-LAGs on Switch A and Switch B.Note

    You must configure status control on both Switch A and Switch B that host the MC-LAGs. If one peer is in active mode, the other must be in standby mode.

    Switch A:

    Switch B:

    Note

    If you configure both nodes as prefer-status-control-active, you must also configure ICCP peering using the peer’s loopback address to make sure that the ICCP session does not go down because of physical link failures. Additionally, you must configure backup liveness detection on both of the MC-LAG nodes.

  6. To minimize traffic loss, specify the number of seconds by which to delay bringing the multichassis aggregated Ethernet interface back to the up state when you reboot Switch A or Switch B.
  7. Specify the same LACP system ID for each MC-LAG on Switch A and Switch B.
  8. Specify the same LACP administration key on both Switch A and Switch B.
  9. Enable a VLAN for each MC-LAG on Switch A and Switch B.
  10. Configure ae1 and ae2 as trunk interfaces between Switch A and Switch B.

Step-by-Step Procedure

To enable VRRP on the MC-LAGs:

  1. Create an integrated routing and bridging (IRB) interface for each MC-LAG, assign a virtual IP address that is shared between each switch in the VRRP groups, and assign an individual IP address for each switch in the VRRP groups.

    Switch A:

    Switch B:

  2. Assign the priority for each switch in the VRRP groups.Note

    The switch configured with the highest priority is the master.

    Switch A:

    Switch B:

  3. Enable the switch to accept all packets destined for the virtual IP address if it is the master in a VRRP group.

    Switch A:

    Switch B:

  4. Configure Layer 3 connectivity between Switch A and Switch B.

Step-by-Step Procedure

To enable IGMP snooping:

  • Enable IGMP snooping for all VLANs on Switch A and Switch B.
    Note

    You must configure the multichassis-lag-replicate-state statement for IGMP snooping to work properly in an MC-LAG environment.

Step-by-Step Procedure

To configure OSPF as the Layer 3 protocol:

  1. Configure an OSPF area on Switch A and Switch B.
  2. Assign the VLAN interfaces for the MC-LAGs as interfaces to the OSPF area on Switch A and Switch B.
  3. Configure the minimum receive interval, minimum transmit interval, and transmit interval threshold for a Bidirectional Forwarding Detection (BFD) session for the OSPF interfaces on Switch A and Switch B.

Step-by-Step Procedure

To configure Protocol Independent Multicast (PIM) as the multicast protocol:

  1. Configure a static rendezvous point (RP) address on Switch A and Switch B.
  2. Configure the address ranges of the multicast groups for which Switch A and Switch B can be an RP.
  3. Configure the priority of each PIM interface for being selected as the designated router.

    An interface with a higher priority value has a higher probability of being selected as the designated router.

    Note

    Configure the IP address on the active MC-LAG peer with a high IP address or a high designated router priority.

    Switch A:

    Switch B:

Results

From configuration mode on Switch A, confirm your configuration by entering the show chassis, show interfaces, show multi-chassis, show protocols, and show vlans commands. If the output does not display the required configuration, repeat the instructions in this example to correct the configuration.

Switch A

Switch B

Verification

Verify that the configuration is working properly.

Confirm That Switch A Is the Master Designated Router

Purpose

Verify that Switch A is the master designated router (DR).

Action

user@switch> show pim interfaces

Meaning

The DDR-DR state of the VLAN interfaces in the output shows that Switch A is the master designated router.

Verifying That Switch B Is the Backup Designated Router

Purpose

Confirm that Switch B is the backup designated router.

Action

user@switch> show pim interfaces

Meaning

The DDR-BDR state of the VLAN interfaces in the output shows that Switch B is the backup designated router.

There are two methods for enabling Layer 3 unicast functionality across a multichassis link aggregation group (MC-LAG). You can choose either to synchronize the MAC addresses for the Layer 3 interfaces of the switches participating in the MC-LAG, or you can configure Virtual Router Redundancy Protocol (VRRP), but you cannot configure both at the same time. Because RVI interfaces share the same MAC address, if you enable MAC address synchronization, packets received on an MC-LAG peer with a destination MAC address that is the same as that of the peer’s IRB MAC address will not be forwarded. The procedure to configure MAC address synchronization is included in this example. For more information on configuring VRRP for use in a Layer 3 unicast MC-LAG, see Example: Configuring Multichassis Link Aggregation for Layer 3 Unicast Using VRRP.

Requirements

This example uses the following hardware and software components:

  • Junos OS Release 12.3 or later for the QFX Series

    • Revalidated with Junos OS Release 17.3R1 on QFX5100 and QFX10000 switches.

  • Two QFX3500, QFX3600, EX4600, QFX5100, or QFX10000 switches

Before you configure an MC-LAG for Layer 3 unicast, be sure that you understand how to:

Overview

In this example, you configure an MC-LAG across two switches by including interfaces from both switches in an aggregated Ethernet interface (ae1). To support the MC-LAG, you create a second aggregated Ethernet interface (ae0) for the interchassis link-protection link (ICL-PL). You also configure a multichassis protection link for the ICL-PL, Inter-Chassis Control Protocol (ICCP) for the peers hosting the MC-LAG, and Layer 3 connectivity between MC-LAG peers.

Note

Layer 3 connectivity is required for ICCP.

Note

On QFX5100 and QFX10000 switches, if you try to configure both VRRP over IRB and MAC synchronization, you will receive a commit error.

To complete the configuration, configure MAC address synchronization between the peers and specify the same IP address on both Layer 3 interface members (also known as the routed VLAN interface [RVI] or the integrated routing and bridging (IRB) interface) in the MC-LAG VLAN.

Topology

The topology used in this example consists of two switches hosting an MC-LAG. The two switches are connected to a server. Figure 14 shows the topology of this example.

Figure 14: Configuring a Multichassis LAG Between Switch A and Switch B
Configuring a Multichassis LAG
Between Switch A and Switch B

Table 2 details the topology used in this configuration example.

Table 2: Components of the Topology for Configuring a Multichassis LAG Between Two Switches

HostnameBase HardwareMultichassis Link Aggregation Group

Switch A

Switch B

QFX3500, QFX3600, EX4600, QFX5100, or QFX10000 switch

QFX3500, QFX3600, EX4600, QFX5100, or QFX10000 switch

ae0 is configured as an aggregated Ethernet interface, and is used as an ICL-PL. The following interfaces are part of ae0: xe-0/0/12and xe-0/0/13 on Switch A and

xe-0/0/12 and xe-0/0/13 on Switch B. These interfaces are included in VLAN v500.

ae1 is configured as an MC-LAG, and the following two interfaces are part of ae1:

xe-0/0/44 on Switch A and

xe-0/0/46 on Switch B. These interfaces are included in VLAN v100.

Configuration

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them in a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

Switch A

Switch B

Configuring MC-LAG on Two Switches

Step-by-Step Procedure

To enable multichassis protection link between MC-LAG peers:

The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode.

To enable multichassis protection link between MC-LAG peers:

  1. Configure the number of LAGs on both Switch A and Switch B.

    Switch A and Switch B:

  2. Add member interfaces to the aggregated Ethernet interfaces on both Switch A and Switch B.

    Switch A and Switch B:

    Switch B

  3. Configure a trunk interface between Switch A and Switch B.
  4. Configure a multichassis protection link between Switch A and Switch B.

Step-by-Step Procedure

To enable ICCP:

The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode .

  1. Configure the local IP address to be in the ICCP connection on Switch A and Switch B.
    [edit protocols]

    user@switch# set iccp local-ip-addr 10.3.3.2

    Switch B:

    [edit protocols]

    user@switch# set iccp local-ip-addr 10.3.3.1
  2. Configure the peer IP address and minimum receive interval for a Bidirectional Forwarding Detection (BFD) session for ICCP on Switch A and Switch B.

    Switch A:

    Switch B:

  3. Configure the peer IP address and minimum transmit interval for a BFD session for ICCP on Switch A and Switch B.Note

    Configure at least 1000 milliseconds as the transmit interval minimum interval.

    Switch A:

    Switch B:

  4. (Optional) Configure the time during which an ICCP connection must succeed between MC-LAG peers on Switch A and Switch B.Note

    On QFX and EX Series switches, the default session establishment hold time is 300 seconds. However, the session establishment time must be at least 100 seconds higher than the init delay time. You can optionally update the session establishment time to be 340 seconds and the init delay time to be 240 seconds.

    Switch A:

    Switch B:

  5. (Optional) Configure the backup IP address to be used for backup liveness detection on both Switch A and Switch B.Note

    By default, backup liveness detection is not enabled. Configuring a backup IP address helps achieve sub-second traffic loss during an MC-LAG peer reboot.

    Switch A:

    Switch B:

  6. Configure Layer 3 connectivity across the ae0 ICCP link by adding a Layer 3 interface on both Switch A and Switch B.

Step-by-Step Procedure

To enable the MC-LAG interface:

The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode .

  1. Enable LACP on the MC-LAG interface on Switch A and Switch B.Note

    At least one end needs to be active. The other end can be either active or passive.

    [edit interfaces]

    user@switch# set ae1 aggregated-ether-options lacp active
  2. Specify the same multichassis aggregated Ethernet identification number on both MC-LAG peers on Switch A and Switch B.
    [edit interfaces]

    user@switch# set ae1 aggregated-ether-options mc-ae mc-ae-id 3
  3. Specify a unique chassis ID for the MC-LAG on the MC-LAG peers on Switch A and Switch B.

    Switch A:

    [edit interfaces]

    user@switch# set ae1 aggregated-ether-options mc-ae chassis-id 0

    Switch B:

    [edit interfaces]

    user@switch# set ae1 aggregated-ether-options mc-ae chassis-id 1
  4. Specify the operating mode of the MC-LAG on both Switch A and Switch B.Note

    Only active-active mode is supported at this time.

    [edit interfaces]

    user@switch# set ae1 aggregated-ether-options mc-ae mode active-active
  5. Specify the status control for MC-LAG on Switch A and Switch B.Note

    You must configure status control on both Switch A and Switch B hosting the MC-LAG. If one peer is in active mode, the other must be in standby mode.

    Switch A:

    [edit interfaces]

    user@switch# set ae1 aggregated-ether-options mc-ae status-control active

    Switch B:

    [edit interfaces]

    user@switch# set ae1 aggregated-ether-options mc-ae status-control standby
    Note

    If you configure both nodes as prefer-status-control-active, you must also configure ICCP peering using the peer’s loopback address to make sure that the ICCP session does not go down because of physical link failures. Additionally, you must configure backup liveness detection on both of the MC-LAG nodes.

  6. Specify the number of seconds by which the bring-up of the MC-AE interface should be deferred after you reboot Switch A and Switch B.Note

    On QFX and EX Series switches, the default session establishment hold time is 300 seconds. However, the session establishment time must be at least 100 seconds higher than the init delay time. You can optionally update the session establishment time to be 340 seconds and the init delay time to be 240 seconds.

    [edit interfaces]

    user@switch# set ae1 aggregated-ether-options mc-ae init-delay-time 240
  7. Specify the same LACP system ID for the MC-LAG on Switch A and Switch B.
    [edit interfaces]

    user@switch# set ae1 aggregated-ether-options lacp system-ID 00:01:02:03:04:05
  8. Specify the same LACP administration key on both Switch A and Switch B.
    [edit interfaces]

    user@switch# set ae1 aggregated-ether-options lacp admin-key 3
  9. Enable a VLAN on the MC-LAG on Switch A and Switch B.
    [edit interfaces]

    user@switch# set ae1 unit 0 family ethernet-switching interface-mode trunk
    [edit]

    user@switch# set vlans v100 vlan-id 100
    [edit interfaces]

    user@switch# set ae1 unit 0 family ethernet-switching vlan members v100
  10. Create a Layer 3 interface for the MC-LAG VLAN and assign the same IP address on both Switch A and Switch B.
    [edit]

    user@switch# set vlans v100 l3-interface irb.100
    [edit interfaces]

    user@switch# set irb unit 100 family inet address 10.1.1.10/24
  11. Configure MAC address synchronization in the MC-LAG VLAN on both Switch A and Switch B.
    [edit]

    user@switch# set vlans v100 mcae-mac-synchronize

Step-by-Step Procedure

To enable RSTP:

The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode .

  1. Enable RSTP globally on all interfaces on Switch A and Switch B.
    [edit]

    user@switch# set protocols rstp interface all mode point-to-point
  2. Disable RSTP on the ICL-PL interfaces on Switch A and Switch B:
    [edit]

    user@switch# set protocols rstp interface ae0.0 disable
  3. Configure the MC-LAG interfaces as edge ports on Switch A and Switch B.Note

    The ae1 interface is a downstream interface. This is why RSTP and bpdu-block-on-edge need to be configured.

    [edit]

    user@switch# set protocols rstp interface ae1.0 edge
  4. Enable BPDU blocking on all interfaces except for the ICL-PL interfaces on Switch A and Switch B.Note

    The ae1 interface is a downstream interface. This is why RSTP and bpdu-block-on-edge need to be configured.

    [edit]

    user@switch# set protocols rstp bpdu-block-on-edge

Results

Display the results of the configuration on Switch A.

Display the results of the configuration on Switch B.

Verification

To verify that the MC-LAG group has been created and is working properly, perform these tasks:

Verifying That ICCP Is Working on Switch A

Purpose

Verify that ICCP is running on Switch A.

Action

[edit]

user@switch# show iccp

Meaning

This output shows that the TCP connection between the peers hosting the MC-LAG is up, liveness detection is up, and lacpd, and l2ald_iccpd_client applications are running.

Verifying That ICCP Is Working on Switch B

Purpose

Verify that ICCP is running on Switch B.

Action

show iccp

[edit]

user@switch# show iccp

Meaning

This output shows that the TCP connection between the peers hosting the MC-LAG is up, liveness detection is up, and lacpd, and l2ald_iccpd_client applications are running.

Verifying That LACP Is Active on Switch A

Purpose

Verify that LACP is active on Switch A.

Action

[edit]

user@switch# show lacp interfaces

Meaning

This output shows that Switch A is participating in LACP negotiation.

Verifying That LACP Is Active on Switch B

Purpose

Verify that LACP is active on Switch B

Action

[edit]

user@switch# show lacp interfaces

Meaning

This output shows that Switch B is participating in LACP negotiation.

Verifying That the MC-AE and ICL-PL Interfaces Are Up on Switch A

Purpose

Verify that the MC-AE and ICL-PL interfaces are up on Switch A.

Action

[edit]

user@switch# show interfaces mc-ae

Meaning

This output shows that the MC-AE interface on Switch A is up and active.

Verifying That the MC-AE and ICL-PL Interfaces Are Up on Switch B

Purpose

Verify that the MC-AE and ICL-PL interfaces are up on Switch B.

Action

[edit]

user@switch# show interfaces mc-ae

Meaning

This output shows that the MC-AE interface on Switch B is up and active.

Verifying MAC Address Synchronization on Switch A and Switch B

Purpose

Confirm that MAC address synchronization is working on both Switch A and Switch B.

Action

[edit]

user@switch# show ethernet-switching table vlan-id 100

Meaning

This output shows that the downstream server MAC address is visible. From the server-side, both of the MC-LAG peer MAC addresses are visible.

Verifying MAC Address Synchronization on Switch A and Switch B Using MC-LAG Consistency Check

Purpose

On the QFX10000 switch only, if you have MC-LAG consistency check enabled, you can verify that MAC address synchronization is working on both Switch A and Switch B. If you do not have MC-LAG consistency check enabled, issue the set multi-chassis mc-lag consistency-check command and then commit this change.

Action

[edit]

user@switch> show multi-chassis mc-lag configuration-consistency

Troubleshooting

Troubleshooting a LAG That Is Down

Problem

The show interfaces terse command shows that the MC-LAG is down

Solution

Check the following:

  • Verify that there is no configuration mismatch.

  • Verify that all member ports are up.

  • Verify that the MC-LAG is part of family Ethernet switching (Layer 2 LAG).

  • Verify that the MC-LAG member is connected to the correct MC-LAG member at the other end.

Note

Multichassis link aggregation (MC-LAG) is supported on QFX3500 and QFX3600 standalone switches running the original CLI, and QFX3500, QFX3600, QFX5100, EX4600, and QFX10000 standalone switches running Enhanced Layer 2 Software.

There are two methods for enabling Layer 3 unicast functionality across a multichassis link aggregation group (MC-LAG) to control traffic flow. You can choose either to synchronize the MAC addresses for the Layer 3 interfaces of the switches participating in the MC-LAG , or you can configure Virtual Router Redundancy Protocol (VRRP), but you cannot configure both at the same time. Because RVI interfaces share the same MAC address, if you enable MAC address synchronization, packets received on an MC-LAG peer with a destination MAC address that is the same as that of the peer’s IRB MAC address will not be forwarded. The procedure to configure VRRP for use in a Layer 3 unicast MC-LAG is included in this example. For more information about configuring MAC address synchronization, see Example: Configuring Multichassis Link Aggregation for Layer 3 Unicast using MAC Address Synchronization.

Requirements

This example uses the following hardware and software components:

  • Junos OS Release 12.3 or later for the QFX Series

  • Two QFX3500, QFX3600, EX4600, QFX5100, or QFX10000 switches

Before you configure an MC-LAG, be sure that you understand how to:

Overview

In this example, you configure an MC-LAG across two switches by including interfaces from both switches in an aggregated Ethernet interface (ae1). To support the MC-LAG, create a second aggregated Ethernet interface (ae0) for the interchassis control link-protection link (ICL-PL). Configure a multichassis protection link for the ICL-PL, the Inter-Chassis Control Protocol (ICCP) for the peers hosting the MC-LAG, and Layer 3 connectivity between MC-LAG peers.

Note

Layer 3 connectivity is required for ICCP.

Note

On QFX5100 and QFX10000 switches, if you try to configure both VRRP over IRB and MAC synchronization, you will receive a commit error.

To complete the configuration, enable VRRP by completing the following steps:

  1. Create a routed VLAN interface (RVI).

  2. Create a VRRP group and assign a virtual IP address that is shared between each switch in the VRRP group.

  3. Enable a member of a VRRP group to accept all packets destined for the virtual IP address if it is the master in the VRRP group.

  4. Configure Layer 3 connectivity between the VRRP groups.

Topology

The topology used in this example consists of two switches hosting MC-LAGs. The two switches are connected to a server. Figure 15 shows the topology of this example.

Figure 15: Configuring a Multichassis LAG Between Switch A and Switch B
Configuring a Multichassis LAG Between
Switch A and Switch B

Table 3 details the topology used in this configuration example.

Table 3: Components of the Topology for Configuring a Multichassis LAG Between Two Switches

HostnameBase HardwareMultichassis Link Aggregation Group

Switch A

Switch B

QFX3500, QFX3600, EX4600, QFX5100, QFX10000 switch

QFX3500, QFX3600, EX4600, QFX5100, QFX10000 switch

ae0 is configured as an aggregated Ethernet interface, and is used as an ICL-PL. The following interfaces are part of ae0: xe-0/0/12 and xe-0/0/13 on Switch A and

xe-0/0/12 and xe-0/0/13 on Switch B.

ae1 is configured as an MC-LAG, and the following two interfaces are part of ae1:

xe-0/0/44 on Switch A and

xe-0/0/46 on Switch B.

.

Configuration

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them in a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

Note

This example shows how to configure MC-LAG using both the original CLI and Enhanced Layer 2 Software (ELS).

In ELS, there are three statements and one additional statement that are different from the original CLI:

  • The port-mode statement in the [edit interfaces interface-name unit number family ethernet-switching] hierarchy is not supported. Use the interface-mode statement instead.

  • The vlan statement in the [edit interfaces interface-name] hierarchy is not supported. Use the irb statement instead.

  • The vlan.logical-interface-number option in the [edit vlans vlan-name l3-interface] option is not supported. Use the irb.logical-interface-number option instead.

  • The service-id statement in the [edit switch-options] hierarchy is required in the ELS CLI.

Switch A—Original CLI:

set chassis aggregated-devices ethernet device-count 2
set interfaces xe-0/0/12 ether-options 802.3ad ae0
set interfaces xe-0/0/13 ether-options 802.3ad ae0
set interfaces xe-0/0/44 ether-options 802.3ad ae1
set interfaces ae0 unit 0 family ethernet-switching port-mode trunk
set interfaces ae0 unit 0 family ethernet-switching vlan members v500 v100
set interfaces ae1 aggregated-ether-options lacp active
set interfaces ae1 aggregated-ether-options lacp system-id 00:01:02:03:04:05
set interfaces ae1 aggregated-ether-options lacp admin-key 3
set interfaces ae1 aggregated-ether-options mc-ae mc-ae-id 3
set interfaces ae1 aggregated-ether-options mc-ae chassis-id 0
set interfaces ae1 aggregated-ether-options mc-ae mode active-active
set interfaces ae1 aggregated-ether-options mc-ae status-control active
set interfaces ae1 aggregated-ether-options mc-ae init-delay-time 240
set interfaces ae1 unit 0 family ethernet-switching port-mode trunk
set interfaces ae1 unit 0 family ethernet-switching vlan members v100
set interfaces vlan unit 100 family inet address 10.1.1.11/8 vrrp-group 1 virtual-address 10.1.1.1
set interfaces vlan unit 100 family inet address 10.1.1.11/8 vrrp-group 1 priority 200
set interfaces vlan unit 100 family inet address 10.1.1.11/8 vrrp-group 1 accept-data
set interfaces vlan unit 500 family inet address 10.3.3.2/8
set vlans v100 vlan-id 100
set vlans v100 l3-interface vlan.100
set vlans v500 vlan-id 500
set vlans v500 l3-interface vlan.500
set protocols iccp local-ip-addr 10.3.3.2
set protocols iccp peer 10.3.3.1 session-establishment-hold-time 340
set protocols iccp peer 10.3.3.1 backup-liveness-detection backup-peer-ip 10.207.64.233
set protocols iccp peer 10.3.3.1 liveness-detection minimum-receive-interval 1000
set protocols iccp peer 10.3.3.1 liveness-detection transmit-interval minimum-interval 1000
set protocols rstp interface ae0 disable
set protocols rstp interface ae1 edge
set protocols rstp interface all mode point-to-point
set protocols rstp bpdu-block-on-edge
set multi-chassis multi-chassis-protection 10.3.3.1 interface ae0

Switch A—ELS

set chassis aggregated-devices ethernet device-count 2
set interfaces xe-0/0/12 ether-options 802.3ad ae0
set interfaces xe-0/0/13 ether-options 802.3ad ae0
set interfaces xe-0/0/44 ether-options 802.3ad ae1
set interfaces ae0 unit 0 family ethernet-switching interface-mode trunk
set interfaces ae0 unit 0 family ethernet-switching vlan members v500 v100
set interfaces ae1 aggregated-ether-options lacp active
set interfaces ae1 aggregated-ether-options lacp system-id 00:01:02:03:04:05
set interfaces ae1 aggregated-ether-options lacp admin-key 3
set interfaces ae1 aggregated-ether-options mc-ae mc-ae-id 3
set interfaces ae1 aggregated-ether-options mc-ae chassis-id 0
set interfaces ae1 aggregated-ether-options mc-ae mode active-active
set interfaces ae1 aggregated-ether-options mc-ae status-control active
set interfaces ae1 aggregated-ether-options mc-ae init-delay-time 240
set interfaces ae1 unit 0 family ethernet-switching interface-mode trunk
set interfaces ae1 unit 0 family ethernet-switching vlan members v100
set interfaces irb unit 100 family inet address 10.1.1.11/8 vrrp-group 1 virtual-address 10.1.1.1
set interfaces irb unit 100 family inet address 10.1.1.11/8 vrrp-group 1 priority 200
set interfaces irb unit 100 family inet address 10.1.1.11/8 vrrp-group 1 accept-data
set interfaces irb unit 500 family inet address 10.3.3.2/8
set vlans v100 vlan-id 100
set vlans v100 l3-interface irb.100
set vlans v500 vlan-id 500
set vlans v500 l3-interface irb.500
set protocols iccp local-ip-addr 10.3.3.2
set protocols iccp peer 10.3.3.1 session-establishment-hold-time 340
set protocols iccp peer 10.3.3.1 backup-liveness-detection backup-peer-ip 10.207.64.233
set protocols iccp peer 10.3.3.1 liveness-detection minimum-receive-interval 1000
set protocols iccp peer 10.3.3.1 liveness-detection transmit-interval minimum-interval 1000
set protocols rstp interface ae1 edge
set protocols rstp interface ae1 mode point-to-point
set protocols rstp bpdu-block-on-edge
set multi-chassis multi-chassis-protection 10.3.3.1 interface ae0
set switch-options service-id 10

Switch B—Original CLI:

set chassis aggregated-devices ethernet device-count 2
set interfaces xe-0/0/12 ether-options 802.3ad ae0
set interfaces xe-0/0/13 ether-options 802.3ad ae0
set interfaces xe-0/0/46 ether-options 802.3ad ae1
set interfaces ae0 unit 0 family ethernet-switching port-mode trunk
set interfaces ae0 unit 0 family ethernet-switching vlan members v500 v100
set interfaces ae1 aggregated-ether-options lacp active
set interfaces ae1 aggregated-ether-options lacp system-id 00:01:02:03:04:05
set interfaces ae1 aggregated-ether-options lacp admin-key 3
set interfaces ae1 aggregated-ether-options mc-ae mc-ae-id 3
set interfaces ae1 aggregated-ether-options mc-ae chassis-id 1
set interfaces ae1 aggregated-ether-options mc-ae mode active-active
set interfaces ae1 aggregated-ether-options mc-ae status-control standby
set interfaces ae1 aggregated-ether-options mc-ae init-delay-time 240
set interfaces ae1 unit 0 family ethernet-switching port-mode trunk
set interfaces ae1 unit 0 family ethernet-switching vlan members v100
set interfaces vlan unit 100 family inet address 10.1.1.10/8 vrrp-group 1 virtual-address 10.1.1.1
set interfaces vlan unit 100 family inet address 10.1.1.10/8 vrrp-group 1 priority 150
set interfaces vlan unit 100 family inet address 10.1.1.10/8 vrrp-group 1 accept-data
set interfaces vlan unit 500 family inet address 10.3.3.1/8
set vlans v100 vlan-id 100
set vlans v100 l3-interface vlan.100
set vlans v500 vlan-id 500
set vlans v500 l3-interface vlan.500
set protocols iccp local-ip-addr 10.3.3.1
set protocols iccp peer 10.3.3.2 session-establishment-hold-time 340
set protocols iccp peer 10.3.3.2 backup-liveness-detection backup-peer-ip 10.207.64.234
set protocols iccp peer 10.3.3.2 liveness-detection minimum-receive-interval 1000
set protocols iccp peer 10.3.3.2 liveness-detection transmit-interval minimum-interval 1000
set protocols rstp interface ae0 disable
set protocols rstp interface ae1 edge
set protocols rstp interface all mode point-to-point
set protocols rstp bpdu-block-on-edge
set multi-chassis multi-chassis-protection 10.3.3.2 interface ae0

Switch B—ELS:

set chassis aggregated-devices ethernet device-count 2
set interfaces xe-0/0/12 ether-options 802.3ad ae0
set interfaces xe-0/0/13 ether-options 802.3ad ae0
set interfaces xe-0/0/46 ether-options 802.3ad ae1
set interfaces ae0 unit 0 family ethernet-switching interface-mode trunk
set interfaces ae0 unit 0 family ethernet-switching vlan members v500 v100
set interfaces ae1 aggregated-ether-options lacp active
set interfaces ae1 aggregated-ether-options lacp system-id 00:01:02:03:04:05
set interfaces ae1 aggregated-ether-options lacp admin-key 3
set interfaces ae1 aggregated-ether-options mc-ae mc-ae-id 3
set interfaces ae1 aggregated-ether-options mc-ae chassis-id 1
set interfaces ae1 aggregated-ether-options mc-ae mode active-active
set interfaces ae1 aggregated-ether-options mc-ae status-control standby
set interfaces ae1 aggregated-ether-options mc-ae init-delay-time 240
set interfaces ae1 unit 0 family ethernet-switching interface-mode trunk
set interfaces ae1 unit 0 family ethernet-switching vlan members v100
set interfaces irb unit 100 family inet address 10.1.1.10/8 vrrp-group 1 virtual-address 10.1.1.1
set interfaces irb unit 100 family inet address 10.1.1.10/8 vrrp-group 1 priority 150
set interfaces irb unit 100 family inet address 10.1.1.10/8 vrrp-group 1 accept-data
set interfaces irb unit 500 family inet address 10.3.3.1/8
set vlans v100 vlan-id 100
set vlans v100 l3-interface irb.100
set vlans v500 vlan-id 500
set vlans v500 l3-interface irb.500
set protocols iccp local-ip-addr 10.3.3.1
set protocols iccp peer 10.3.3.2 session-establishment-hold-time 340
set protocols iccp peer 10.3.3.2 backup-liveness-detection backup-peer-ip 10.207.64.234
set protocols iccp peer 10.3.3.2 liveness-detection minimum-receive-interval 1000
set protocols iccp peer 10.3.3.2 liveness-detection transmit-interval minimum-interval 1000
set protocols rstp interface ae1 edge
set protocols rstp interface ae1 mode point-to-point
set protocols rstp bpdu-block-on-edge
set multi-chassis multi-chassis-protection 10.3.3.2 interface ae0
set switch-options service-id 10

Configuring MC-LAG on Two Switches

Step-by-Step Procedure

The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode.

To enable multichassis protection link between MC-LAG peers:

  1. Configure the number of LAGs on both Switch A and Switch B.
  2. Add member interfaces to the aggregated Ethernet interfaces on both Switch A and Switch B.

    Switch A and Switch B:

    Switch A:

    Switch B:

  3. Configure a trunk interface between Switch A and Switch B using the original CLI.
  4. Configure a trunk interface between Switch A and Switch B using ELS.
  5. Configure a multichassis protection link between Switch A and Switch B.

    Switch A:

    Switch B:

Step-by-Step Procedure

To enable ICCP:

  1. Configure the local IP address to be in the ICCP connection on Switch A and Switch B.

    Switch A:

    Switch B:

  2. Configure the peer IP address and minimum receive interval for a Bidirectional Forwarding Detection (BFD) session for ICCP on Switch A and Switch B.

    Switch A:

    Switch B:

  3. Configure the peer IP address and minimum transmit interval for a BFD session for ICCP on Switch A and Switch B.

    Switch A:

    Switch B:

  4. (Optional) Configure the time during which an ICCP connection must succeed between MC-LAG peers on Switch A and Switch B.Note

    On QFX and EX Series switches, the default session establishment hold time is 300 seconds. However, the session establishment time must be at least 100 seconds higher than the init delay time. You can optionally update the session establishment time to be 340 seconds and the init delay time to be 240 seconds.

    Switch A:

    Switch B:

  5. (Optional) Configure the backup IP address to be used for backup liveness detection on both Switch A and Switch B.Note

    By default, backup liveness detection is not enabled. Configuring a backup IP address helps achieve sub-second traffic loss during an MC-LAG peer reboot.

    Switch A:

    Switch B:

  6. Configure Layer 3 connectivity between the MC-LAG peers on both Switch A and Switch B using the original CLI.
  7. Configure Layer 3 connectivity between the MC-LAG peers on both Switch A and Switch B using ELS.

Step-by-Step Procedure

To enable the MC-LAG interface:

  1. Enable LACP on the MC-LAG interface on Switch A and Switch B.Note

    At least one end needs to be active. The other end can be either active or passive.

  2. Specify the same multichassis aggregated Ethernet identification number on both MC-LAG peers on Switch A and Switch B.
  3. Specify the same service ID on Switch A and Switch B.

    ELS:

  4. Specify a unique chassis ID for the MC-LAG on the MC-LAG peers on Switch A and Switch B.

    Switch A:

    Switch B:

  5. Specify the operating mode of the MC-LAG on both Switch A and Switch B.Note

    Only active-active mode is supported at this time.

  6. Specify the status control for MC-LAG on Switch A and Switch B.Note

    You must configure status control on both Switch A and Switch B hosting the MC-LAG. If one peer is in active mode, the other must be in standby mode.

    Switch A:

    Switch B:

    Note

    f you configure both nodes as prefer-status-control-active, you must also configure ICCP peering using the peer’s loopback address to make sure that the ICCP session does not go down because of physical link failures. Additionally, you must configure backup liveness detection on both of the MC-LAG nodes.

  7. Specify the number of seconds by which the bring-up of the multichassis aggregated Ethernet interface should be deferred after you reboot Switch A and Switch B.Note

    On QFX and EX Series switches, the default session establishment hold time is 300 seconds. However, the session establishment time must be at least 100 seconds higher than the init delay time. You can optionally update the session establishment time to be 340 seconds and the init delay time to be 240 seconds.

  8. Specify the same LACP system ID for the MC-LAG on Switch A and Switch B.
  9. Specify the same LACP administration key on both Switch A and Switch B.
  10. Enable a VLAN on the MC-LAG on Switch A and Switch B using the original CLI.
  11. Enable a VLAN on the MC-LAG on Switch A and Switch B using ELS.
  12. Enable VRRP on the MC-LAG on Switch A and Switch B:
    • Create a routed VLAN interface (RVI), assign a virtual IP address that is shared between each switch in the VRRP group, and assign an individual IP address for each switch in the VRRP group:

      Switch A:

      Switch B:

    • Assign the priority for each switch in the VRRP group:

      Note

      The switch configured with the highest priority is the master.

      Switch A:

      Switch B:

    • Enable the switch to accept all packets destined for the virtual IP address if it is the master in the VRRP group:

      Switch A:

      Switch B:

    • Configure Layer 3 connectivity between Switch A and Switch B.

  13. Enable VRRP on the MC-LAG on Switch A and Switch B using ELS:
    • Create a routed VLAN interface (RVI), assign a virtual IP address that is shared between each switch in the VRRP group, and assign an individual IP address for each switch in the VRRP group:

      Switch A:

      Switch B:

    • Assign the priority for each switch in the VRRP group:

      Note

      The switch configured with the highest priority is the master.

      Switch A:

      Switch B:

    • Enable the switch to accept all packets destined for the virtual IP address if it is the master in the VRRP group:

      Switch A:

      Switch B:

    • Configure Layer 3 connectivity between Switch A and Switch B.

Step-by-Step Procedure

To enabled RSTP:

  1. Enable RSTP globally on all interfaces on Switch A and Switch B.Note

    The all option is not available on ELS, so you cannot issue this command on ELS.

    ELS:

  2. Disable RSTP on the ICL-PL interfaces on Switch A and Switch B.Note

    This command is not needed on ELS.

  3. Configure the MC-LAG interfaces as edge ports on Switch A and Switch B.Note

    The ae1 interface is a downstream interface. This is why RSTP and bpdu-block-on-edge need to be configured.

  4. Enable BPDU blocking on all interfaces except for the ICL-PL interfaces on Switch A and Switch B.Note

    The ae1 interface is a downstream interface. This is why RSTP and bpdu-block-on-edge need to be configured.

Results

From configuration mode , confirm your configuration by entering the show chassis, show interfaces, show protocols, show multi-chassis, and show vlans commands. If the output does not display the intended configuration, repeat the instructions in this example to correct the configuration.

Switch A—Original CLI

Switch A—ELS

Switch B—Original CLI

Switch B—ELS

Verification

Verify that the configuration is working properly.

Verifying That ICCP Is Working on Switch A

Purpose

Verify that ICCP is running on Switch A.

Action

[edit]

user@switch> show iccp

Meaning

This output shows that the TCP connection between the peers hosting the MC-LAG is up, liveness detection is up, and MCSNOOPD and ESWD client applications are running.

Verifying That ICCP Is Working on Switch B

Purpose

Verify that ICCP is running on Switch B.

Action

show iccp

[edit]

user@switch> show iccp

Meaning

This output shows that the TCP connection between the peers hosting the MC-LAG is up, liveness detection is up, and MCSNOOPD and ESWD client applications are running.

Verifying That LACP Is Active on Switch A

Purpose

Verify that LACP is active on Switch A.

Action

[edit]

user@switch> show lacp interfaces

Meaning

This output shows that Switch A is participating in LACP negotiation.

Verifying That LACP Is Active on Switch B

Purpose

Verify that LACP is active on Switch B.

Action

[edit]

user@switch> show lacp interfaces

Meaning

This output shows that Switch B is participating in LACP negotiation.

Verifying That the multichassis aggregated Ethernet and ICL-PL Interfaces Are Up on Switch A

Purpose

Verify that the multichassis aggregated Ethernet and Inter-chassis Link Protection (ICL-PL) interfaces are up on Switch A.

Action

[edit]

user@switch> show interfaces mc-ae

Meaning

This output shows that the multichassis aggregated Ethernet and ICL-PL on Switch A is up and active.

Verifying That the Multichassis Aggregated Ethernet and ICL-PL Interfaces Are Up on Switch B

Purpose

Verify that the multichassis aggregated Ethernet and ICL-PL interfaces are up on Switch B.

Action

[edit]

user@switch> show interfaces mc-ae

Meaning

This output shows that the multichassis aggregated Ethernet and ICL-PL interface on Switch B is up and active.

Verifying that MAC Learning Is Occurring on Switch A

Purpose

Verify that MAC learning is working on Switch A.

Action

[edit]

user@switch> show ethernet-switching table

Meaning

The output shows two static MAC address in VLAN v100 and one static MAC address in VLAN v500. These addresses belong to the Layer 3 RVI addresses on both Switch A and Switch B that you configured in the MC-LAG. The ICL-PL interface configured on the VRRP master member learned the VLAN v100 Learn (R) MAC address of the VRRP backup member.

Verifying that MAC Learning Is Occurring on Switch B

Purpose

Verify that MAC learning is working on Switch B.

Action

[edit]

user@switch> show ethernet-switching table

Meaning

The output shows two static MAC address in VLAN v100 and one static MAC address in VLAN v500. These addresses belong to the Layer 3 RVI addresses on both Switch A and Switch B that you configured in the MC-LAG. The ICL-PL interface configured on the VRRP backup member learned the VLAN v100 Learn (R) MAC address of the VRRP master member.

Verifying that Switch A is the Master in the VRRP Group

Purpose

Verify that Switch A is the master member in the VRRP group.

Action

[edit]

user@switch> show vrrp

Meaning

The output shows that Switch A is the master member in the VRRP group.

Verifying that Switch B is the Backup Member in the VRRP Group

Purpose

Verify that Switch B is the backup member in the VRRP group.

Action

[edit]

user@switch> show vrrp

Meaning

The output shows that Switch B is the backup member in the VRRP group.

Verifying that the Virtual IP Address is Attached to an Individual Address on Switch A

Purpose

Action

[edit]

user@switch# run show interfaces terse vlan

Meaning

The output shows that the virtual IP address (10.1.1.1/8) is bound to the individual IP address (10.1.1.11/8) on Switch A.

Verifying that the Virtual IP Address is Attached to an Individual Address on Switch B

Purpose

Action

[edit]

user@switch# run show interfaces terse vlan

Meaning

The output shows that the virtual IP address (10.1.1.1/8) is bound to the individual IP address (10.1.1.10/8) on Switch B.

Troubleshooting

Troubleshooting a LAG That Is Down

Problem

The show interfaces terse command shows that the MC-LAG is down.

Solution

Check the following:

  1. Verify that there is no configuration mismatch.

  2. Verify that all member ports are up.

  3. Verify that the MC-LAG is part of family Ethernet switching (Layer 2 LAG).

  4. Verify that the MC-LAG member is connected to the correct MC-LAG member at the other end.

Note

Multichassis link aggregation (MC-LAG) is supported on QFX3500 and QFX3600 standalone switches running the original CLI and QFX3500, QFX3600, QFX5100, EX4600, and QFX10000 standalone switches running Enhanced Layer 2 Software (ELS).

There are two methods for enabling Layer 3 multicast functionality across a multichassis link aggregation group (MC-LAG) to control traffic. You can choose either to synchronize the MAC addresses for the Layer 3 interfaces of the switches participating in the MC-LAG , or you can configure Virtual Router Redundancy Protocol (VRRP), but you cannot configure both at the same time. Because RVI interfaces share the same MAC address, if you enable MAC address synchronization, packets received on an MC-LAG peer with a destination MAC address that is the same as that of the peer’s IRB MAC address will not be forwarded. The procedure to configure VRRP for use in a Layer 3 multicast MC-LAG is included in this example.

Requirements

This example uses the following hardware and software components:

  • Junos OS Release 12.3 or later for the QFX3500 and QFX3600 standalone switches and Junos OS Release 13.2X51-D10 or later for the QFX5100 and EX4600 standalone switches, and Junos OS Release 15.1X53-D10 or later for the standalone QFX10000 switches.

  • Two QFX3500 or QFX3600 standalone switches, two QFX5100 standalone switches, two EX4600, or two QFX10002 standalone switches.

Before you configure an MC-LAG for Layer 3 multicast using VRRP, be sure that you understand how to:

Overview

In this example, you configure two MC-LAGs across two switches, consisting of two aggregated Ethernet interfaces (ae1 and ae2). To support the MC-LAG, create a third aggregated Ethernet interface (ae0) for the interchassis control link-protection link (ICL-PL). Configure a multichassis protection link for the ICL-PL, the Inter-Chassis Control Protocol (ICCP) for the peers hosting the MC-LAG, and Layer 3 connectivity between MC-LAG peers.

Note

Layer 3 connectivity is required for ICCP.

Note

On QFX5100 and QFX10000 switches, if you try to configure both VRRP over IRB and MAC synchronization, you will receive a commit error.

To complete the configuration, enable VRRP by completing the following steps for each MC-LAG:

  1. Create a routed VLAN interface (RVI).

  2. Create a VRRP group and assign a virtual IP address that is shared between each switch in the VRRP group.

  3. Enable a member of a VRRP group to accept all packets destined for the virtual IP address if it is the master in the VRRP group.

  4. Configure Layer 3 connectivity between the VRRP groups.

Topology

The topology used in this example consists of two switches hosting two MC-LAGs—ae1 and ae2. The two switches are connected to a multicast source (Server 1) over the MC-LAG ae1, and a multicast receiver (Server 2) over the MC-LAG ae2. Figure 16 shows the topology of this example.

Figure 16: Configuring a Multichassis LAG for Layer 3 Multicast Using VRRP
Configuring a Multichassis LAG
for Layer 3 Multicast Using VRRP

Table 4 details the topology used in this configuration example.

Table 4: Components of the Topology for Configuring a Multichassis LAG for Layer 3 Multicast Using VRRP

HostnameBase HardwareMultichassis Link Aggregation Group

Switch A

Switch B

QFX3500, QFX3600, EX4600, QFX5100, or QFX10000 standalone switch

QFX3500, QFX3600, EX4600, QFX5100, or QFX10000 standalone switch

  • ae0 is configured as an aggregated Ethernet interface, and is used as an ICL-PL. The following two interfaces are part of ae0:

    xe-0/0/12 and xe-0/0/13 on Switch A and

    xe-0/0/12 and xe-0/0/13 on Switch B.

  • ae1 is configured as an MC-LAG for the multicast source (Server 1), and the following two interfaces are part of ae1:

    xe-0/0/44 on Switch A and

    xe-0/0/46 on Switch B.

  • ae2 is configured as an MC-LAG for the multicast receiver (Server 2), and the following two interfaces are part of ae2:

    xe-0/0/45 on Switch A and

    xe-0/0/47 on Switch B.

Configuration

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them in a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level , and then enter commit from configuration mode.

Note

This example shows how to configure MC-LAG using both the original CLI and Enhanced Layer 2 Software (ELS).

In ELS, there are three different statements and one different option from the original CLI:

  • The port-mode statement in the [edit interfaces interface-name unit number family ethernet-switching] hierarchy is not supported. Use the interface-mode statement instead.

  • The vlan statement in the [edit interfaces interface-name] hierarchy is not supported. Use the irb statement instead.

  • The vlan.logical-interface-number hierarchy in the [edit vlans vlan-name l3-interface] option is not supported. Use the irb.logical-interface-number option instead.

  • The service-id statement in the [edit switch-options] hierarchy is required in the ELS CLI.

Switch A—Original CLI

Switch A—ELS

Switch B—Original CLI:

Switch B—ELS

Configuring MC-LAG for Layer 3 Multicast Using VRRP on Two Switches

Step-by-Step Procedure

The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode .

To enable multichassis protection link between MC-LAG peers:

  1. Configure the number of LAGs on both Switch A and Switch B.
  2. Add member interfaces to the aggregated Ethernet interfaces on both Switch A and Switch B.

    Switch A and Switch B:

    Switch A:

    Switch B:

  3. Configure ae0 as the trunk interface between Switch A and Switch B.

    Switch A and Switch B:

    Switch A and Switch B Using ELS:

  4. Configure ae0 as the multichassis protection link between Switch A and Switch B.

    Switch A:

    Switch B:

Step-by-Step Procedure

To enable ICCP:

The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode .

  1. Configure the local IP address to be in the ICCP connection on Switch A and Switch B.

    Switch A:

    Switch B:

  2. Configure the peer IP address, minimum receive interval, and minimum transmit interval for a Bidirectional Forwarding Detection (BFD) session for ICCP on Switch A and Switch B.

    Switch A:

    Switch B:

  3. (Optional) Configure the time during which an ICCP connection must succeed between MC-LAG peers on Switch A and Switch B.Note

    On QFX and EX Series switches, the default session establishment hold time is 300 seconds. However, the session establishment time must be at least 100 seconds higher than the init delay time. You can optionally update the session establishment time to be 340 seconds and the init delay time to be 240 seconds.

    Switch A:

    Switch B:

  4. (Optional) Configure the backup IP address to be used for backup liveness detection on both Switch A and Switch B.Note

    By default, backup liveness detection is not enabled. Configuring a backup IP address helps achieve sub-second traffic loss during an MC-LAG peer reboot.

    Switch A:

    Switch B:

  5. Configure Layer 3 connectivity between the MC-LAG peers on both Switch A and Switch B.Note

    In ELS, use the irb.logical-interface-number instead.

    Switch A and Switch B:

    Switch A and Switch B Using ELS:

    Switch A and Switch B:

    Switch A Using the Original CLI:

    Switch A Using ELS:

    Switch B Using the Original CLI:

    Switch B Using ELS:

Step-by-Step Procedure

To enable the ae1 and ae2 MC-LAG interfaces:

  1. Enable LACP on the MC-LAG interfaces on Switch A and Switch B.Note

    At least one end needs to be active. The other end can be either active or passive.

  2. Specify the same multichassis aggregated Ethernet identification number for each MC-LAG peer on Switch A and Switch B.
  3. Specify the same service ID on Switch A and Switch B.

    ELS:

  4. Specify a unique chassis ID for the MC-LAG on the MC-LAG peers on Switch A and Switch B.

    Switch A:

    Switch B:

  5. Specify the operating mode of the MC-LAGs on both Switch A and Switch B.Note

    Only active-active mode is supported at this time.

  6. Specify the status control for the MC-LAGs on Switch A and Switch B.Note

    You must configure status control on both Switch A and Switch B hosting the MC-LAGs. If one peer is in active mode, the other must be in standby mode.

    Switch A:

    Switch B:

    Note

    If you configure both nodes as prefer-status-control-active, you must also configure ICCP peering using the peer’s loopback address to make sure that the ICCP session does not go down because of physical link failures. Additionally, you must configure backup liveness detection on both of the MC-LAG nodes.

  7. Specify the number of seconds by which the bring-up of the MC-LAG interfaces should be deferred after you reboot Switch A or Switch B.Note

    The recommended value for maximum VLAN configuration (for example, 4,000 VLANS) is 240 seconds. If IGMP snooping is enabled on all of the VLANs, the recommended value is 420 seconds.

  8. Specify the same LACP system ID for each MC-LAG on Switch A and Switch B.
  9. Specify the same LACP administration key on both Switch A and Switch B.
  10. Enable a VLAN for each MC-LAG on Switch A and Switch B.
  11. Configure ae1 and ae2 as trunk interfaces between Switch A and Switch B.

    Original CLI:

    ELS:

Step-by-Step Procedure

To enable VRRP on the MC-LAGs on Switch A and Switch B:

  1. Create a routed VLAN interface (RVI) for each MC-LAG, assign a virtual IP address that is shared between each switch in the VRRP groups, and assign an individual IP address for each switch in the VRRP groups.

    Switch A Using the Original CLI:

    Switch A Using ELS:

    Switch B Using the Original CLI:

    Switch B Using ELS:

  2. Assign the priority for each switch in the VRRP groups:Note

    The switch configured with the highest priority is the master.

    Switch A Using the Original CLI:

    Switch A Using ELS:

    Switch B Using the Original CLI:

    Switch B Using ELS:

  3. Enable the switch to accept all packets destined for the virtual IP address if it is the master in a VRRP group:

    Switch A Using the Original CLI:

    Switch A Using ELS:

    Switch B Using the Original CLI:

    Switch B Using ELS:

  4. Configure Layer 3 connectivity between Switch A and Switch B.

    Original CLI:

    ELS:

Step-by-Step Procedure

To enable IGMP snooping:

  1. Enable IGMP snooping for all VLANs on Switch A and Switch B.

Step-by-Step Procedure

To configure OSPF as the Layer 3 protocol:

  1. Configure an OSPF area on Switch A and Switch B.
  2. Assign the VLAN interfaces for the MC-LAGs as interfaces to the OSPF area on Switch A and Switch B.
  3. Configure the minimum receive interval, minimum transmit interval, and transmit interval threshold for a Bidirectional Forwarding Detection (BFD) session for the OSPF interfaces on Switch A and Switch B.Note

    On a QFX5100 switch, the minimum transmit interval must be 1000 milliseconds or greater. Sub-second timers are not supported in Junos OS 13.2X51-D10 and later. If you configure the minimum transmit interval timer lower than 1000 milliseconds, the state of the MC-LAG can be affected.

Step-by-Step Procedure

To configure PIM as the multicast protocol:

  1. Configure a static rendezvous point (RP) address on Switch A and Switch B.
  2. Configure the address ranges of the multicast groups for which Switch A and Switch B can be a rendezvous point (RP).
  3. Enable PIM on the VLAN interfaces for the MC-LAGs on Switch A and Switch B.
  4. Configure each PIM interface’s priority for being selected as the designated router (DR) on Switch A and Switch B.

    An interface with a higher priority value has a higher probability of being selected as the DR.

    Switch A:

    Switch B:

  5. Configure the minimum receive interval, minimum transmit interval, and transmit interval threshold for a Bidirectional Forwarding Detection (BFD) session for the PIM interfaces on Switch A and Switch B.

Step-by-Step Procedure

To enable RSTP:

  1. Enable RSTP on Switch A and Switch B.

    Original CLI:

  2. Enable RSTP on Switch B.
  3. Disable RSTP on the ICL-PL interfaces on Switch A and Switch B.Note

    This command does not apply on ELS.

  4. Configure the MC-LAG interfaces as edge ports on Switch A and Switch B.Note

    The ae1 and ae2 interfaces are downstream interfaces. This is why RSTP and bpdu-block-on-edge need to be configured.

  5. Enable BPDU blocking on all interfaces except for the ICL-PL interfaces on Switch A and Switch B.Note

    The ae1 and ae2 interfaces are downstream interfaces. This is why RSTP and bpdu-block-on-edge need to be configured.

Results

From configuration mode on Switch A, confirm your configuration by entering the show chassis, show interfaces, show multi-chassis, show protocols, and show vlans commands. If the output does not display the intended configuration, repeat the instructions in this example to correct the configuration.

Switch A—Original CLI:

Switch A—ELS

From configuration mode on Switch B, confirm your configuration by entering the show chassis, show interfaces, show multi-chassis, show protocols, and show vlans commands. If the output does not display the intended configuration, repeat the instructions in this example to correct the configuration.

Switch B--Original CLI

Switch B–ELS

Verification

Verify that the configuration is working properly.

Verifying That Switch A is the Master Designated Router

Purpose

Verify that Switch A is the master designated router (DR).

Action

From operational mode, enter the show pim interfaces command.

user@switch> show pim interfaces

Meaning

The DDR-DR state of the VLAN interfaces in the output shows that Switch A is the master designated router.

Verifying That Switch B is the Backup Designated Router

Purpose

Verify that Switch B is the backup designated router (BDR).

Action

From operational mode, enter the show pim interfaces command.

user@switch> show pim interfaces

Meaning

The DDR-BDR state of the VLAN interfaces in the output shows that Switch B is the backup designated router.

There are two methods for enabling Layer 3 multicast functionality across a multichassis link aggregation group (MC-LAG). You can choose either to configure the Virtual Router Redundancy Protocol (VRRP) or synchronize the MAC addresses for the Layer 3 interfaces of the routers participating in the MC-LAG to load balance the traffic. The procedure to configure VRRP for use in a Layer 3 multicast MC-LAG is included in this example.

Requirements

This example uses the following hardware and software components:

  • Four Juniper Networks MX Series routers

  • Junos OS Release 11.2 or later running on all four routers

Before you configure an MC-LAG for Layer 3 multicast using VRRP, be sure that you understand how to:

  • Configure aggregated Ethernet interfaces on a router.

  • Configure the Link Aggregation Control Protocol (LACP) on aggregated Ethernet interfaces on a router.

  • Configure the Virtual Router Redundancy Protocol (VRRP) on a router.

Overview

In this example, you configure an MC-LAG across two routers by including interfaces from both routers in an aggregated Ethernet interface (ae1). To support the MC-LAG, create a second aggregated Ethernet interface (ae0) for the interchassis control link-protection link (ICL-PL). Configure a multichassis protection link for the ICL-PL, Inter-Chassis Control Protocol (ICCP) for the peers hosting the MC-LAG, and Layer 3 connectivity between MC-LAG peers.

Note

Layer 3 connectivity is required for ICCP.

To complete the configuration, enable VRRP by completing the following steps:

  • Create a routed VLAN interface (RVI).

  • Create a VRRP group and assign a virtual IP address that is shared between each router in the VRRP group.

  • Enable a member of a VRRP group to accept all packets destined for the virtual IP address if it is the master in the VRRP group.

Consider a sample topology in which a customer edge router, CE, is connected to two provider edge (PE) routers, PE1 and PE2, respectively. The two PE devices each have a link aggregation group (LAG) connected to the CE device. The configured mode is active-active, meaning that both PE routers’ LAG ports are active and carrying traffic at the same time. PE1 and PE2 are connected to a single service provider router, P.

From the perspective of the CE device, all four ports belonging to a LAG are connected to a single service provider device. Because the configured mode is active-active, all four ports are active, and the CE device load-balances the traffic to the peering PE devices. On the PE routers, a regular LAG is configured facing the CE device.

On one end of an MC-LAG is an MC-LAG client device, such as a server, that has one or more physical links in a LAG. This client device does not need to detect the MC-LAG. On the other side of an MC-LAG are two MC-LAG routers. Each of the routers has one or more physical links connected to a single client device. The routers coordinate with each other to ensure that data traffic is forwarded properly.

Topology Diagram

Figure 17 shows the topology used in this example.

Figure 17: MC-LAG Active-Active on MX Series Routers
 MC-LAG Active-Active
on MX Series Routers

Configuring the PE Routers

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them in a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level , and then enter commit from configuration mode.

Router PE1

Router PE2

Configuring the PE1 Router

Step-by-Step Procedure

The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode .

Note

Repeat the procedure for Router PE2, after modifying the appropriate interface names, addresses, and other parameters.

To configure Router PE1:

  1. Specify the number of aggregated Ethernet interfaces to be created.

  2. Specify the members to be included within the aggregated Ethernet bundles.

  3. Configure the interfaces that connect to senders or receivers, the interchassis link(ICL) interfaces, and the ICCP interfaces.

  4. Configure parameters on the aggregated Ethernet bundles.

  5. Configure LACP on the aggregated Ethernet bundles.

  6. Configure the MC-LAG interfaces.

    The multichassis aggregated Ethernet identification number (mc-ae-id) specifies which link aggregation group the aggregated Ethernet interface belongs to. The ae0 interfaces on Router PE1 and Router PE2 are configured with mc-ae-id 5. The ae1 interfaces on Router PE1 and Router PE2 are configured with mc-ae-id 10.

    The redundancy-group 10 statement is used by ICCP to associate multiple chassis that perform similar redundancy functions and to establish a communication channel so that applications on peering chassis can send messages to each other. The ae0 and ae1 interfaces on Router PE1 and Router PE2 are configured with the same redundancy group redundancy-group 10.

    The chassis-id statement is used by LACP for calculating the port number of the MC-LAG's physical member links. Router PE1 uses chassid-id 1 to identify both its ae0 and ae1 interfaces. Router PE2 uses chassis-id 0 to identify both its ae0 and ae1 interfaces.

    The mode statement indicates whether an MC-LAG is in active-standby mode or active-active mode. Chassis that are in the same group must be in the same mode.

  7. Configure a domain that includes the set of logical ports.

    The ports within a bridge domain share the same flooding or broadcast characteristics in order to perform Layer 2 bridging.

    The bridge-level service-id statement is required to link related bridge domains across peers (in this case Router PE1 and Router PE2), and should be configured with the same value.

  8. Configure ICCP parameters.

  9. Configure the service ID at the global level.

    You must configure the same unique network-wide configuration for a service in the set of PE routers providing the service. This service ID is required if the multichassis aggregated Ethernet interfaces are part of a bridge domain.

Step-by-Step Procedure

To enable VRRP on the MC-LAGs :

  1. Assign the priority for each router in the VRRP groups.Note

    The router configured with the highest priority is the master.

    PE1

    PE2

  2. Enable the router to accept all packets destined for the virtual IP address if it is the master in a VRRP group.

    PE1

    PE2

Step-by-Step Procedure

To configure OSPF as the Layer 3 protocol:

  1. Configure an OSPF area .
  2. Assign the VLAN interfaces for the MC-LAGs as interfaces to the OSPF area .
  3. Configure the minimum receive interval, minimum transmit interval, and transmit interval threshold for a Bidirectional Forwarding Detection (BFD) session for the OSPF interfaces .

Step-by-Step Procedure

To configure PIM as the multicast protocol:

  1. Configure a static rendezvous point (RP) address .
  2. Configure the address ranges of the multicast groups for which PE1 and PE2 can be a rendezvous point (RP).
  3. Enable PIM on the VLAN interfaces for the MC-LAGs on PE1 and PE2.
  4. Configure each PIM interface’s priority for being selected as the designated router (DR).

    An interface with a higher priority value has a higher probability of being selected as the DR.

  5. Configure the minimum receive interval, minimum transmit interval, and transmit interval threshold for a Bidirectional Forwarding Detection (BFD) session for the PIM interfaces on PE1 and PE2.

Step-by-Step Procedure

To enable RSTP:

  1. Enable RSTP globally on all interfaces.
    [edit]

    user@PE1# set protocols rstp interface ae1.0 mode point-to-point
  2. Configure the MC-LAG interfaces as edge ports on PE1 and PE2.Note

    The ae1 interface is a downstream interface. This is why RSTP and bpdu-block-on-edge need to be configured.

    [edit]

    user@PE1# set protocols rstp interface ae1.0 edge
  3. Enable BPDU blocking on all interfaces except for the ICL-PL interfaces on PE1 and PE2.Note

    The ae1 interface is a downstream interface. This is why RSTP and bpdu-block-on-edge need to be configured.

    [edit]

    user@PE1# set protocols rstp bpdu-block-on-edge

Results

From configuration mode, confirm your configuration by entering the show bridge-domains, show chassis, show interfaces, show protocols, and show switch-options commands. If the output does not display the intended configuration, repeat the instructions in this example to correct the configuration.

If you are done configuring the device, enter commit from configuration mode.

Repeat the procedure for Router PE2, using the appropriate interface names and addresses.

Configuring the CE Device

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them in a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

Device CE

Configuring the CE Device

Step-by-Step Procedure

The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode .

To configure Device CE:

  1. Specify the number of aggregated Ethernet interfaces to be created.

  2. Specify the members to be included within the aggregated Ethernet bundle.

  3. Configure an interface that connects to senders or receivers.

  4. Configure parameters on the aggregated Ethernet bundle.

  5. Configure LACP on the aggregated Ethernet bundle.

    The active statement initiates transmission of LACP packets.

    For the system-priority statement, a smaller value indicates a higher priority. The device with the lower system priority value determines which links between LACP partner devices are active and which are in standby mode for each LACP group. The device on the controlling end of the link uses port priorities to determine which ports are bundled into the aggregated bundle and which ports are put in standby mode. Port priorities on the other device (the noncontrolling end of the link) are ignored.

  6. Configure a domain that includes the set of logical ports.

    The ports within a bridge domain share the same flooding or broadcast characteristics in order to perform Layer 2 bridging.

Results

From configuration mode, confirm your configuration by entering the show bridge-domains, show chassis, and show interfaces commands. If the output does not display the intended configuration, repeat the instructions in this example to correct the configuration.

If you are done configuring the device, enter commit from configuration mode.

Configuring the Provider Router

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them in a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level , and then enter commit from configuration mode.

Router P

Configuring the P Router

Step-by-Step Procedure

The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode .

To configure Router P:

  1. Specify the number of aggregated Ethernet interfaces to be created.

  2. Specify the members to be included within the aggregated Ethernet bundle.

  3. Configure an interface that connects to senders or receivers.

  4. Configure parameters on the aggregated Ethernet bundle.

  5. Configure LACP on the aggregated Ethernet bundle.

  6. Configure a domain that includes the set of logical ports.

Results

From configuration mode, confirm your configuration by entering the show bridge-domains, show chassis, and show interfaces commands. If the output does not display the intended configuration, repeat the instructions in this example to correct the configuration.

If you are done configuring the device, enter commit from configuration mode.

Verification

Confirm that the configuration is working properly by running the following commands:

  • show iccp

  • show interfaces ae0

  • show interfaces ae1

  • show interfaces mc-ae

  • show pim interfaces

  • show vrrp

  • show l2-learning instance extensive

Troubleshooting

Troubleshooting a LAG That Is Down

Problem

The show interfaces terse command shows that the MC-LAG is down.

Solution

Check the following:

  • Verify that there is no configuration mismatch.

  • Verify that all member ports are up.

  • Verify that the MC-LAG is part of family Ethernet switching (Layer 2 LAG).

  • Verify that the MC-LAG member is connected to the correct MC-LAG member at the other end.

There are two methods for enabling Layer 3 unicast functionality across a multichassis link aggregation group (MC-LAG) to control traffic flow. You can choose either to configure Virtual Router Redundancy Protocol (VRRP) or synchronize the MAC addresses for the Layer 3 interfaces of the routers participating in the MC-LAG. The procedure to configure VRRP for use in a Layer 3 unicast MC-LAG is included in this example.

Requirements

This example uses the following hardware and software components:

  • Four Juniper Networks MX Series routers

  • Junos OS Release 11.2 or later running on all four routers

Before you configure an MC-LAG, be sure that you understand how to:

  • Configure aggregated Ethernet interfaces on a router.

  • Configure Link Aggregation Control Protocol (LACP) on aggregated Ethernet interfaces on a router.

  • Configure Virtual Router Redundancy Protocol (VRRP) on a router.

Overview

In this example, you configure an MC-LAG across two routers by including interfaces from both routers in an aggregated Ethernet interface (ae0). Configure a multichassis protection link for the ICL-PL, Inter-Chassis Control Protocol (ICCP) for the peers hosting the MC-LAG, and Layer 3 connectivity between MC-LAG peers.

Note

Layer 3 connectivity is required for ICCP.

To complete the configuration, enable VRRP by completing the following steps:

  1. Create a routed VLAN interface (RVI).

  2. Create a VRRP group and assign a virtual IP address that is shared between each router in the VRRP group.

  3. Enable a member of a VRRP group to accept all packets destined for the virtual IP address.

Consider a sample topology in which a customer edge router, CE, is connected to two provider edge (PE) routers, PE1 and PE2, respectively. The two PE devices each have a LAG connected to the CE device. The configured mode is active-active, meaning that both PE routers’ LAG ports are active and carrying traffic at the same time. PE1 and PE2 are connected to a single service provider router, the P router.

In this example, the CE router is not aware that its aggregated Ethernet links are connected to two separate PE devices. The two PE devices each have a LAG connected to the CE device. The configured mode is active-active, meaning that both PE routers’ LAG ports are active and carrying traffic at the same time.

From the perspective of Router CE, the two ports belonging to a LAG are connected to a single service provider device. Because the configured mode is active-active, the two ports are active, and the CE device load-balances the traffic to the peering PE devices. On the PE routers, a regular LAG is configured facing the CE device.

On one end of an MC-LAG is an MC-LAG client device, such as a server, that has one or more physical links in a link aggregation group (LAG). This client device does not need to detect the MC-LAG. On the other side of an MC-LAG are two MC-LAG routers. Each of the routers has one or more physical links connected to a single client device. The routers coordinate with each other to ensure that data traffic is forwarded properly.

ICCP messages are sent between the two PE devices. In this example, you configure an MC-LAG across two routers, consisting of two aggregated Ethernet interfaces, an interchassis link-protection link (ICL-PL), multichassis protection link for the ICL-PL, and ICCP for the peers hosting the MC-LAG.

Topology Diagram

Figure 18 shows the topology used in this example.

Figure 18: MC-LAG Active-Active on MX Series Routers
MC-LAG Active-Active
on MX Series Routers

Configuring the PE Routers

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them in a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

Router PE1

Router PE2

Configuring the PE1 Router

Step-by-Step Procedure

The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode .

Note

Repeat this procedure for Router PE2, after modifying the appropriate interface names, addresses, and other parameters.

To configure Router PE1:

  1. Specify the number of aggregated Ethernet interfaces to be created.

  2. Specify the members to be included within the aggregated Ethernet bundles.

  3. Configure the interfaces that connect to senders or receivers, the ICL interfaces, and the ICCP interfaces.

  4. Configure parameters on the aggregated Ethernet bundles.

  5. Configure LACP on the aggregated Ethernet bundles.

  6. Configure the MC-LAG interfaces.

    The multichassis aggregated Ethernet identification number (mc-ae-id) specifies which link aggregation group the aggregated Ethernet interface belongs to. The ae0 interface on Router PE1 is configured with mc-ae-id 1.

    The redundancy-group 1 statement is used by ICCP to associate multiple chassis that perform similar redundancy functions and to establish a communication channel so that applications on peering chassis can send messages to each other. The ae0 interfaces on Router PE1 and Router PE2 are configured with the same redundancy group, redundancy-group 1.

    The chassis-id statement is used by LACP for calculating the port number of the MC-LAG's physical member links. Router PE1 uses chassid-id 1 to identify ae0 interface.

    The mode statement indicates whether an MC-LAG is in active-standby mode or active-active mode. Chassis that are in the same group must be in the same mode.

  7. Configure a domain that includes the set of logical ports.

    The ports within a bridge domain share the same flooding or broadcast characteristics in order to perform Layer 2 bridging.

    The bridge-level service-id statement is required to link related bridge domains across peers (in this case Router PE1 and Router PE2), and should be configured with the same value.

  8. Configure ICCP parameters.

  9. Configure the service ID at the global level.

    You must configure the same unique network-wide configuration for a service in the set of PE routers providing the service. This service ID is required if the multichassis aggregated Ethernet interfaces are part of a bridge domain.

Step-by-Step Procedure

To enable VRRP on the MC-LAGs :

  • Enable VRRP on the MC-LAG.
    • Create a Integrated Routing and Bridging (IRB), assign a virtual IP address that is shared between each router in the VRRP group, and assign an individual IP address for each router in the VRRP group.

    • Assign the priority for each router in the VRRP group.

      Note

      The router configured with the highest priority is the master.

    • Enable the router to accept all packets destined for the virtual IP address.

Configuring the PE2 Router

Step-by-Step Procedure

The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode .

To configure Router PE2:

  1. Specify the number of aggregated Ethernet interfaces to be created.

  2. Specify the members to be included within the aggregated Ethernet bundles.

  3. Configure the interfaces that connect to senders or receivers, the ICL interfaces, and the ICCP interfaces.

  4. Configure parameters on the aggregated Ethernet bundles.

  5. Configure LACP on the aggregated Ethernet bundles.

  6. Configure the MC-LAG interfaces.

    The multichassis aggregated Ethernet identification number (mc-ae-id) specifies which link aggregation group the aggregated Ethernet interface belongs to. The ae0 interface on Router PE2 is configured with mc-ae-id 1.

    The redundancy-group 1 statement is used by ICCP to associate multiple chassis that perform similar redundancy functions and to establish a communication channel so that applications on peering chassis can send messages to each other. The ae0 interfaces on Router PE1 and Router PE2 are configured with the same redundancy group, redundancy-group 1.

    The chassis-id statement is used by LACP for calculating the port number of the MC-LAG's physical member links. Router PE2 uses chassid-id 0 to identify its ae0.

    The mode statement indicates whether an MC-LAG is in active-standby mode or active-active mode. Chassis that are in the same group must be in the same mode.

  7. Configure a domain that includes the set of logical ports.

    The ports within a bridge domain share the same flooding or broadcast characteristics in order to perform Layer 2 bridging.

    The bridge-level service-id statement is required to link related bridge domains across peers (in this case Router PE1 and Router PE2), and should be configured with the same value.

  8. Configure ICCP parameters.

  9. Configure the service ID at the global level.

    You must configure the same unique network-wide configuration for a service in the set of PE routers providing the service. This service ID is required if the multichassis aggregated Ethernet interfaces are part of a bridge domain.

Step-by-Step Procedure

To enable VRRP on the MC-LAGs :

  • Enable VRRP on the MC-LAG.
    • Create a Integrated and Bridging Interface (IRB), assign a virtual IP address that is shared between each router in the VRRP group, and assign an individual IP address for each router in the VRRP group.

    • Assign the priority for each router in the VRRP group.

      Note

      The router configured with the highest priority is the master.

    • Enable the router to accept all packets destined for the virtual IP address.

Results

From configuration mode, confirm your configuration by entering the show bridge-domains, show chassis, show interfaces, show protocols, and show switch-options commands. If the output does not display the intended configuration, repeat the instructions in this example to correct the configuration.

If you are done configuring the device, enter commit from configuration mode.

Configuring the CE Device

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them in a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

Device CE

Configuring the CE Device

Step-by-Step Procedure

The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode .

To configure Device CE:

  1. Specify the number of aggregated Ethernet interfaces to be created.

  2. Specify the members to be included within the aggregated Ethernet bundle.

  3. Configure an interface that connects to senders or receivers.

  4. Configure parameters on the aggregated Ethernet bundle.

  5. Configure LACP on the aggregated Ethernet bundle.

    The active statement initiates transmission of LACP packets.

    For the system-priority statement, a smaller value indicates a higher priority. The device with the lower system priority value determines which links between LACP partner devices are active and which are in standby mode for each LACP group. The device on the controlling end of the link uses port priorities to determine which ports are bundled into the aggregated bundle and which ports are put in standby mode. Port priorities on the other device (the noncontrolling end of the link) are ignored.

  6. Configure a domain that includes the set of logical ports.

    The ports within a bridge domain share the same flooding or broadcast characteristics in order to perform Layer 2 bridging.

Results

From configuration mode, confirm your configuration by entering the show bridge-domains, show chassis, and show interfaces commands. If the output does not display the intended configuration, repeat the instructions in this example to correct the configuration.

If you are done configuring the device, enter commit from configuration mode.

Configuring the Provider Router, P

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them in a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

Router P

Configuring the P Router

Step-by-Step Procedure

The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode .

To configure Router P:

  1. Specify the number of aggregated Ethernet interfaces to be created.

  2. Specify the members to be included within the aggregated Ethernet bundle.

  3. Configure an interface that connects to senders or receivers.

  4. Configure parameters on the aggregated Ethernet bundle.

  5. Configure LACP on the aggregated Ethernet bundle.

  6. Configure a domain that includes the set of logical ports.

Results

From configuration mode, confirm your configuration by entering the show bridge-domains, show chassis, and show interfaces commands. If the output does not display the intended configuration, repeat the instructions in this example to correct the configuration.

If you are done configuring the device, enter commit from configuration mode.

Verification

Confirm that the configuration is working properly by running the following commands:

Troubleshooting

Troubleshooting a LAG That Is Down

Problem

The show interfaces terse command shows that the MC-LAG is down.

Solution

Check the following:

  1. Verify that there is no configuration mismatch.

  2. Verify that all member ports are up.

  3. Verify that the MC-LAG is part of family Ethernet switching (Layer 2 LAG).

  4. Verify that the MC-LAG member is connected to the correct MC-LAG member at the other end.