Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
 

Optimized Inter-Subnet Multicast in EVPN Networks

SUMMARY Enable inter-subnet multicast (OISM) to optimize multicast traffic routing and forwarding in an EVPN edge-routed bridging (ERB) overlay fabric. With OISM, your network can also support multicast traffic flow between devices inside and outside of the EVPN data center.

Overview of OISM

Optimized inter-subnet multicast (OISM) is a multicast traffic optimization feature. It operates at Layer 2 and Layer 3 in EVPN-VXLAN edge-routed bridging (ERB) overlay fabrics. The design is based on the IETF draft specification https://datatracker.ietf.org/doc/html/draft-ietf-bess-evpn-irb-mcast. You can apply OISM configuration and operation to multicast traffic but not to broadcast or unknown unicast traffic..

In EVPN ERB overlay fabric designs, the leaf devices in the fabric route traffic between tenant bridge domains (that is, between VLANs). When you enable OISM, the leaf devices route inter-subnet multicast traffic locally through IRB interfaces using the control plane multicast states. With local routing between VLANs, the receiver IRB interface doesn't send the routed multicast traffic out into the EVPN core. The local routing model helps minimize the traffic load within the EVPN core. It also avoids traffic hairpinning.

OISM leaf devices also selectively forward traffic into the EVPN core only toward other EVPN devices with interested receivers. Selective forwarding further improves multicast traffic performance in the EVPN data center.

Finally, with OISM enabled, data centers with the ERB overlay model support multicast traffic flow between devices inside and outside of the EVPN fabric. Without OISM, data center designers must use the centrally routed bridging (CRB) overlay model to support multicast with external sources or receivers. OISM uses IRB interfaces on a multicast VLAN (M-VLAN) on the border leaf devices to route traffic to and from an external Protocol-Independent Multicast (PIM) domain. The border leaf devices employ a single supplemental bridge domain (SBD) to carry the traffic from external sources toward receivers within the EVPN fabric.

Note:

You can also use OISM to handle external multicast traffic using pure Layer 3 interfaces rather than M-VLAN IRB interfaces. This documentation focuses on the M-VLAN use case.

Benefits of OISM

  • Enables EVPN-VXLAN data centers with the ERB overlay model to support multicast traffic with sources and receivers outside of the data center.
  • Minimizes multicast control packets and replicated data packets in the EVPN fabric core to optimize data center multicast performance in scaled designs.

OISM with Other Multicast Protocols and Optimizations in EVPN Fabrics

OISM works with the following multicast protocols:

  • IGMPv2.

  • PIM, which facilitates both local routing and external multicast traffic routing.

OISM works with the following other EVPN multicast optimizations:

  • IGMP snooping on the access side on the leaf devices.

    With IGMP snooping enabled, a leaf device that receives multicast traffic forwards it only toward other devices with interested receivers.

  • Multihoming support in an Ethernet segment using EVPN Type 7 (Join Sync) and Type 8 (Leave Sync) routes.

    EVPN fabric devices advertise these route types to synchronize the multicast state among EVPN devices that are multihoming peers.

  • Selective multicast Ethernet tag (SMET) forwarding in the EVPN fabric core using EVPN Type 6 routes.

    EVPN devices use Type 6 routes to limit forwarding within the EVPN core only to receivers interested in receiving traffic for a multicast group. You can use OISM to make this optimization work in EVPN ERB overlay fabrics. When you configure IGMP snooping, the fabric enables SMET forwarding with OISM automatically.

OISM Components

The OISM environment includes:

  • Leaf devices in the EVPN fabric that function in border roles and server access roles.

  • External multicast sources and receivers in an external Layer 3 PIM domain.

  • Bridge domain (VLAN) configurations that enable the fabric to route multicast traffic between internal and external devices.

The EVPN-VXLAN ERB overlay design includes lean spine devices that support Layer 3 transit functions for the leaf devices. The lean spine devices don't usually perform any OISM functions.

The following sections describe these OISM components.

OISM Device Roles

Figure 1 shows a simple EVPN-VXLAN ERB overlay fabric and the OISM device roles in the fabric.

Figure 1: EVPN Fabric with OISM EVPN Fabric with OISM

Table 1 summarizes the device roles.

Table 1: EVPN Fabric OISM Device Roles
Device Role Description

Border leaf (BL)

OISM leaf devices in the EVPN fabric underlay and overlay. Border leaf devices function as gateways interconnecting the EVPN fabric to multicast devices (sources and receivers) outside the data center in an external PIM domain. These devices serve in the PIM EVPN gateway (PEG) role.

Lean spine (LS)

Spine devices in the underlay of the EVPN data center fabric. These devices usually operate as lean spines that support the EVPN underlay as IP transit devices. The lean spines might also act as route reflectors in the fabric.

Note:

You don't need to configure OISM on the lean spine devices unless the devices also serve as border devices for external multicast traffic. In that case, you configure the same OISM elements as you configure on border leaf devices.

Server leaf (Leaf)

OISM leaf devices on the access side in the EVPN fabric underlay and overlay. Server leaf devices are often top-of-rack (ToR) switches. These devices connect the EVPN fabric to multicast sources and multicast receiver hosts on bridge domains or VLANs within the data center.

PIM Domain with External Multicast Sources and Receivers

In Figure 1, the OISM border leaf devices connect to multicast sources and receivers outside the EVPN fabric in a representative external PIM domain. The multicast devices in the external PIM domain follow standard PIM protocol procedures; their operation is not specific to OISM. External multicast source and receiver traffic flow at Layer 3 through the PIM domain.

You can use OISM to route and to forward multicast traffic in an EVPN-VXLAN ERB overlay fabric between devices in the following use cases:

  • Internal multicast sources and internal multicast receivers

  • Internal multicast sources and external multicast receivers

  • External multicast sources and internal multicast receivers

For simplicity, the external PIM domain represented in this documentation comprises:

  • A PIM router (a device such as an MX Series router) that doubles as the PIM rendezvous point (RP).

  • An external source.

  • An external receiver.

Note:

The OISM border leaf devices can route multicast traffic from or to devices outside of the fabric using either of the following:

  • IRB interfaces on a multicast VLAN (M-VLAN)

  • Pure Layer 3 interfaces

This document focuses on the M-VLAN solution. The next sections describe more about how OISM uses an M-VLAN for external multicast traffic.

See Overview of Multicast Forwarding with IGMP Snooping or MLD Snooping in an EVPN-VXLAN Environment for an overview of connecting EVPN-VXLAN fabrics to an external PIM domain using a Layer 2 M-VLAN or multiple Layer 3 links.

OISM Bridge Domains (VLANs)

Table 2 summarizes the OISM bridge domains or VLANs and describes how OISM uses them.

Note:

References in this document to all OISM devices correspond to the border leaf and server leaf devices on which you enable OISM.

Table 2: OISM Bridge Domains or VLANs
Bridge Domain/VLAN Description Configure on:

Multicast VLAN (M-VLAN)

(Also called the external VLAN)

VLAN that connects the EVPN fabric to multicast sources and multicast receivers outside the data center through an external multicast router.

You can multihome the external multicast router to multiple border leaf device M-VLAN IRB interfaces in the same EVPN Ethernet segment. The usual EVPN multihoming designated forwarder (DF) rules apply to send only one copy of the traffic in the ES on the M-VLAN.

Configure the M-VLAN as a VLAN that is not the SBD or any of the revenue bridge domains (or VLANs) in the EVPN data center.

Border leaf devices

Revenue bridge domains (VLANs}

Bridge domains or VLANs for subscribers to the services that the data center provides. We sometimes call revenue bridge domains customer VLANs. These VLANs are not specific to OISM, but the multicast sources and receivers in the data center are in these bridge domains.

All OISM devices

Supplemental bridge domain (SBD)

Bridge domain (VLAN) that enables support for external multicast traffic and implements SMET optimization in the EVPN core. One SBD serves all OISM leaf devices in the fabric. The SBD carries:

  • Routed multicast data traffic from external multicast sources to EVPN devices in the fabric with interested receivers.
  • SMET EVPN Type 6 routes from originating leaf devices to other EVPN leaf devices, which enables selective forwarding in the EVPN core.
Note:

The SBD is central to OISM operation. The SBD IRB interface must always be up for OISM to work.

Configure the SBD as a distinct VLAN that is not the M-VLAN or a revenue bridge domain in the EVPN data center.

All OISM devices

The OISM implementation uses a symmetric bridge domains model.

The following figure shows how you configure the bridge domains and VLANs in the EVPN fabric on the OISM devices with this model.

Figure 2: EVPN Fabric with OISM Symmetric Bridge Domains Model EVPN Fabric with OISM Symmetric Bridge Domains Model

In the symmetric bridge domains model, you must configure the SBD and all revenue bridge domains on all OISM border leaf and server leaf devices. You configure the M-VLAN on the border leaf devices that connect to the external PIM domain. The lean spine devices in the fabric usually serve only as IP transit devices and possibly as route reflectors. As a result, you don't need to configure these elements on the lean spine devices.

Configuration Elements for OISM Devices (Symmetric Bridge Domains)

This section summarizes the elements you need to configure on:

  • All OISM devices—devices in both the border leaf and server leaf roles.
  • Border leaf devices only.
  • Server leaf devices only.

Some elements are optional, which the description notes.

Table 3 lists the elements you configure on all OISM devices.

Table 3: Configuration Elements on All OISM Devices
Configuration Element Description

OISM

Enable OISM globally, and enable OISM routing functions in Layer 3 virtual routing and forwarding (VRF) instances.

A device with OISM enabled advertises EVPN Type 3 inclusive multicast Ethernet tag (IMET) routes as follows:

  • The device advertises an IMET route for each configured revenue bridge domain (VLAN) and the SBD.

  • The IMET routes include the EVPN multicast flags extended community with a flag indicating OISM support.

Revenue bridge domains (customer VLANs) and corresponding IRB interfaces

Configure revenue bridge domains (customer VLANs) according to your data center services requirements. You must configure all revenue bridge domain VLANs and corresponding IRB interfaces on all OISM devices. See OISM Bridge Domains (VLANs) for more information.

SBD (VLAN) and corresponding IRB interface

Configure the SBD and its IRB interface on all OISM devices. The SBD can be any VLAN that is distinct from the M-VLAN or any revenue bridge domain VLANs in the EVPN data center. See OISM Bridge Domains (VLANs) for more information.

You identify this VLAN as the SBD in the Layer 3 virtual routing and forwarding (VRF) instance that supports OSIM routing. EVPN IMET routes for the SBD IRB interface include the OISM SBD flag in the multicast flags extended community.

Layer 2 multicast optimizations—IGMP snooping in proxy mode, SMET

Enable IGMP snooping as part of OISM optimizations. With IGMP snooping enabled, the device routes multicast traffic or forwards the traffic only toward interested access-side receivers. (Receivers send IGMP reports to express interest in receiving traffic for a multicast group.)

Enabling IGMP snooping also automatically enables the device to advertise SMET Type 6 routes. With SMET, the device sends copies of the traffic into the EVPN core only toward other devices that have interested receivers.

Layer 3 virtual routing and forwarding (VRF) instance

Configure a routing instance (instance type vrf) that supports the Layer 3 OISM routing functions.

originate-smet-on-revenue-vlan-too

(Optional) Enable the device to originate SMET Type 6 routes for revenue bridge domains (as well as on the SBD) upon receiving local IGMP reports. By default, OISM devices originate Type 6 routes only on the SBD.

Use this option for compatibility with other vendor devices that don't support OISM. Those devices can't create the right states for the revenue VLANs upon receiving Type 6 routes on the SBD.

Table 4 list the elements you configure on the border leaf devices.

Table 4: Configuration Elements on OISM Border Leaf Devices
Configuration Element Description

M-VLAN (external VLAN) and corresponding IRB interfaces

Configure a VLAN to serve as the M-VLAN. This VLAN must be distinct from the SBD or any revenue bridge domain VLANs in the EVPN data center. See OISM Bridge Domains (VLANs) for more information.

You can link multiple border leaf device M-VLAN IRB interfaces to the external multicast router in the same EVPN Ethernet segment. The usual EVPN multihoming DF rules apply to prevent sending duplicate traffic on the M-VLAN.

Layer 2 multicast router interface on M-VLAN ports

Configure this setting on the interfaces that link the border leaf device to the external PIM domain at Layer 2. These interfaces support multicast traffic when the external domain router is multihomed to the border leaf devices. As a result, multihomed M-VLAN use cases require this configuration.

You configure the Layer 2 port on which you enable IGMP snooping for the M-VLAN as the multicast router interface.

PIM on M-VLAN IRB interfaces

Configure standard PIM mode on an M-VLAN IRB interface. With this setting, the device:

  • Forms PIM neighbor relationships with the other border leaf devices and the external PIM router. The external PIM router might be multihomed in an ES. As a result, EVPN-forwarded traffic might come to a different peer border leaf device.

  • Elects a single last-hop router (LHR) designated router (DR) for the M-VLAN, so only one device does source registration for an internal source toward the PIM RP.

PIM on SBD IRB interfaces

Configure standard PIM mode for SBD IRB interface routing and forwarding. with this setting, the device:

  • Routes external multicast source traffic from the M-VLAN to the SBD, and forwards copies toward server leaf devices with multicast receivers.

  • Forms PIM neighbor relationships with the other border leaf devices.

  • Elects a single LHR DR on the SBD among the peer border leaf devices. Only this elected PIM DR forwards the external multicast source traffic on the SBD. This election prevents peer border leaf devices from forwarding duplicate traffic into the EVPN core.

PIM-EVPN gateway (PEG) role on M-VLAN IRB interfaces

Configure this role on the interfaces that connect the border leaf device to the external PIM router at Layer 2 over the M-VLAN. In this role, the M-VLAN IRB interface uses traditional PIM routing behavior and does local routing as follows:

  • Routes external source traffic from the M-VLAN to local receivers on revenue BDs.

  • Routes external source traffic from the M-VLAN to the SBD. Forward the traffic to the EVPN core only on the SBD to other OISM leaf devices.

  • Locally routes internal source traffic that arrives on a revenue BD to other revenue BDs and the M-VLAN. The device doesn't forward the traffic back into the EVPN core.

EVPN IMET routes for PEG interfaces include the OISM PEG flag in the multicast flags extended community field of the route.

PIM distributed designated router (DR) mode on revenue BD IRB interfaces

Configure PIM distributed DR mode (distributed-dr) to support revenue BD multicast routing and forwarding. In this mode, the device:

  • Forms PIM neighbor relationships with other PIM devices to support multihomed external PIM router connections.

  • Acts as the LHR DR on its revenue BD IRB interfaces, and creates the PIM state locally from received IGMP reports. As a result:

    • The device can do local multicast routing between the revenue BDs (VLANs).

    • One peer border leaf device becomes the PIM DR that performs source registration toward the PIM RP. PIM hello messages and the PIM DR election process determine the PIM DR.

PIM accept-join-always-from option and policy on M-VLAN IRB interfaces

Set this option on the M-VLAN IRB interfaces when the external PIM router is multihomed to more than one EVPN border leaf device. With this option, the device can accept and install the same PIM (S,G) join states on multihoming peer border leaf devices. This option supports sending multicast traffic from sources inside the fabric to receivers in the external PIM domain.

With multihoming on the M-VLAN, the usual EVPN multihoming DF rules apply in an ES to prevent sending duplicate traffic. If peer border leaf devices have the same valid join states in place, any device that is the EVPN DF can forward the multicast traffic.

Configure this statement with policies that specify the interface should always install PIM joins from upstream neighbor addresses that correspond to the external PIM router.

Table 5 lists the elements you configure on the server leaf devices.

Table 5: Configuration Elements on OISM Server Leaf Devices
Configuration Element Description

PIM in passive mode on all revenue VLANs and the SBD

Configure this mode to facilitate local routing without all of the traditional PIM protocol functions. The server leaf device:

  • Doesn't form PIM neighbor relationships with any other devices (avoids sending or receiving PIM hello messages).

  • Acts as a PIM local RP. The device creates the PIM state locally from IGMP reports. The device also doesn't do source registration. (Only the border leaf devices perform source registration toward the external PIM RP.)

PIM with the accept-remote-source option on SBD IRB interfaces

This option enables an SBD IRB interface to accept multicast traffic from a source that isn't on the same subnet. The server leaf devices require this setting because:

  • The border leaf devices route the multicast source traffic from the M-VLAN to the SBD toward the server leaf devices.

  • The traffic arrives at the server leaf device on the SBD IRB interface, which is not a PIM neighbor. The source is not a local source, so without this setting, the device would otherwise drop the traffic.

See the following sections for more details on configuring OISM devices:

How OISM Works

The following sections describe how OISM works and show how the multicast traffic flows in several common use cases.

Local Routing on OISM Devices

InFigure 3, we illustrate how local routing and forwarding works in general on OISM devices. As the figure shows, OISM local routing forwards the traffic on the source VLAN. Each leaf device routes the traffic locally to its receivers on other VLANS, which avoids hairpinning for inter-subnet routing on the same device.

Figure 3: Local Routing with OISM Local Routing with OISM

In this case, the source traffic comes from Mcast-Src-1 on VLAN-1, the blue VLAN. Server leaf devices use IRB interfaces and PIM in passive mode to route traffic between VLANs. With PIM in passive mode, server leaf devices:

  • Don’t become PIM neighbors with the other leaf devices.

  • Act as a local PIM RP, create local PIM state upon receiving IGMP reports, and avoid doing source registration.

As a result, the server leaf devices forward and route multicast traffic within the fabric as follows to receivers interested in the multicast group:

  • The ingress leaf device (Leaf-1) forwards the traffic on the source VLAN into the EVPN fabric toward the other leaf devices with interested receivers.

  • All of the server leaf devices don't need to forward the traffic back into the EVPN core to another device that is a designated router. Server leaf devices can locally:

    • Forward the traffic on the source VLAN toward local interested receivers on the source VLAN.

    • Route the traffic from the source VLAN through the IRB interfaces toward local interested receivers in other VLANs.

Multicast Traffic Forwarding and Routing Within the EVPN Data Center

When the multicast source is inside the EVPN data center, the server leaf devices receive the multicast traffic on the source VLAN. Then they locally route or forward the traffic as described in Local Routing on OISM Devices.

The following figure illustrate OISM local routing and forwarding within an EVPN data center in detail. The figure also shows how local routing works with EVPN multihoming for a multicast receiver.

Figure 4: OISM with an Internal Multicast Source and Multiple Internal Multicast Receivers OISM with an Internal Multicast Source and Multiple Internal Multicast Receivers

In Figure 4:

  • Receivers on all three server leaf devices sent IGMP reports (join messages) expressing interest in receiving the traffic for a multicast group.

  • The multicast source, Mcast-Src-1, is on Leaf-1. The source VLAN is VLAN-1 (the blue VLAN).

  • Leaf-1 forwards the traffic on the source VLAN to both Leaf-2 and Leaf-3 because both leaf devices have interested receivers. In this case, both receivers use single-homing.

  • Leaf-2 and Leaf-3 forward or locally route the traffic to their interested receivers (Rcvr-2, Rcvr-3, and Rcvr-4) as described in Local Routing on OISM Devices.

  • Rcvr-1 on VLAN-2 is multihomed to Leaf-1 and Leaf-2 in an EVPN Ethernet segment, and has expressed interest in receiving the multicast traffic, so:

    • Both server leaf devices, Leaf-1 and Leaf-2, receive the IGMP report.
    • Both Leaf-1 and Leaf-2 locally route the traffic from the source VLAN (VLAN-1) because each device has the PIM passive mode configuration.
    • However, because Leaf-1 is the DF for the EVPN ES, only Leaf-1 forwards the traffic to Rcvr-1.
  • The border leaf devices receive the multicast traffic through the EVPN fabric on the source VLAN. Note that the border leaf devices could have local receivers, although we don't show that case. With local receivers, the device also locally routes or forwards the traffic to those receivers the same way the server leaf devices do.

Figure 4 also shows that the border leaf devices locally route the traffic from the source VLAN to the M-VLAN toward any remote multicast receivers. Because the M-VLAN has the PEG configuration on both border leaf devices, both devices route traffic to their M-VLAN IRB interface. However, only the DF for the M-VLAN Ethernet segment actually forwards the data on the M-VLAN toward the external PIM domain. EVPN DF rules ensure that the PIM domain doesn't receive duplicate copies of the traffic. The next section describes the multicast control and data traffic flow for an external receiver use case.

Multicast Traffic to Receivers Outside the EVPN Data Center

In Figure 5, we illustrate the OISM use case where a multicast source inside the EVPN data center sends multicast traffic to an interested receiver outside the data center. This use case has a multihomed multicast source, and also covers local routing to receivers inside the data center.

The border leaf devices you configure in the OISM PEG role receive the multicast traffic on the source VLAN through the EVPN core. Then the border leaf devices replicate the traffic and route it onto the M-VLAN toward the external PIM domain to reach the external receiver.

Note:

PEG border leaf devices only send multicast source traffic received on the revenue bridge domains to the M-VLAN. These devices don't forward the traffic back into the EVPN core toward the other border leaf devices.

Figure 5: OISM with an Internal Multicast Source and an External Multicast Receiver OISM with an Internal Multicast Source and an External Multicast Receiver

In Figure 5, the internal source for a multicast group is Mcast-Src-2, the same device as Rcvr-1, on VLAN-2 (the red VLAN). The external receiver, Ext-Mcast-Rcvr, expresses interest in receiving the multicast traffic for that multicast group (sends a join message). Internal receivers Rcvr-3 (on VLAN-1) and Rcvr-4 (on VLAN-2) also request to join the multicast group and receive the traffic.

Note that the PIM router is multihomed to both BL-1 and BL-2, the PEG devices, in the EVPN fabric. Those connections are in the same ES; the DF election process chooses one of these devices as the DF for the Ethernet segment. Only the DF will forward traffic (on the M-VLAN) toward external receivers.

The source traffic reaches the interested internal and external receivers as follows:

  • In the external PIM domain, the PIM RP enters a PIM (*,G) multicast routing table entry. The entry includes the Layer 3 interface toward Ext-Mcast-Rcvr as the downstream interface.

  • Mcast-Src-2 (also labeled Rcvr-1) originates the traffic on VLAN-2, the red VLAN. Because the device is multihomed to Leaf-1 and Leaf-2, the device hashes the traffic on VLAN-2 to one of those server leaf devices. In this case, Leaf-2 receives the traffic.

  • The red arrows show that Leaf-2 forwards the traffic on the source VLAN, VLAN-2, to:

    • The other server leaf devices with interested receivers—in this case, only Leaf-3.

    • The border leaf devices, which both act in the OISM PEG role.

    The figure doesn't show that Rcvr-2 or any receivers behind Leaf-1 sent an IGMP report to join the multicast group. With IGMP snooping and SMET forwarding, Leaf-2 doesn't forward the traffic to Leaf-1 because Leaf-1 has no interested receivers. Leaf-2 also doesn't locally route the traffic to Rcvr-2 for the same reason.

  • Leaf-3 receives the source traffic on VLAN-2. Then Leaf-3 routes the traffic locally to VLAN-1 to Rcvr-3. Leaf-3 also forwards the traffic to Rcvr-4 on VLAN-2.

  • Both border leaf devices BL-1 and BL-2 also receive the source traffic from the EVPN core. The IRB interface on VLAN-2 on one of these border leaf devices is the PIM DR for VLAN-2. In this case, The PIM DR is on BL-1, so BL-1 sends a PIM Register message toward the PIM RP on the M-VLAN IRB interface.

  • The PIM RP sends a PIM Join message back toward BL-1. BL-1 creates an (S,G) multicast routing table entry as follows:

    • The source address is the IP address of Mcast-Src-2 in VLAN-2.
    • The downstream interface is the M-VLAN IRB interface.
  • Both BL-1 and BL-2 are PEG devices and configured in PIM distributed DR mode. As a result, both BL-1 and BL-2 receive the PIM Join and create a similar (S,G) state. Both devices route the traffic locally from VLAN-2 to the M-VLAN.

    However, only the DF for the M-VLAN ES actually forwards the data on the M-VLAN to the external PIM domain. In this case, BL-1 is the DF and sends the traffic toward the external receiver. (See the label "MVLAN ESI DF" and the black arrow between BL-1 and the PIM router in Figure 5.)

  • The PIM RP receives the traffic from the OISM M-VLAN. The PIM router sends the traffic to a Layer 3 interface toward the external receiver.

Multicast Traffic from Sources Outside the EVPN Data Center

Figure 6 illustrates the OISM use case where a multicast source outside the EVPN data center sends multicast traffic to receivers inside the data center. This use case exhibits the two main ways OISM uses the SBD in the EVPN core—to carry external multicast source traffic and to advertise SMET Type 6 routes. The Type 6 routes ensure that the border leaf devices only forward the traffic toward the EVPN devices with interested receivers.

OISM border leaf devices receive external multicast source traffic through the M-VLAN IRB interfaces. OISM devices use the SBD to forward traffic toward EVPN server leaf devices with interested receivers on the revenue bridge domains (VLANs). Each leaf device then locally forwards or routes the traffic on the revenue bridge domains (VLANs).

The figure also shows the flow for an internal receiver that is multihomed to two server leaf devices.

Figure 6: OISM with an External Multicast Source and an Internal Multihomed Multicast Receiver OISM with an External Multicast Source and an Internal Multihomed Multicast Receiver

In Figure 6, Rcvr-1 within the EVPN data center is multihomed to server leaf devices Leaf-1 and Leaf-2. Rcvr-1 expresses interest in receiving traffic from a multicast group. The source for the multicast traffic for the group is Ext-Mcast-Src in the external PIM domain. The

The external source traffic reaches the interested multihomed receiver, Rcvr-1, as follows:

  • Rcvr-1 sends an IGMP join message to both multihoming peers Leaf-1 and Leaf-2.

  • Both Leaf-1 and Leaf-2 generate an EVPN Type 6 route toward the EVPN core on the SBD. The Type 6 route advertises that Rcvr-1 is interested in the multicast data.

  • Both border leaf devices BL-1 and BL-2 receive the Type 6 route on the SBD.

  • The Type 6 route (on the SBD) signals the border leaf devices to create a PIM join toward the PIM RP (reachable through the M-VLAN). However, to avoid duplicate join messages, only the border leaf device that is the PIM DR for the SBD generates the PIM join message. In this case, the figure shows the PIM DR for the SBD is BL-1. BL-1 sends the PIM join message toward the PIM RP by way of its neighbor, the M-VLAN IRB interface.

  • The PIM RP receives the join message. Then the PIM RP creates a PIM (*,G) entry in the multicast routing table with the M-VLAN IRB interface as the downstream interface.

  • The external source Ext-Mcast-Src registers with the PIM RP. The PIM RP has a multicast route for the group with the M-VLAN IRB interface as the downstream interface. As a result, the PIM RP routes the multicast traffic coming in at Layer 3 onto the M-VLAN toward BL-1 or BL-2. in this case, BL-1 receives the traffic on its M-VLAN IRB interface.

  • For simplicity, here BL-1 is the PIM DR for the SBD, so BL-1 then routes and forwards the external source traffic as follows:

    • BL-1 forwards a copy of the traffic on the M-VLAN toward BL-2 because both border leaf devices are in the PEG role.

      See the black arrow from BL-1 toward BL-2.

    • BL-1 routes the traffic locally to the SBD IRB interface because BL-1 is the PIM DR for the SBD.

      See the small gray arrow from the M-VLAN to the SBD on BL-1.

    • BL-1 then forwards a copy of the traffic on the SBD into the EVPN core to BL-2.

      BL-2 drops the traffic because as a PEG device, BL-2 expects external source traffic only on the M-VLAN IRB interface. BL-1 doesn't expect external source traffic on the SBD IRB interface from BL-1. In other words, BL-2 sees a source interface mismatch (a reverse path forwarding [RFP] failure). See the green arrow from BL-1 toward BL-2.

      Note:

      One reason why the ingress border leaf device also forwards a copy on the SBD is to ensure that the other border leaf device can receive external source traffic if its M-VLAN interface is down. Then any interested local receivers on the other border leaf device can still get the traffic.

    • BL-1 also selectively forwards copies to the server leaf devices with interested receivers, based on the advertised Type 6 routes. In this case, Leaf-1 and Leaf-2 have a multihomed interested receiver, Rcvr-1, on VLAN-2 (the red VLAN).

      See the green arrows from BL-1 toward Leaf-1 and Leaf-2.

    Note:

    In a similar use case, BL-1 can receive the external multicast source traffic, but BL-2 is the PIM DR on the SBD. In that case, the green arrows you see in Figure 6 flow from BL-2 (instead of BL-1) toward the other EVPN devices.

  • Leaf-1 and Leaf-2 locally route the traffic from the SBD IRB interface to the revenue BD IRB interface for VLAN-2 (the red VLAN). However, only the EVPN DF in the ES forwards the traffic toward Rcvr-1. In this case, Leaf-1 is the EVPN DF.

Local Receivers on Border Leaf Devices

Figure 6 doesn't show local receivers attached to the border leaf devices. However, let's look briefly at PIM join message flow and how external source traffic reaches local receivers on border leaf devices.

Consider that BL-1 or BL-2 has an interested receiver on a revenue BD (VLAN) in the data center. In that case, both devices generate a PIM join on the IRB interfaces for the revenue bridge domain toward the PIM RP.

You configure the border leaf devices with PIM in distributed DR mode on the revenue BD (VLAN) IRB interfaces. That way, neither BL-1 nor BL-2 acts as the PIM DR alone. Both devices can locally route external multicast source traffic coming in on the M-VLAN IRB interface to the appropriate revenue BD (VLAN) IRB interface.

Configure Common OISM Elements on Border Leaf Devices and Server Leaf Devices (Symmetric Bridge Domains)

Follow these steps to configure elements common to border leaf and server leaf devices in an EVPN-VXLAN fabric running optimized inter-subnet multicast (OISM) with the symmetric bridge domains model. This configuration assumes an EVPN-VXLAN fabric configuration that supports OISM and has:

  • An EBGP underlay with Bidirectional Forwarding Detection (BFD) and Ethernet operations, administration and management (OAM) link detection.

  • An edge-routed bridging overlay design.

  • Lean spine devices that act only as IP transit nodes in the fabric.

  • Server leaf devices configured as EVPN-VXLAN Layer 2 gateways.

  • Border leaf devices configured as EVPN-VXLAN Layer 2 and Layer 3 gateways.

In the symmetric bridge domains model, you must configure all of the revenue bridge domains (VLANs) and the supplemental bridge domain (SBD) on all OISM devices in the fabric. See Configuration Elements for OISM Devices (Symmetric Bridge Domains) for a summary of what elements to configure on border leaf and server leaf devices, and why you configure those elements.

The sample configuration blocks that we provide for these configuration steps use an OISM environment with the following elements:

  • M-VLAN: VLAN-900

    M-VLAN IRB interface: irb.900

  • SBD: VLAN-300

    SBD IRB interface: irb.300

  • Revenue bridge domains: VLAN-100 and VLAN-200

    Revenue bridge domain IRB interfaces: irb.100 and irb.200

  • Layer 3 VRF routing instance: L3VRF

Configure these OISM statements on both border leaf devices and server leaf devices:

  1. Enable OISM globally on the device.

    Configure the evpn irb oism option at the [edit forwarding-options multicast-replication] hierarchy level. This option enables OISM optimizations and external multicast capabilities in an edge-routed bridging (ERB) overlay fabric. This option takes the place of the evpn irb local-only option you would otherwise use in ERB overlay fabrics for multicast without OISM.

  2. Configure all of the revenue bridge domains (VLANs) with an IRB interface and a VXLAN network identifier (VNI) mapping for each revenue BD. For example, if the revenue BDs are VLAN-100 and VLAN-200:
  3. Configure a VLAN for the SBD with an IRB interface and VXLAN VNI mapping. For example, for SBD VLAN-300:
  4. Configure the revenue BD and the SBD IRB interface IP addresses. For example:
  5. Extend the revenue BDs and the SBD VNIs in the EVPN-VXLAN overlay. For example:

    Prior to this step, we assume you have set up the EVPN fabric with VXLAN encapsulation by configuring the vxlan statement at the [edit protocols evpn encapsulation] hierarchy level.

  6. Enable IGMP snooping in proxy mode on all the configured VLANs using the igmp-snooping statement. This statement also automatically enables advertising SMET Type 6 routes in the EVPN core based on received IGMP reports.
  7. Configure a Layer 3 VRF routing instance (instance-type vrf) that you associate with OISM routing functions. Include the revenue BD IRB interfaces and the SBD IRB interface in the routing instance. For example:
  8. In the Layer 3 VRF routing instance, specify the VLAN and its IRB interface that acts as the OISM SBD. For example:
  9. (Recommended) To avoid traffic loss at the onset of multicast flows, enable the install-star-g-routes option at the [edit multicast-snooping-options oism] hierarchy level on all OISM devices. With this option, upon receiving an EVPN Type 6 route on the SBD, the device immediately installs corresponding (*,G) routes on the Packet Forwarding Engine for the revenue VLANs in the Layer 3 VRF instance. For full details on OISM device behavior with or without this option, see oism (Multicast Snooping Options).

    You might choose not to enable this option if space on the Packet Forwarding Engine is at a premium and you don't have stringent latency requirements.

Configure Border Leaf Device OISM Elements (Symmetric Bridge Domains)

First configure the optimized inter-subnet multicast (OISM) elements described in Configure Common OISM Elements on Border Leaf Devices and Server Leaf Devices (Symmetric Bridge Domains) in an EVPN-VXLAN fabric.

Then follow these steps to configure the required OISM elements on the border leaf devices. The same EVPN-VXLAN fabric base and sample OISM environment apply to the additional border leaf configuration steps here.

See Configuration Elements for OISM Devices (Symmetric Bridge Domains) for more information about why OISM border leaf devices require these settings.

  1. Configure the M-VLAN similarly to how you configure the revenue bridge domains and the SBD.
    1. Configure the M-VLAN with an IRB interface and a VXLAN network identifier (VNI) mapping. For example, if the M-VLAN is VLAN-900:
    2. Configure the M-VLAN IRB interface IP address. For example:
    3. Extend the M-VLAN in the EVPN-VXLAN overlay. For example:
  2. In the common leaf device configuration steps, you enable IGMP snooping in proxy mode for all VLANs. On border leaf devices, also configure the multicast-router-interface option on the Layer 2 ports that connect to the external multicast PIM router. For example, if xe-0/0/4.0 is the Layer 2 interface that connects the EVPN fabric to the external multicast router on the M-VLAN:
  3. In the common leaf device configuration steps, you configure a Layer 3 VRF routing instance associated with OISM routing functions. On the border leaf devices, also include the M-VLAN IRB interfaces in that Layer 3 VRF. The following configuration block shows the common Layer 3 VRF configuration with the additional M-VLAN IRB interface configuration statement highlighted:
  4. In the Layer 3 VRF routing instance, set the OISM PIM EVPN gateway (PEG) role on the M-VLAN IRB interface. For example:
  5. Configure PIM in the Layer 3 VRF routing instance for a border leaf device.
    1. In the Layer 3 VRF routing instance, configure PIM in distributed DR mode on the revenue BD IRB interfaces using the distributed-dr statement at the [edit routing-instances name protocols pim interface irb-interface-name] hierarchy level. In this step, you configure PIM in standard mode on the SBD IRB interface and the M-VLAN IRB interface. You also configure a PIM static RP that corresponds to the external PIM RP router. In the sample use cases in this documentation, the external PIM router serves as the PIM RP. For example, if the revenue BDs are VLAN-100 and VLAN-200, the SBD is VLAN-300, and the M-VLAN is VLAN-900:
    2. In the Layer 3 VRF routing instance, set the accept-join-always-from option at the [edit routing-instances name protocols pim interface irb-interface-name] hierarchy level on the M-VLAN IRB interface. Configure a policy option along with this statement so that the device always installs PIM joins from the external PIM router. See Configuration Elements for OISM Devices (Symmetric Bridge Domains) for more information about why OISM border leaf devices require this configuration. This sample configuration block represents the external PIM router as an MX Series router. For the policy prefix list, include the IP address of the M-VLAN interface on the MX Series router that connects to the border leaf device. For example:

Configure Server Leaf Device OISM Elements (Symmetric Bridge Domains)

First configure the OISM elements described in Configure Common OISM Elements on Border Leaf Devices and Server Leaf Devices (Symmetric Bridge Domains) in an EVPN-VXLAN fabric.

Then follow these steps to configure the additional required OISM elements on the server leaf devices. The same EVPN-VXLAN fabric base and sample OISM environment apply to the additional server leaf configuration steps here.

See Configuration Elements for OISM Devices (Symmetric Bridge Domains) for more information about why OISM server leaf devices require these settings.

  1. Configure PIM passive mode on server leaf devices. For example:
  2. Enable the server leaf device to accept multicast traffic from the SBD IRB interface as the source interface using the accept-remote-source statement at the [edit routing-instances name protocols pim interface irb-interface-name] hierarchy level. For example, for our sample SBD, VLAN-300, the IRB interface is irb.300:

CLI Commands to Verify the OISM Configuration

To verify your OISM configuration:
  1. Use the show evpn oism command to view the SBD IRB interface in each Layer 3 VRF routing instance that you configured on the device for OISM. For example:

    To view information for a specific routing instance or to see more details about the configuration, use the extensive option with this show command. For example:

  2. Use the show route table bgp.evpn.0 ...extensive command to see the OISM capabilities you have enabled on an OISM leaf device. These capabilities are in the EVPN multicast flags extended community in EVPN Type 3 routes, which are also called inclusive multicast Ethernet tag (IMET) routes. The OISM-related flags include:
    • igmp-snooping—You enabled IGMP snooping. (The flags value for IGMP snooping without OISM or PEG configuration = 0x1.)

    • oism—You globally enabled OISM. (The flags value for OISM and IGMP snooping without PEG configuration = 0x9.)

    • peg—You configured PEG mode on the interface. (The flags value for OISM and IGMP snooping with PEG configuration = 0x19.)

    For example, a route advertised for an M-VLAN IRB interface on a border leaf device has the peg flag set:

    You don't see the peg flag set for a revenue BD interface on a border leaf device or a server leaf device:

  3. Enter the show igmp snooping evpn status <vlan name> <detail> command to see the EVPN-VXLAN Layer 2 multicast context for the OISM bridge domains (VLANs) on the device. The default output includes the VXLAN VNI mappings of the VLANs. For example:

    The detail option also displays whether a VLAN is the OISM SBD (Supplementary BD) or the M-VLAN (External VLAN). Both of those output fields display No for revenue bridge domains (VLANs) or if you don't enable OISM.

    Here is an example on a server leaf device in a fabric where the SBD is VLAN-300:

    Here is another example on a border leaf device in a fabric where the M-VLAN is VLAN-900:

  4. Other show commands display multicast information that isn't specific to OISM elements but can help check configured multicast options and traffic flow.

    For example, use the show evpn multicast-snooping status to see if you enabled IGMP snooping on VLANs you configured for OISM elements.

    In the following sample command, if the revenue BDs on a server leaf device are VLAN-100 and VLAN-200, the SBD is VLAN-300, and you enabled IGMP snooping (but not MLD snooping):

    • show evpn multicast-snooping status—See if you enabled IGMP snooping on VLANs you configured for OISM elements.

      In the following case, if the revenue BDs on a server leaf device are VLAN-100 and VLAN-200, the SBD is VLAN-300, and you enabled IGMP snooping (but not MLD snooping):