Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
 

Optimized Intersubnet Multicast in EVPN Networks

SUMMARY Enable intersubnet multicast (OISM) to optimize multicast traffic routing and forwarding in an EVPN edge-routed bridging (ERB) overlay fabric. With OISM, your network can also support multicast traffic flow between devices inside and outside of the EVPN fabric.

Overview of OISM

Optimized intersubnet multicast (OISM) is a multicast traffic optimization feature. This feature operates at Layer 2 and Layer 3 in EVPN-VXLAN edge-routed bridging (ERB) overlay fabrics. The design is based on the IETF draft specification https://datatracker.ietf.org/doc/html/draft-ietf-bess-evpn-irb-mcast. You can apply OISM configuration and operation to multicast traffic but not to broadcast or unknown unicast traffic.

In EVPN ERB overlay fabric designs, the leaf devices in the fabric route traffic between tenant bridge domains (that is, between VLANs). When you enable OISM, the leaf devices route intersubnet multicast traffic locally through IRB interfaces using the control plane multicast states. With local routing between VLANs, the receiver IRB interface doesn't send the routed multicast traffic out into the EVPN core. The local routing model helps minimize the traffic load within the EVPN core. It also avoids traffic hairpinning.

OISM leaf devices also selectively forward traffic into the EVPN core only toward other EVPN devices with interested receivers. Selective forwarding further improves multicast traffic performance in the EVPN fabric.

With OISM enabled, ERB overlay fabrics can efficiently and effectively support multicast traffic flow between devices inside and outside the EVPN fabric. Without OISM, fabric designers must use the centrally routed bridging (CRB) overlay model to support multicast with external sources or receivers. OISM border leaf devices support different methods to route traffic to and from an external PIM domain. These methods use either integrated routing and bridging (IRB) interfaces or Layer 3 (L3) interfaces. The border leaf devices also employ a supplemental bridge domain (SBD) inside the fabric to carry the traffic from external sources toward receivers within the EVPN fabric.

Benefits of OISM

  • Enables EVPN-VXLAN fabrics with the ERB overlay model to support multicast traffic with sources and receivers outside of the fabric.
  • Minimizes multicast control packets and replicated data packets in the EVPN fabric core to optimize fabric multicast performance in scaled designs.

OISM Support in EVPN Instances

We support OISM in EVPN-VXLAN fabrics in the following types of EVPN instances:

  • EVPN in the default switch instance:

    • Starting in Junos OS Release 21.2R1 on QFX5110, QFX5120, and QFX10002 (except QFX10002-60C) switches.

    • Starting in Junos OS Release 22.2R1 on EX4650, QFX10008, and QFX10016 switches.

    • Starting in Junos OS Release 22.3R1 on EX4300-48MP and EX4400 switches.

  • MAC-VRF EVPN routing instances with vlan-aware and vlan-based service types only (see MAC-VRF Routing Instance Type Overview):

    • Starting in Junos OS Evolved Release 22.1R1 on QFX5130-32CD and QFX5700 switches.

    • Starting in Junos OS Release 22.2R1 on EX4650, QFX5110, QFX5120, QFX10002 (except QFX10002-60C), QFX10008, and QFX10016 switches.

    • Starting in Junos OS Evolved Release 22.3R1 on PTX10001-36MR, PTX10004, PTX10008, and PTX10016 routers.

On Junos OS Evolved devices, we support EVPN-VXLAN using EVPN configurations with MAC-VRF instances only, and not in the default switch instance. As a result, on these devices we support OISM only in MAC-VRF EVPN instances.

OISM with Multicast Protocols and Other Multicast Optimizations in EVPN Fabrics

OISM works with the following multicast protocols and other EVPN multicast optimization features:

Multicast Protocols Supported with OISM

  • IGMPv2:

    • Starting in Junos OS Release 21.2R1 on QFX5110, QFX5120, and QFX10002 (except QFX10002-60C) switches.

    • Starting in Junos OS Evolved Release 22.1R1 on QFX5130-32CD and QFX5700 switches.

    • Starting in Junos OS Release 22.2R1 on EX4650, QFX10008, and QFX10016 switches.

    • Starting in Junos OS Release 22.3R1 on EX4300-48MP and EX4400 switches.

    • Starting in Junos OS Evolved Release 22.3R1 on PTX10001-36MR, PTX10004, PTX10008, and PTX10016 routers.

  • IGMPv3:

    • Starting in Junos OS Evolved Release 22.1R1 on QFX5130-32CD and QFX5700 switches.

    • Starting in Junos OS Release 22.2R1 on EX4650, QFX5110, QFX5120, QFX10002 (except QFX10002-60C), QFX10008, and QFX10016 switches.

    • Starting in Junos OS Evolved Release 22.3R1 on PTX10001-36MR, PTX10004, PTX10008, and PTX10016 routers.

  • PIM, which facilitates both local routing and external multicast traffic routing.

OISM supports IGMP snooping with both IGMPv2 and IGMPv3 on the same device at the same time only under certain configuration constraints. See IGMPv2 and IGMPv3 with IGMP Snooping and OISM in the Same EVPN-VXLAN Fabric for details.

Also see Supported IGMP or MLD Versions and Group Membership Report Modes for information on IGMP any-source multicast (ASM) and source-specific multicast (SSM) mode support with IGMP in EVPN-VXLAN fabrics.

Other Multicast Optimization Features That Work with OISM

OISM works with these other multicast optimization features:

  • IGMP snooping on the access side on the leaf devices.

    With IGMP snooping enabled, a leaf device that receives multicast traffic forwards it only toward other devices with interested receivers.

  • Multihoming support in an Ethernet segment using EVPN Type 7 (Join Sync) and Type 8 (Leave Sync) routes.

    EVPN fabric devices advertise these route types to synchronize the multicast state among EVPN devices that are multihoming peers.

  • Selective multicast Ethernet tag (SMET) forwarding in the EVPN fabric core using EVPN Type 6 routes.

    EVPN devices use Type 6 routes to limit forwarding within the EVPN core only to receivers interested in receiving traffic for a multicast group. You can use OISM to make this optimization work in EVPN ERB overlay fabrics. When you configure IGMP snooping, the fabric enables SMET forwarding with OISM automatically.

  • Assisted replication (AR).

    You can integrate AR into a fabric running OISM as follows, depending on the platforms that support the different AR and OISM device roles:

    • Starting in Junos OS Release 22.2R1 on EX4650, QFX5110, QFX5120, QFX10002 (except QFX10002-60C), QFX10008, and QFX10016 switches:

      • You can configure the AR leaf role on any of these devices that are also acting as OISM border leaf or server leaf devices.

      • You can configure only QFX10002 (except QFX10002-60C), QFX10008, and QFX10016 switches as AR replicators, in either of these modes:

        Collocated mode: The device acts as both an AR replicator device and an OISM border leaf device.

        Standalone mode: The device is an AR replicator but isn't also an OISM border leaf or server leaf device.

    • Starting in Junos OS Evolved Release 22.2R1 on QFX5130-32CD and QFX5700 switches:

      • You can configure the AR leaf role on OISM border leaf or server leaf devices.

      • You can configure these devices as AR replicators with OISM in standalone mode only. In standalone mode, the AR replicator device doesn't also operate as an OISM border leaf or server leaf.

    See Assisted Replication Multicast Optimization in EVPN Networks for how AR works.

    See AR with Optimized Intersubnet Multicast (OISM) for more on integrating AR with OISM.

OISM Components

The OISM environment includes:

  • Leaf devices in the EVPN fabric that function in border roles and server access roles.

  • External multicast sources and receivers in an external L3 PIM domain.

  • Bridge domain (VLAN) configurations that enable the fabric to route multicast traffic between internal and external devices.

The EVPN-VXLAN ERB overlay design includes lean spine devices that support L3 transit functions for the leaf devices. The lean spine devices don't usually perform any OISM functions.

The following sections describe these OISM components.

OISM Device Roles

Figure 1 shows a simple EVPN-VXLAN ERB overlay fabric and the OISM device roles in the fabric.

Figure 1: EVPN Fabric with OISM EVPN Fabric with OISM

Table 1 summarizes the device roles.

Table 1: EVPN Fabric OISM Device Roles
Device Role Description

Border leaf (BL)

OISM leaf devices in the EVPN fabric underlay and overlay. Border leaf devices function as gateways interconnecting the EVPN fabric to multicast devices (sources and receivers) outside the fabric in an external PIM domain. These devices serve in the PIM EVPN gateway (PEG) role.

Lean spine (LS)

Spine devices in the underlay of the EVPN fabric. These devices usually operate as lean spines that support the EVPN underlay as IP transit devices. The lean spines might also act as route reflectors in the fabric.

You configure OISM elements on the lean spine devices only in the following use cases:

  • The devices also serve as border devices for external multicast traffic. In this case, you configure the same OISM elements as you configure on border leaf devices.

  • The lean spine device serves as a standalone AR replicator when you integrate AR with OISM in the fabric. In this case, on the AR replicator spine device, you configure the same common OISM elements that you configure on the border leaf and server leaf devices. You don't need to configure any of the PIM or external multicast elements specific to border leaf or server leaf devices. (See AR with Optimized Intersubnet Multicast (OISM).)

Server leaf (Leaf)

OISM leaf devices on the access side in the EVPN fabric underlay and overlay. Server leaf devices are often top-of-rack (ToR) switches. These devices connect the EVPN fabric to multicast sources and multicast receiver hosts on bridge domains or VLANs within the fabric.

See Configuration Elements for OISM Devices (Symmetric Bridge Domains) for details on the configuration elements that are common and those that are different for each device role.

PIM Domain with External Multicast Sources and Receivers

In Figure 1, the OISM border leaf devices connect to multicast sources and receivers outside the EVPN fabric in a representative external PIM domain. The multicast devices in the external PIM domain follow standard PIM protocol procedures; their operation is not specific to OISM. External multicast traffic flows at Layer 3 through the PIM domain.

You can use OISM to route and to forward multicast traffic in an EVPN-VXLAN ERB overlay fabric between devices in the following use cases:

  • Internal multicast sources and receivers

  • Internal multicast sources and external multicast receivers

  • External multicast sources and internal multicast receivers

For simplicity, in this documentation we represent the external PIM domain as:

  • A PIM router (a device such as an MX Series router) that doubles as the PIM rendezvous point (RP).

  • An external source.

  • An external receiver.

Supported Methods for Multicast Data Transfer to or from an External PIM Domain

OISM border leaf devices support one or more methods to route multicast traffic to and from devices outside of the fabric. Supported methods are platform-dependent.

Some platforms don't support the border leaf role. If you don't see a platform listed in Table 2 in the Supported Platforms column for any of the external multicast methods, that means the platform doesn't support the border leaf role.

Table 2: External Multicast Connection Methods
Name Connection Method Supported Platforms

M-VLAN IRB method

IRB interfaces on a multicast VLAN (M-VLAN) that you extend in the EVPN instance. The fabric uses the M-VLAN and corresponding IRB interfaces only for external multicast traffic flow to and from the external PIM domain.

This method supports EVPN Ethernet segment identifier (ESI) multihoming to connect the external PIM router to more than one OISM border leaf device in the fabric.

PTX10001-36MR

PTX10004

PTX10008

PTX10016

QFX10002 (except QFX10002-60C)

QFX10008

QFX10016

Classic L3 interface method

Classic physical L3 interfaces on OISM border leaf devices that connect individually to the external PIM domain on different subnets.

These interfaces aren't associated with a VLAN. You don't configure these interfaces in the EVPN instances. Instead, you assign IP addresses to these interfaces and include them in the tenant L3 virtual routing and forwarding (VRF) instances.

Note:

The L3 interface connection can be an aggregated Ethernet (AE) interface bundle.

EX4650

PTX10001-36MR

PTX10004

PTX10008

PTX10016

QFX5120

QFX5130-32CD

QFX5700

QFX10002 (except QFX10002-60C)

QFX10008

QFX10016

Non-EVPN IRB method

IRB interfaces on an extra VLAN that you don't extend in the EVPN instance. You include these logical interfaces in the tenant L3 VRF instances.

On each border leaf device, you assign a unique extra VLAN ID and subnet for the associated IRB interface.

We call this type of interface a non-EVPN IRB interface for external multicast.

PTX10001-36MR

PTX10004

PTX10008

PTX10016

QFX5130-32CD

QFX5700

See Overview of Multicast Forwarding with IGMP Snooping or MLD Snooping in an EVPN-VXLAN Environment for a general overview of connecting EVPN-VXLAN fabrics to an external PIM domain using a Layer 2 M-VLAN or L3 links.

OISM Bridge Domains (VLANs)

Table 3 summarizes the OISM bridge domains or VLANs and describes how OISM uses them.

Note:

References in this document to all OISM devices correspond to the border leaf and server leaf devices on which you enable OISM.

Table 3: OISM Bridge Domains or VLANs
Bridge Domain/VLAN Description Configure On:

Multicast VLAN (M-VLAN)

(M-VLAN IRB method for external multicast) A VLAN in the EVPN fabric with associated IRB interfaces that connect the fabric to an external multicast router. This VLAN and IRB interface enable traffic flow between devices inside and outside the fabric. To support IGMP snooping with both IGMPv2 and IGMPv3 traffic, you assign separate M-VLANs to carry traffic for each IGMP version.

You extend this VLAN in the EVPN instance. You can also multihome the external multicast router to multiple border leaf device M-VLAN IRB interfaces in the same EVPN Ethernet segment. The usual EVPN multihoming designated forwarder (DF) rules apply to send only one copy of the traffic in the Ethernet Segment on the M-VLAN.

Configure the M-VLAN as a VLAN that is not the SBD or any of the revenue bridge domains in the EVPN fabric.

Note:

See Supported Methods for Multicast Data Transfer to or from an External PIM Domain for other supported methods to connect to external sources and receivers.

Border leaf devices

Extra non-EVPN VLAN

(Non-EVPN IRB method for external multicast) An extra VLAN that isn't in the EVPN instances in the fabric. You configure associated IRB interfaces in the tenant L3 VRF instances. This VLAN and IRB interface enable multicast traffic flow between devices inside the fabric and devices outside the fabric.

You must assign distinct extra VLANs and corresponding IRB interface subnets on each border leaf device that are unique across the fabric. Also, to support IGMP snooping with both IGMPv2 and IGMPv3 traffic, use separate distinct extra non-EVPN VLANs to carry traffic for each IGMP version.

Note:

See Supported Methods for Multicast Data Transfer to or from an External PIM Domain for other supported methods to connect to external sources and receivers.

Border leaf devices

Revenue bridge domains (VLANs)

Bridge domains for subscribers to the services that the fabric provides. You configure the revenue bridge domains as VLANs in the fabric.

The revenue bridge domains correspond to the customer VLANs in the fabric. These VLANs are not specific to OISM, but the multicast sources and receivers in the fabric are in these bridge domains.

See IGMPv2 and IGMPv3 with IGMP Snooping and OISM in the Same EVPN-VXLAN Fabric for details on how to allocate VLANs for these bridge domains if you want to support IGMP snooping with both IGMPv2 and IGMPv3 receivers in the fabric.

All OISM devices

Supplemental bridge domain (SBD)

Bridge domain that enables support for external multicast traffic and implements SMET optimization in the EVPN core.

You configure the SBD as a regular VLAN that is different from the revenue bridge domains, the M-VLAN or the extra non-EVPN VLANs.

SBD usually serves all OISM leaf devices in the fabric. To support IGMP snooping with both IGMPv2 and IGMPv3 traffic, you assign separate SBDs to carry traffic for each IGMP version. The SBD carries:

  • Routed multicast data traffic from external multicast sources to EVPN devices in the fabric with interested receivers.
  • SMET EVPN Type 6 routes from originating leaf devices to other EVPN leaf devices, which enables selective forwarding in the EVPN core.
Note:

The SBD is central to OISM operation for external multicast traffic routing and forwarding. The SBD IRB interface must always be up for OISM to work.

All OISM devices

The OISM implementation uses a symmetric bridge domains model. The following figure shows how you set up the bridge domains and VLANs in the EVPN fabric on the OISM devices with this model.

Figure 2: EVPN Fabric with OISM Symmetric Bridge Domains Model EVPN Fabric with OISM Symmetric Bridge Domains Model

In the symmetric bridge domains model, you must configure the SBD and all revenue bridge domains on all OISM border leaf and server leaf devices.

You also configure the following VLANs uniformly on the border leaf devices that connect to the external PIM domain:

  • The M-VLAN, if you use the M-VLAN IRB method for external multicast.

  • A unique extra non-EVPN VLAN on each border leaf device, if you use the non-EVPN IRB method for external multicast.

The lean spine devices in the fabric usually serve only as IP transit devices and possibly as route reflectors. As a result, you don't usually need to configure these elements on the lean spine devices. (See the Lean Spine row in Table 1 for some exceptions.)

Configuration Elements for OISM Devices (Symmetric Bridge Domains)

This section summarizes the elements you need to configure on:

  • All OISM devices—Devices in the border leaf role and the server leaf role, and lean spine devices that also serve as AR replicators in standalone mode when you integrate AR with OISM.

    See Table 4.

  • Server leaf devices only.

    See Table 5.

  • Border leaf devices only, based on the method you use for external multicast:

    • M-VLAN IRB interface

    • Classic L3 interface

    • Non-EVPN IRB interface

    See Table 6.

Some elements are optional, which the description notes.

Note:

EX4650, QFX5110, and QFX5120 switches support enterprise style interface configurations for OISM elements, but not service provider style interface configurations. For more information on these interface configuration styles, see Flexible Ethernet Services Encapsulation and Understanding Flexible Ethernet Services Support With EVPN-VXLAN.

Table 4 lists the elements you configure on all OISM devices.

Table 4: Configuration Elements on all OISM Devices
Configuration Element Description

OISM

Enable OISM globally, and enable OISM routing functions in L3 VRF instances.

A device with OISM enabled advertises EVPN Type 3 inclusive multicast Ethernet tag (IMET) routes as follows:

  • The device advertises an IMET route for each configured revenue bridge domain (VLAN) and the SBD.

  • The IMET routes include the EVPN multicast flags extended community with a flag indicating OISM support.

Revenue bridge domains (customer VLANs) and corresponding IRB interfaces

Configure revenue bridge domains (customer VLANs) according to your data center services requirements. You must configure all revenue bridge domain VLANs and corresponding IRB interfaces on all OISM devices. See OISM Bridge Domains (VLANs) for more information.

See IGMPv2 and IGMPv3 with IGMP Snooping and OISM in the Same EVPN-VXLAN Fabric for special considerations to configure revenue bridge domains to support IGMP snooping with traffic for both IGMPv2 and IGMPv3 receivers.

SBD (VLAN) and corresponding IRB interface

Configure the SBD and its IRB interface on all OISM devices. The SBD can be any VLAN that is distinct from the M-VLAN, any of the non-EVPN VLANs or any revenue bridge domain VLANs in the EVPN fabric. See OISM Bridge Domains (VLANs) for more information.

You identify this VLAN as the SBD in the L3 VRF instance that supports OISM routing. EVPN IMET routes for the SBD IRB interface include the OISM SBD flag in the multicast flags extended community.

Layer 3 multicast protocols (IGMPv2 or v3)

Enable IGMPv2 or IGMPv3 L3 multicast protocols. Receivers send IGMP reports to express interest in receiving traffic for a multicast group.

You can use IGMPv2 or IGMPv3 in any-source multicast (ASM) mode, or IGMPv3 in source-specific multicast (SSM) mode.

Note that you can't enable IGMP snooping for both IGMP versions together for the same VLAN or in the same VRF instance with OISM enabled. However, to support IGMP snooping with IGMPv2 and IGMPv3 receivers in the same fabric, you can enable IGMP snooping with IGMPv2 for specific VLANs in one VRF instance, and enable IGMP snooping with IGMPv3 for other VLANs in another VRF instance. See IGMPv2 and IGMPv3 with IGMP Snooping and OISM in the Same EVPN-VXLAN Fabric for details.

IGMPv2 is the default IGMP version. For IGMPv3, you must specify the version 3 option.

Layer 2 multicast optimizations—IGMP snooping and SMET

Enable IGMP snooping at L2 with IGMPv2 or IGMPv3 protocols as part of the optimizations OISM provides. With IGMP snooping, the device routes or forwards multicast traffic only toward interested access-side receivers. Receivers send IGMP reports to express interest in receiving traffic for a multicast group.

When you enable IGMP snooping, the device also automatically advertises SMET Type 6 routes. With SMET, the device sends copies of the traffic into the EVPN core only toward other devices that have interested receivers.

Configure IGMP snooping as follows:

  • On server leaf and border leaf devices in the EVPN instance(s) for each OISM revenue VLAN and the SBD.

  • On border leaf devices for external multicast, only with the M-VLAN IRB method or the non-EVPN IRB method, as follows:

    • M-VLAN IRB method: In the EVPN instance(s) for the M-VLAN IRB multicast router interface. Include the proxy mode option with IGMPv2. Include the evpn-ssm-reports-only option only with IGMPv3.

    • Non-EVPN IRB method: Globally for the non-EVPN IRB interface configured as a multicast router interface.

      With IGMPv2 (and IGMPv3, if desired), include the proxy mode option. With IGMPv3, you don't need the evpn-ssm-reports-only option because the external multicast interface isn't extended in the EVPN instance.
    Note:

    With the classic L3 interface method for external multicast, you don't configure IGMP snooping, an L2 optimization, on the pure L3 interfaces.

Layer 3 VRF instance

Configure a routing instance (instance type vrf) that supports the L3 OISM routing functions.

originate-smet-on-revenue-vlan-too

(Optional) Enable the device to originate SMET Type 6 routes for revenue bridge domains (as well as on the SBD) upon receiving local IGMP reports. By default, OISM devices originate Type 6 routes only on the SBD.

Use this option for compatibility with other vendor devices that don't support OISM. Those devices can't create the right states for the revenue bridge domain VLANs upon receiving Type 6 routes on the SBD.

install-star-g-routes

(Required on the QFX10000 line of switches and PTX10001-36MR, PTX10004, PTX10008, and PTX10016 routers; required on QFX5130-32CD and QFX5700 switches when you configure these switches with OISM and AR in the AR replicator role; optional but not generally recommended on EX4300-48MP, EX4400, EX4650, QFX5110 and QFX5120 switches, or on QFX5130-32CD and QFX5700 switches without the AR replicator role)

Enable the Routing Engine on the device to install (*,G) multicast routes on the Packet Forwarding Engine for all of the revenue bridge domain VLANs in the routing instance immediately upon receiving an EVPN Type 6 route. Setting this option helps minimize traffic loss when multicast traffic first arrives.

See Latency and Scaling Trade-Offs for Installing Multicast Routes with OISM (install-star-g-routes Option) for details on how and when to set this option.

Table 5 lists the elements you configure on the server leaf devices.

Table 5: Configuration Elements on OISM Server Leaf Devices
Configuration Element Description

PIM in passive mode on all revenue bridge domains and the SBD

Configure this mode to facilitate local routing without all of the traditional PIM protocol functions. The server leaf device:

  • Doesn't form PIM neighbor relationships with any other devices (avoids sending or receiving PIM hello messages).

  • Acts as a PIM local RP. The device creates the PIM state locally from IGMP reports. The device also doesn't do source registration. Only the border leaf devices perform source registration toward the external PIM RP.

PIM with the accept-remote-source option on SBD IRB interfaces

This option enables an SBD IRB interface to accept multicast traffic from a source that isn't on the same subnet. The server leaf devices require this setting because:

  • The border leaf devices route the multicast source traffic from the external multicast interfaces to the SBD toward the server leaf devices.

  • The traffic arrives at the server leaf device on the SBD IRB interface, which is not a PIM neighbor. The source is not a local source, so without this setting, the device would otherwise drop the traffic.

OSPF for peer connectivity to support external multicast traffic on SBD

Configure OSPF so the server leaf devices learn routes to external sources when multicast traffic from outside the fabric arrives on the SBD. The device creates the PIM (S,G) entries it needs to forward the traffic from the SBD to the revenue bridge domains.

On the server leaf devices, you configure OSPF on the device loopback interface (lo0) and the SBD IRB interface.

Table 6 lists the elements you configure on the border leaf devices based on the external multicast method you use.

Table 6: Configuration Elements on OISM Border Leaf Devices
Configuration Element External Multicast Method Description

M-VLAN and corresponding IRB interface (in EVPN instance)

M-VLAN IRB method

Configure a VLAN to serve as the M-VLAN, and extend this VLAN in the EVPN instance. This VLAN must be distinct from the SBD or any revenue bridge domain VLANs in the EVPN fabric. Also configure an M-VLAN IRB interface in the EVPN instance. See OISM Bridge Domains (VLANs) for more information about the M-VLAN.

You can link multiple border leaf device M-VLAN IRB interfaces to the external multicast router in the same EVPN Ethernet segment. The usual EVPN multihoming DF rules apply to prevent sending duplicate traffic on the M-VLAN.

Layer 2 multicast router interface on external multicast ports

M-VLAN IRB method or non-EVPN IRB method

Configure the multicast-router-interface option with IGMP snooping on the L2 ports that link the border leaf device to the external PIM domain at L2.

With the M-VLAN IRB method, these interfaces support multicast traffic when the external domain router is multihomed to the border leaf devices. As a result, multihomed M-VLAN use cases require this configuration. This setting is also required with the non-EVPN IRB method.

PIM on M-VLAN IRB interface

M-VLAN IRB method

Configure PIM in distributed designated router (DR) mode (distributed-dr) or standard PIM mode on an M-VLAN IRB interface. We recommend using distributed DR mode in most cases, especially on border leaf devices where the external PIM router is multihomed to multiple border leaf devices.

The device uses PIM to:

  • Form PIM neighbor relationships with the other border leaf devices and the external PIM router. The external PIM router might be multihomed in an Ethernet segment. As a result, EVPN-forwarded traffic might come to a different peer border leaf device.

  • Elect a single last-hop router (LHR) designated router (DR) for the M-VLAN, so only one device does source registration for an internal source toward the PIM RP.

You configure PIM on the M-VLAN IRB interface in the tenant VRF instances. similar to how you configure PIM on the revenue bridge domains.

L3 physical interface with IP address

Classic L3 interface

Configure a physical L3 interface with an IP address for external multicast that connects the border leaf device to the external PIM domain at L3.

Define the external multicast L3 interface in a different subnet on each border leaf device.

Note:

The L3 interface connection can be an AE interface bundle.

PIM on the logical interface for the external multicast physical L3 interface

Classic L3 interface

Configure the logical interface (unit 0) for the external multicast L3 interface in the tenant VRF instances. Configure standard PIM mode on the logical interface.

With this setting, the border leaf device forms a PIM neighbor relationship with the external PIM router to send join messages and transmit or receive external multicast traffic.

Extra VLAN and corresponding IRB interface (not in EVPN instance)

Non-EVPN IRB method

Configure an extra VLAN and IRB interface globally for external multicast without EVPN signaling. This VLAN and the IRB interface subnet must be distinct from the SBD, any revenue bridge domain VLANs, or the extra VLAN on any other border leaf device in the EVPN fabric. See OISM Bridge Domains (VLANs) for more about this extra VLAN and external multicast method.

PIM on non-EVPN IRB interface

Non-EVPN IRB method

Configure PIM on the non-EVPN IRB interface in the tenant VRF instances.

With this setting, the border leaf device forms a PIM neighbor relationship with the external PIM router to send join messages and transmit or receive external multicast traffic.

PIM on SBD IRB interface

All

Configure standard PIM mode on the SBD IRB interface in the tenant VRF instances for SBD routing and forwarding. With this setting, the border leaf device:

  • Routes external multicast source traffic from the external multicast interfaces to the SBD, and forwards copies toward server leaf devices with multicast receivers.

  • Forms PIM neighbor relationships with the other border leaf devices.

  • Elects a single LHR DR on the SBD among the peer border leaf devices. Only this elected PIM DR forwards the external multicast source traffic on the SBD. This election prevents peer border leaf devices from forwarding duplicate traffic into the EVPN core.

PIM EVPN gateway (PEG) role

All

(include the external IRB option for EVPN, external-irb interface-name, only with the M-VLAN IRB method)

Configure the pim-evpn-gateway role on the border leaf device to connect to the external PIM router. In this role, the border leaf device uses traditional PIM routing behavior and does local routing, as follows:

For externally-sourced traffic:

  • Routes external source traffic from the M-VLAN IRB interface, L3 interface, or non-EVPN IRB interface to any local receivers on the revenue bridge domains on the border leaf device.

  • Routes external source traffic from the M-VLAN IRB interface, L3 interface, or non-EVPN IRB interface to the SBD to reach any internal receivers in the fabric. (The device forwards the traffic to the EVPN core only on the SBD to other OISM leaf devices.)

For internally-sourced traffic:

  • Locally routes traffic that arrives on a revenue bridge domain to other revenue bridge domains.

  • Routes the traffic to external multicast receivers through the M-VLAN IRB interface, L3 interface, or non-EVPN IRB interface. The device doesn't forward the traffic back into the EVPN core.

EVPN IMET routes for PEG interfaces include the OISM PEG flag in the multicast flags extended community field of the route.

OSPF for external multicast interface peer connectivity

All

Configure an OSPF area in the tenant L3 VRF instance so the border leaf device learns routes to the multicast sources. The device requires these routes to support forwarding multicast traffic:

  • From sources inside the fabric toward receivers outside the fabric.

  • From sources outside the fabric toward receivers inside the fabric.

The device needs this route information to create the PIM (S,G) entries to forward the traffic on the external multicast interfaces, the SBD, and the revenue bridge domains.

On the border leaf devices, you configure OSPF on the device loopback interface, the SBD IRB interface, and the external multicast interface (M-VLAN IRB interface, L3 interface, or non-EVPN IRB interface).

PIM distributed DR mode on revenue bridge domain IRB interfaces

All

Configure PIM in distributed DR mode (distributed-dr) on the revenue bridge domain IRB interfaces in the tenant VRF instances. In this mode, the border leaf device:

  • Forms PIM neighbor relationships with other PIM devices to support multihomed external PIM router connections.

  • Acts as the LHR DR on its revenue bridge domain IRB interfaces, and creates the PIM state locally from received IGMP reports. As a result:

    • The device can do local multicast routing between the revenue bridge domains.

    • One peer border leaf device becomes the PIM DR that performs source registration toward the PIM RP. PIM hello messages and the PIM DR election process determine the PIM DR.

PIM accept-join-always-from option and policy on M-VLAN IRB interface M-VLAN IRB method

Set this option on the M-VLAN IRB interface in the tenant VRF instances when the external PIM router is multihomed to more than one EVPN border leaf device. With this option, the device can accept and install the same PIM (S,G) join states on multihoming peer border leaf devices. This option supports sending multicast traffic from sources inside the fabric to receivers in the external PIM domain.

With multihoming on the M-VLAN, the usual EVPN multihoming DF rules apply in an Ethernet segment to prevent sending duplicate traffic. If peer border leaf devices have the same valid join states in place, any device that is the EVPN DF can forward the multicast traffic.

Configure this statement with policies that specify the interface should always install PIM joins from upstream neighbor addresses that correspond to the external PIM router.

Note:

You don't use this option with the classic L3 interface and non-EVPN IRB methods. Those methods don't extend the external multicast interfaces in the EVPN instance.

See the following sections for more details on configuring OISM devices:

How OISM Works

The following sections describe how OISM works and show how the multicast traffic flows in several common use cases.

Local Routing on OISM Devices

In Figure 3, we illustrate how local routing and forwarding works in general on OISM devices. As the figure shows, OISM local routing forwards the traffic on the source VLAN. Each leaf device routes the traffic locally to its receivers on other VLANS, which avoids hairpinning for intersubnet routing on the same device.

Figure 3: Local Routing with OISM Local Routing with OISM

In this case, the source traffic comes from Mcast-Src-1 on VLAN-1, the blue VLAN. Server leaf devices use IRB interfaces and PIM in passive mode to route traffic between VLANs. With PIM in passive mode, server leaf devices:

  • Don’t become PIM neighbors with the other leaf devices.

  • Act as a local PIM RP, create local PIM state upon receiving IGMP reports, and avoid doing source registration.

As a result, the server leaf devices forward and route multicast traffic within the fabric as follows to receivers interested in the multicast group:

  • The ingress leaf device (Leaf-1) forwards the traffic on the source VLAN into the EVPN fabric toward the other leaf devices with interested receivers.

  • All of the server leaf devices don't need to forward the traffic back into the EVPN core to another device that is a designated router. Server leaf devices can locally:

    • Forward the traffic on the source VLAN toward local interested receivers on the source VLAN.

    • Route the traffic from the source VLAN through the IRB interfaces toward local interested receivers in other VLANs.

Multicast Traffic Forwarding and Routing with Source and Receivers Inside the EVPN Data Center

When the multicast source is inside the EVPN fabric, the server leaf devices receive the multicast traffic on the source VLAN. Then they locally route or forward the traffic as described in Local Routing on OISM Devices.

The following figure illustrates OISM local routing and forwarding within an EVPN fabric in detail. The figure also shows how local routing works with EVPN multihoming for a multicast receiver.

Figure 4: OISM with an Internal Multicast Source and Internal Multicast Receivers OISM with an Internal Multicast Source and Internal Multicast Receivers

In Figure 4, the multicast source, Mcast-Src-1, is single-homed to Leaf-1. The source VLAN is VLAN-1 (the blue VLAN). Multicast control and data traffic flow proceeds as follows:

  1. Receivers on all three server leaf devices sent IGMP reports (join messages) expressing interest in receiving the traffic for a multicast group.

  2. Leaf-1 forwards the traffic on the source VLAN to both Leaf-2 and Leaf-3 because both leaf devices have interested receivers. In this case, the receivers on Leaf-2 and Leaf-3 use single-homing.

  3. Leaf-2 and Leaf-3 forward or locally route the traffic to their interested receivers (Rcvr-2, Rcvr-3, and Rcvr-4) as described in Local Routing on OISM Devices.

  4. Rcvr-1 on VLAN-2 is multihomed to Leaf-1 and Leaf-2 in an EVPN Ethernet segment. Rcvr-1 has expressed interest in receiving the multicast traffic, so:

    • Both server leaf devices, Leaf-1 and Leaf-2, receive the IGMP report.
    • Both Leaf-1 and Leaf-2 locally route the traffic from the source VLAN (VLAN-1) because each device has the PIM passive mode configuration.
    • However, because Leaf-1 is the DF for the EVPN Ethernet segment, only Leaf-1 forwards the traffic to Rcvr-1.
  5. The border leaf devices receive the multicast traffic through the EVPN fabric on the source VLAN. Note that the border leaf devices could have local receivers, although we don't show that case. With local receivers, the device also locally routes or forwards the traffic to those receivers the same way the server leaf devices do.

Figure 4 also shows that the border leaf devices locally route the traffic from the source VLAN toward any external multicast receivers in the external PIM domain. See Supported Methods for Multicast Data Transfer to or from an External PIM Domain for the available external multicast methods by platform. Later sections describe the multicast control and data traffic flow for external source and external receiver use cases.

Multicast Traffic From an Internal Source to Receivers Outside the EVPN Data Center—M-VLAN IRB Method

In Figure 5, we illustrate the OISM use case where a multicast source inside the EVPN fabric sends multicast traffic to an interested receiver outside the fabric using the M-VLAN IRB method for external multicast. (Supported Methods for Multicast Data Transfer to or from an External PIM Domain lists external multicast method support by platform.)

The border leaf devices you configure in the OISM PEG role receive the multicast traffic on the source VLAN through the EVPN core. Then the border leaf devices replicate the traffic and route it onto the M-VLAN toward the external PIM domain to reach the external receiver.

Note:

PEG border leaf devices only send multicast source traffic received on the revenue bridge domains to the M-VLAN. These devices don't forward the traffic back into the EVPN core toward the other border leaf devices.

This use case also shows an internal multihomed multicast source and local routing to single-homed receivers inside the fabric.
Figure 5: OISM with an Internal Multicast Source and an External Multicast Receiver—M-VLAN IRB Method OISM with an Internal Multicast Source and an External Multicast Receiver—M-VLAN IRB Method

In Figure 5, the internal source for a multicast group is Mcast-Src-2, the same device as Rcvr-1, which is multihomed to Leaf-1 and Leaf 2. The source sends the multicast traffic on VLAN-2. The external receiver, Ext-Mcast-Rcvr, expresses interest in receiving the multicast traffic for that multicast group (sends a join message). Internal receivers Rcvr-3 (on VLAN-1) and Rcvr-4 (on VLAN-2) also request to join the multicast group and receive the traffic.

Note that the PIM router is multihomed to both BL-1 and BL-2, the PEG devices, in the EVPN fabric. Those connections are in the same Ethernet segment; the DF election process chooses one of these devices as the DF for the Ethernet segment. Only the DF will forward traffic (on the M-VLAN) toward external receivers.

The source traffic reaches the interested internal and external receivers as follows:

Traffic Flow from Multihomed Source to Internal Receivers

These steps summarize the multicast control and data traffic flow from the multihomed source to the internal receivers:

  1. Mcast-Src-2 (also labeled Rcvr-1) originates the traffic on VLAN-2, the red VLAN. Because the device is multihomed to Leaf-1 and Leaf-2, the device hashes the traffic on VLAN-2 to one of those server leaf devices. In this case, Leaf-2 receives the traffic.

  2. The red arrows show that Leaf-2 forwards the traffic on the source VLAN, VLAN-2, only to:

    • The other server leaf devices with interested receivers—In this case, only Leaf-3.

    • The border leaf devices, which both act in the OISM PEG role.

    Note that no receivers behind Leaf-1 or Leaf-2 sent an IGMP report to join the multicast group. With IGMP snooping and SMET forwarding enabled, Leaf-2 doesn't forward the traffic to Leaf-1 because Leaf-1 has no interested receivers. Leaf-2 also doesn't locally route the traffic to Rcvr-2 for the same reason.

  3. Leaf-3 receives the source traffic on VLAN-2. Then Leaf-3 routes the traffic locally to VLAN-1 to Rcvr-3. Leaf-3 also forwards the traffic to Rcvr-4 on VLAN-2.

  4. Both border leaf devices BL-1 and BL-2 also receive the source traffic from the EVPN core. We describe the external multicast flow next.

Traffic Flow to External Receiver—M-VLAN IRB Method

These steps summarize the multicast control and data traffic flow in Figure 5 from the border leaf devices toward the external receiver using the M-VLAN IRB method:

  1. In the external PIM domain, the PIM RP enters a PIM (*,G) multicast routing table entry. The entry includes the L3 interface toward Ext-Mcast-Rcvr as the downstream interface.

  2. Both border leaf devices BL-1 and BL-2 receive the source traffic from the EVPN core. The IRB interface on VLAN-2 on one of these border leaf devices is the PIM DR for VLAN-2. In this case, the PIM DR is on BL-1, so BL-1 sends a PIM Register message toward the PIM RP on the M-VLAN IRB interface.

  3. The PIM RP sends a PIM Join message back toward BL-1. BL-1 creates an (S,G) multicast routing table entry as follows:

    • The source address is the IP address of Mcast-Src-2 in VLAN-2.
    • The downstream interface is the M-VLAN IRB interface.
  4. Both BL-1 and BL-2 are PEG devices and configured in PIM distributed DR mode for the revenue bridge domain (VLAN-1 and VLAN-2) IRB interfaces. As a result, both BL-1 and BL-2 receive the PIM Join and create a similar (S,G) state. Both devices route the traffic locally from VLAN-2 to the M-VLAN.

    However, only the DF for the M-VLAN Ethernet segment actually forwards the data on the M-VLAN to the external PIM domain. In this case, BL-1 is the DF and sends the traffic toward the external receiver. (See the label "M-VLAN ESI DF" and the black arrow between BL-1 and the PIM router in Figure 5.)

  5. The PIM RP receives the traffic from the OISM M-VLAN IRB interface connection. The PIM router sends the traffic to an L3 interface toward the external receiver.

Multicast Traffic from an Internal Source to Receivers Outside the EVPN Data Center—L3 Interface Method or Non-EVPN IRB Method

In Figure 6, we illustrate the OISM use case where a multicast source inside the EVPN fabric sends multicast traffic to a receiver outside the fabric using either of the following methods for external multicast:

  • Classic L3 interface external multicast method:

    On each border leaf device, you configure a classic L3 interface with family inet that connects to the external PIM router. You assign an IP address to that interface on a different subnet than the L3 interface subnets on other border leaf devices in the fabric.

    You enable PIM on the interface and include the interface in the tenant VRF instances that have multicast data receivers. This method differs from the M-VLAN IRB method because you don't extend this interface in the EVPN instance.

    Note:

    The L3 interface connection here can be an individual physical interface or an AE interface bundle that includes multiple physical L3 interfaces.

  • Non-EVPN IRB external multicast method:

    On each border leaf device, you configure a unique extra VLAN that is only for external multicast. You also configure a corresponding L3 IRB interface with an IP address that connects to the external PIM router. The extra VLAN ID can't be the same as the VLAN ID of the revenue bridge domains, SBD, or extra VLAN on any other border leaf device in the fabric. In addition, similar to the L3 interface method, the non-EVPN IRB interfaces on different border leaf devices should connect to the PIM router on different subnets in the fabric.

    You enable PIM on the IRB interface and include the interface in the tenant VRF instances that have multicast data receivers. This method differs from the M-VLAN IRB method because you don't extend this VLAN or IRB interface in the EVPN instance.

Supported Methods for Multicast Data Transfer to or from an External PIM Domain lists the platforms that support these external multicast methods.

Figure 6 includes the same internal multihomed source, internal receivers, and an external receiver as in Multicast Traffic From an Internal Source to Receivers Outside the EVPN Data Center—M-VLAN IRB Method. See the steps in Traffic Flow from Multihomed Source to Internal Receivers for details on the internal multicast traffic flow. In this section we describe only what's different in this case, which is the multicast traffic flow from the border leaf devices to the external receiver.

Figure 6: OISM with an Internal Multicast Source and an External Multicast Receiver—L3 Interface or Non-EVPN IRB Method OISM with an Internal Multicast Source and an External Multicast Receiver—L3 Interface or Non-EVPN IRB Method

In Figure 6, the internal source for a multicast group is Mcast-Src-2, which is multihomed to Leaf-1 and Leaf-2. The source sends the multicast traffic on VLAN-2, the red VLAN. The external receiver, Ext-Mcast-Rcvr, expresses interest in receiving the multicast traffic for that multicast group (sends a join message).

The external multicast flow in this use case is very similar to the M-VLAN IRB use case. The border leaf devices in the OISM PEG role receive the multicast traffic on the source VLAN through the EVPN core. However, the main difference in this case is that the external multicast interfaces don't use EVPN signaling and don't share an ESI across the border leaf devices. The external multicast interfaces on each border leaf device are distinct, and each have L3 reachability to the external PIM gateway router. The border leaf device that establishes the PIM join state replicates and sends the traffic on the L3 interface or non-EVPN IRB interface to the external PIM domain with the external receiver.

Note:

PEG border leaf devices only send multicast source traffic received on the revenue bridge domains to the external PIM domain. These devices don't forward the traffic back into the EVPN core toward the other border leaf devices.

The following section explains how the source traffic reaches the interested external receiver.

Traffic Flow to External Receiver—L3 Interface or Non-EVPN IRB Method

These steps summarize the multicast control and data traffic flow in Figure 6 from the border leaf devices toward the external receiver using the classic L3 interface method or the non-EVPN IRB method:

  1. In the external PIM domain, the PIM RP enters a PIM (*,G) multicast routing table entry. The entry includes the L3 interface toward Ext-Mcast-Rcvr as the downstream interface.

  2. Both border leaf devices BL-1 and BL-2 receive the source traffic from the EVPN core on VLAN-2. The IRB interface on VLAN-2 on one of these border leaf devices is the PIM DR for VLAN-2. In this case, the PIM DR is on BL-1, so BL-1 sends a PIM Register message toward the PIM RP on its external multicast L3 interface or non-EVPN IRB interface.

  3. The PIM RP sends a PIM Join message back toward BL-1. BL-1 receives the PIM join and creates an (S,G) multicast routing table entry as follows:

    • The source address is the IP address of Mcast-Src-2 in VLAN-2.
    • The downstream interface is the external multicast L3 interface or the non-EVPN IRB interface.
  4. BL-1 routes the traffic from VLAN-2 to its external multicast L3 interface or non-EVPN IRB interface.

  5. The PIM RP receives the traffic from BL-1 on the external multicast interface. The PIM router sends the traffic to an L3 interface toward the external receiver. The external receiver receives the multicast traffic.

Multicast Traffic from an External Source to Receivers Inside the EVPN Data Center—M-VLAN IRB Method

Figure 7 illustrates the OISM use case where a multicast source outside the EVPN fabric sends multicast traffic to receivers inside the fabric. This use case exhibits the two main ways OISM uses the SBD in the EVPN core:

  • To carry external multicast source traffic.

  • To advertise SMET Type 6 routes.

The Type 6 routes ensure that the border leaf devices only forward the traffic toward the EVPN devices with interested receivers.

OISM border leaf devices receive external multicast source traffic through the M-VLAN IRB interfaces. OISM devices use the SBD to forward traffic toward EVPN server leaf devices with interested receivers on the revenue bridge domains. Each leaf device then locally forwards or routes the traffic on the revenue bridge domains to its local receivers.

This use case has an internal receiver that is multihomed to two server leaf devices.

Figure 7: OISM with an External Multicast Source and an Internal Multihomed Multicast Receiver—M-VLAN IRB Method OISM with an External Multicast Source and an Internal Multihomed Multicast Receiver—M-VLAN IRB Method

In Figure 7, Rcvr-1 within the EVPN fabric is multihomed to server leaf devices Leaf-1 and Leaf-2. Rcvr-1 expresses interest in receiving traffic from a multicast group. The source for the multicast traffic for the group is Ext-Mcast-Src in the external PIM domain.

The external source traffic reaches the interested multihomed receiver, Rcvr-1, as follows:

Multicast Control Flow between the Internal Multihomed Receiver and the External Source—M-VLAN IRB Method

These steps summarize the multicast control flow in this use case:

  1. Rcvr-1 sends an IGMP join message to both multihoming peers Leaf-1 and Leaf-2.

  2. Both Leaf-1 and Leaf-2 generate an EVPN Type 6 route toward the EVPN core on the SBD. The Type 6 (SMET) route advertises that Rcvr-1 is interested in the multicast data.

  3. Both border leaf devices BL-1 and BL-2 receive the Type 6 route on the SBD.

  4. The Type 6 route (on the SBD) signals the border leaf devices to create a PIM join toward the PIM RP (reachable through the M-VLAN). However, to avoid duplicate join messages, only the border leaf device that is the PIM DR for the SBD generates the PIM join message. In this case, the figure shows the PIM DR for the SBD is BL-1. BL-1 sends the PIM join message toward the PIM RP by way of its neighbor, the M-VLAN IRB interface.

  5. The PIM RP receives the join message. Then the PIM RP creates a PIM (*,G) entry in the multicast routing table with the M-VLAN IRB interface as the downstream interface.

  6. The external source Ext-Mcast-Src registers with the PIM RP. The PIM RP has a multicast route for the group with the M-VLAN IRB interface as the downstream interface. As a result, the PIM RP routes the multicast traffic coming in at Layer 3 onto its connection to the M-VLAN IRB toward BL-1 or BL-2. In this case, BL-1 sent the PIM join, so BL-1 receives the traffic on its M-VLAN IRB interface.

Traffic Flow from the Border Leaf Devices to Internal Receivers—M-VLAN IRB Method

In Figure 7, BL-1 is the PIM DR for the SBD and sent the PIM join toward the external PIM domain. BL-1 receives and routes (or forwards) the external source traffic as follows:

  1. BL-1 routes the traffic locally from the M-VLAN to the SBD on its SBD IRB interface because BL-1 is the PIM DR for the SBD. See the small gray arrow from the M-VLAN to the SBD on BL-1.

  2. BL-1 forwards a copy of the traffic on the M-VLAN toward BL-2 because both border leaf devices are in the PEG role. See the black arrow from BL-1 toward BL-2.

    As a PEG device using the M-VLAN IRB method, BL-2 expects to receive external multicast traffic only on the M-VLAN IRB interface. If BL-2 has any local receivers, BL-2 can receive the traffic and route it locally to those receivers.

  3. BL-1 also forwards a copy of the traffic on the SBD into the EVPN core to BL-2. See the green arrow from BL-1 toward BL-2.

    BL-2 drops the traffic because again, as a PEG device using the M-VLAN IRB method, BL-2 expects to receive external source traffic only on the M-VLAN IRB interface. BL-2 doesn't expect external source traffic on the SBD IRB interface from BL-1. In other words, BL-2 sees this case as a source interface mismatch (a reverse path forwarding [RFP] failure).

    Note:

    One reason why the ingress border leaf device also forwards a copy on the SBD to other border leaf devices is to ensure that another border leaf device can receive external source traffic if its M-VLAN interface is down. Then any interested local receivers on the other border leaf device can still get the traffic.

  4. BL-1 selectively forwards copies of the traffic on the SBD to the server leaf devices with interested receivers, based on the advertised Type 6 routes.

    In this case, Leaf-1 and Leaf-2 have a multihomed interested receiver, Rcvr-1, on VLAN-2. As a result, BL-1 sends the traffic toward both leaf devices. See the green arrows from BL-1 toward Leaf-1 and Leaf-2.

    Note:

    In a similar use case with the PIM router multihomed to BL-1 and BL-2, BL-1 might receive the external multicast source traffic, but BL-2 is the PIM DR on the SBD. One reason why BL-1 forwards the incoming external multicast traffic toward BL-2 on the M-VLAN is so that BL-2 can handle this use case. See the black arrow from BL-1 toward BL-2 on the M-VLAN in Figure 7. If BL-2 is the PIM DR on the SBD, upon receiving the traffic on the M-VLAN from BL-1, BL-2 forwards the traffic on the SBD toward Leaf-1 and Leaf-2. In this case, the green arrows in the figure would flow from BL-2 toward the other EVPN devices instead of flowing from BL-1.

  5. Leaf-1 and Leaf-2 locally route the traffic from the SBD IRB interface to the revenue bridge domain IRB interface for VLAN-2 toward the interested (multihomed) receiver. However, with EVPN multihoming, only the EVPN DF in the Ethernet segment forwards the traffic toward Rcvr-1 so Rcvr-1 doesn't get duplicate traffic.

    In this case, Leaf-1 is the EVPN DF, so only Leaf-1 forwards the traffic to Rcvr-1.

What Happens When a Multihomed External PIM Router Load-Balances the Traffic—M-VLAN IRB Method

In Figure 7, the external PIM gateway router is multihomed to BL-1 and BL-2 on an Ethernet segment in the EVPN fabric. If the pair of connections on the PIM router side is an AE interface bundle, the PIM router does load balancing among the interfaces in the bundle. In that case, both BL-1 and BL-2 will each receive some of the multicast traffic flow from the external source. However, all receivers should receive all of that traffic. For simplicity, the figure doesn't show traffic arrows for this load balancing, but we'll describe the flow here.

BL-1 and BL-2 each receive part of the external multicast source traffic on their M-VLAN IRB interfaces. However, because BL-1 is the PIM DR on the SBD, only BL-1 will route the traffic onto the SBD into the EVPN fabric, as follows:

  1. BL-1 routes the traffic it receives onto the SBD toward the server leaf devices.

    BL-1 also forwards that traffic on the M-VLAN and routes it on the SBD toward BL-2, in case BL-2 has any local receivers (as described in Traffic Flow from the Border Leaf Devices to Internal Receivers—M-VLAN IRB Method).

  2. BL-2 forwards the traffic it receives from the external PIM domain on the M-VLAN toward BL-1, because BL-1 only expects to receive external source traffic on the M-VLAN.

    Due to DF and split horizon rules, BL-2 won't forward any traffic it receives on the M-VLAN from BL-1 into the EVPN core or back toward the source, BL-1.

  3. BL-1 routes the traffic it receives on the M-VLAN from BL-2 onto the SBD toward the server leaf devices.

What Happens with Local Receivers on Border Leaf Devices—M-VLAN IRB Method

Figure 7 doesn't show local receivers attached to the border leaf devices. However, let's look briefly at PIM join message flow and how external source traffic reaches local receivers on border leaf devices.

Consider that BL-1 or BL-2 has an interested receiver on a revenue bridge domain in the fabric. In that case:

  1. Both devices generate a PIM join on the IRB interfaces for the revenue bridge domain toward the PIM RP.

  2. You configure the border leaf devices with PIM in distributed DR mode on the revenue bridge domain IRB interfaces. That way, neither BL-1 nor BL-2 acts as the PIM DR alone. Both devices locally route external multicast source traffic coming in on the M-VLAN IRB interface to the appropriate revenue bridge domain IRB interface.

Multicast Traffic from an External Source to Receivers Inside the EVPN Data Center—L3 Interface Method or Non-EVPN IRB Method

Figure 8 illustrates the OISM use case where a multicast source outside the EVPN fabric sends multicast traffic to receivers inside the fabric. In this case, the fabric uses the classic L3 interface or non-EVPN IRB external multicast method to connect to the external PIM domain. Also, this case includes the following internal receivers:

  • A receiver that is multihomed to two server leaf devices.

  • A local receiver on one of the border leaf devices.

This use case, like the M-VLAN IRB external source use case in Figure 7, shows the two main ways OISM uses the SBD in the EVPN core—To carry external multicast source traffic and to advertise SMET Type 6 routes. The Type 6 routes ensure that the border leaf devices only forward the traffic toward the EVPN devices with interested receivers.

Figure 8: OISM with an External Multicast Source and an Internal Multihomed Multicast Receiver—L3 Interface or Non-EVPN IRB Method OISM with an External Multicast Source and an Internal Multihomed Multicast Receiver—L3 Interface or Non-EVPN IRB Method

In Figure 8:

  • Rcvr-1 is multihomed to server leaf devices Leaf-1 and Leaf-2 in the EVPN fabric, and expresses interest in receiving traffic from a multicast group.

  • Rcvr-5 on BL-2 in the EVPN fabric is also interested in receiving the multicast traffic.

  • Ext-Mcast-Src in the external PIM domain is the source for the traffic for the multicast group.

The multicast control flow and data traffic flow for external multicast are similar for the classic L3 interface and the non-EVPN IRB interface methods. As a result, in this section we commonly say external multicast interface when we refer to the border leaf device external connection points.

The external source traffic reaches the interested receivers (Rcvr-1 and Rcvr-5) as follows:

Multicast Control Flow between the Internal Receivers and the External Source—L3 Interface or Non-EVPN IRB Method

These steps summarize the multicast control flow in this use case:

  1. Rcvr-1 sends an IGMP join message on VLAN-2 to both multihoming peers Leaf-1 and Leaf-2.

  2. Both Leaf-1 and Leaf-2 generate an EVPN Type 6 route toward the EVPN core on the SBD. The Type 6 (SMET) route advertises that Rcvr-1 is interested in the multicast data.

  3. Both border leaf devices BL-1 and BL-2 receive the Type 6 route on the SBD.

  4. The Type 6 route (on the SBD) signals the border leaf devices to create a PIM join toward the PIM RP (reachable through the external multicast interface). However, to avoid duplicate join messages for server leaf devices on the SBD, only the border leaf device that is the PIM DR for the SBD generates the PIM join message. In this case, the figure shows the PIM DR for the SBD is BL-1. BL-1 sends the PIM join message toward the PIM RP by way of its PIM neighbor, the external multicast interface.

  5. The local receiver on BL-2, Rcvr-5, also sends an IGMP join message on VLAN-2 to BL-2. Note that in this case, BL-1 and BL-2 are not multihoming peers in an EVPN Ethernet segment. As a result, BL-2 sends a separate PIM join message on its external multicast interface because it has a local interested receiver (Rcvr-5).

  6. The PIM RP receives the join messages. The PIM RP creates PIM (*,G) entries in the multicast routing table with the BL-1 and BL-2 external multicast interfaces as downstream interfaces.

  7. The external source Ext-Mcast-Src registers with the PIM RP. The PIM RP has multicast routes for the group with the BL-1 and BL-2 external multicast interfaces as downstream interfaces. As a result, the PIM RP routes the multicast traffic coming in at L3 toward BL-1 and BL-2.

Both BL-1 and BL-2 receive the multicast traffic. The next section explains how the border leaf devices forward or route the traffic in the EVPN fabric.

Traffic Flow from the Border Leaf Devices to Internal Receivers—L3 Interface or Non-EVPN IRB Method

In Figure 8, BL-1 is the PIM DR for the SBD and sent the PIM join message toward the external PIM domain for server leaf devices on the SBD. BL-2 also sent a PIM join message toward the external PIM domain for its interested local receiver.

BL-1 and BL-2 receive the external source traffic, and route (or forward) it as follows:

  1. BL-1 routes the traffic locally from the external multicast interface to the SBD IRB interface because BL-1 is the PIM DR for the SBD. See the small gray arrow from the external multicast interface to the SBD on BL-1.

  2. BL-1 forwards a copy of the traffic on the SBD into the EVPN core to BL-2. See the green arrow from BL-1 toward BL-2.

    However, BL-2 drops the traffic from the SBD because, as a PEG device using the classic L3 interface or non-EVPN IRB method, BL-2 doesn't expect external source traffic on the SBD IRB interface from BL-1. If BL-2 has interested receivers, it would have sent a PIM join message and should receive the same traffic from its external multicast connection.

    Note:

    One reason why the ingress border leaf device also forwards a copy on the SBD to other border leaf devices is to ensure that another border leaf device can receive external source traffic if its external multicast interface is down. Then any interested local receivers on the other border leaf device can still get the traffic.

  3. BL-2 routes the external multicast traffic to its local receiver, Rcvr-5, on VLAN-2. See the small gray arrow on BL-2 from the external multicast interface to VLAN-2.

    Note:

    The border leaf devices configured in PEG mode that are not the PIM DR on the SBD will still locally route the traffic received from the external multicast interface. These devices don't send traffic from external multicast sources to other PEG border leaf devices on the SBD. These devices also don't forward the traffic on the SBD into the EVPN core.

  4. BL-1 (the PIM DR on the SBD) selectively forwards copies of the traffic on the SBD to the server leaf devices with interested receivers (based on the advertised Type 6 routes). See the green arrows from BL-1 toward Leaf-1 and Leaf-2.

    In this case, Leaf-1 and Leaf-2 have a multihomed interested receiver, Rcvr-1, on VLAN-2. As a result, BL-1 sends the traffic on the SBD toward both leaf devices.

  5. Leaf-1 and Leaf-2 locally route the traffic from the SBD IRB interface to the revenue bridge domain IRB interface for VLAN-2 toward the interested (multihomed) receiver. However, with EVPN multihoming, only the EVPN DF in the Ethernet segment forwards the traffic toward Rcvr-1 so Rcvr-1 doesn't get duplicate traffic.

    In this case, Leaf-1 is the EVPN DF, so only Leaf-1 forwards the traffic to Rcvr-1. See the red arrow from Leaf-1 toward Rcvr-1.

Considerations for OISM Configurations

Before you begin to set up your OISM installation, here are a few considerations in specific use cases.

IGMPv2 and IGMPv3 with IGMP Snooping and OISM in the Same EVPN-VXLAN Fabric

You have several options to configure IGMP snooping with IGMPv2, IGMPv3, or both IGMP versions together in an EVPN-VXLAN fabric with OISM. This section describes a few options.

If you have traffic for either IGMPv2 or IGMPv3 on a device with OISM enabled, you can enable that IGMP version globally on the device with IGMP snooping. (You can enable IGMP snooping as needed on all or specific VLANs.) You can alternatively enable that IGMP version only for the interfaces that will handle the multicast traffic.

OISM supports IGMP snooping with both IGMPv2 and IGMPv3 traffic together on a device in the fabric only within the following constraints:

  • You can't enable both IGMP snooping with both IGMPv2 and IGMPv3 for interfaces in the same VLAN.

  • You can't enable IGMP snooping with both IGMPv2 and IGMPv3 for VLANs that are part of the same L3 VRF instance with OISM enabled.

As a result, you can configure the device to support IGMP snooping with both IGMP versions if you configure:

  • One tenant VRF instance to support the IGMPv2 receivers.

  • Another tenant VRF instance to support the IGMPv3 receivers.

as follows:

  1. In your configuration, define VLANs for the IGMPv2 receivers and different VLANs for the IGMPv3 receivers.

  2. Include the IRB interfaces that support IGMPv2 in one VRF instance, and enable IGMPv2 on those IRB interfaces. Enable IGMP snooping on the corresponding VLANs.

  3. Include the IRB interfaces that support IGMPv3 in another VRF instance, and enable IGMPv3 on those IRB interfaces. Enable IGMP snooping (with the evpn-ssm-reports-only option) on the corresponding VLANs.

In this use case, for each IGMP version, allocate a set of VLANs and IRB interfaces for:

  • The OISM revenue bridge domains.

  • The SBD.

  • Any external multicast VLANs and interfaces (depending on the external multicast method you use).

You also define two L3 VRF instances for each tenant instance you need in your installation, one for each IGMP version. If you use MAC-VRF routing instances at L2, you might want to allocate different MAC-VRF EVPN instances for the IGMP snooping traffic for each IGMP version.

For example, consider that you set up a fabric with the M-VLAN IRB method for external multicast. You want to support IGMP snooping with both IGMPv2 and IGMPv3 traffic. In that case, you might configure the following MAC-VRF instances, L3 VRF instances, VLANs, and corresponding IRB interfaces:

  • MAC-VRF2 and L3VRF-A to support IGMPv2 receivers:

    • Revenue bridge domain VLAN-100 with irb.100

    • SBD VLAN-302 with irb.302

    • (Border leaf devices only) M-VLAN VLAN-902 with irb.902

  • MAC-VRF3 and L3VRF-B to support IGMPv3 receivers:

    • Revenue bridge domain VLAN-200 with irb.200

    • SBD VLAN-303 with irb.303

    • (Border leaf devices only) M-VLAN VLAN-903 with irb.903

Then include the IGMPv2 IRB interfaces in L3VRF-A and enable IGMPv2 for those IRB interfaces. Include the IGMPv3 IRB interfaces in L3VRF-B and enable IGMPv3 for those IRB interfaces. For example:

Finally, enable IGMP snooping at L2 in the EVPN instances as follows:

  • Configure igmp-snooping in MAC-VRF2 for the VLANs corresponding to the IGMPv2 IRB interfaces.

    Include the proxy option when you enable IGMP snooping for IGMPv2 traffic (you can optionally do the same for IGMPv3 if needed).

  • Configure igmp-snooping in MAC-VRF3 for the VLANs corresponding to the IGMPv3 IRB interfaces.

    Include the evpn-ssm-reports-only option only when you enable IGMP snooping for IGMPv3 traffic. You can also optionally include the proxy option for IGMP snooping with IGMPv3 if needed.

For example:

Note:

With the non-EVPN IRB method for external multicast, you don't include the evpn-ssm-reports-only option on the non-EVPN IRB interface. You don't need this option because with the non-EVPN IRB method, you don't extend the external multicast interface in the EVPN instance.

When you use the L3 interface method for external multicast, you don't enable IGMP snooping at all on the L3 interface to the external PIM domain. That interface operates at L3, while IGMP snooping operates at L2.

You can scale these simple scenarios to support different tenants with different combinations of IGMP versions.

See Supported IGMP or MLD Versions and Group Membership Report Modes for more information on IGMP any-source multicast (ASM) mode and source-specific multicast (SSM) mode support with IGMPv2 and IGMPv3 in EVPN-VXLAN fabrics.

Latency and Scaling Trade-Offs for Installing Multicast Routes with OISM (install-star-g-routes Option)

Devices in an OISM-enabled fabric send EVPN Type 6 routes so other EVPN devices learn about receivers that are interested in the traffic for a multicast group. The receivers are in different OISM revenue bridge domains in an L3 VRF instance. To save bandwidth in the EVPN fabric core, OISM devices send and receive the Type 6 routes only on the OISM SBD in the routing instance.

To help minimizes packet loss at the onset of a multicast flow, you can configure the install-star-g-routes option at the [edit <routing-instances name> multicast-snooping-options oism] hierarchy level (see oism (Multicast Snooping Options)). When you configure this option, upon receiving a Type 6 route, the Routing Engine on the device immediately installs corresponding (*,G) multicast routes on the Packet Forwarding Engine for all of the revenue bridge domain VLANs in the routing instance.

With this option, you trade off taking up extra Packet Forwarding Engine resources to improve network latency. Lower-scale deployments might have fewer multicast flows but have strict network latency requirements. To improve network latency in that case, the device installs the (*,G) routes in the data plane in advance of any incoming multicast traffic.

Configure this option:

  • Globally if you configure EVPN in the default-switch instance, at the [edit multicast-snooping-options oism] hierarchy level.

  • In the MAC-VRF instances if you configure EVPN in instances of type mac-vrf, at the [edit routing-instances instance-name multicast-snooping-options oism] .

Always configure this option with OISM on:

  • Switches in the QFX10000 line.

  • PTX10001-36MR, PTX10004, PTX10008, and PTX10016 routers.

  • QFX5130-32CD and QFX5700 switches when you configure these devices with AR in the AR replicator role.

Those switches easily operate at scale and multicast traffic flows well with this option's behavior. Without this option, in the above use cases, the Routing Engine on those switches doesn't properly install the multicast routes.

We don't recommend setting this option with OISM on:

  • EX4300-48MP, EX4400, EX4650, QFX5110 or QFX5120 switches.

  • QFX5130 or QFX5700 switches that you don't configure as a collocated AR replicator.

Consider this option in the above use cases only if you have very stringent latency requirements and can trade off higher scaling to achieve better network latency.

Default Behavior without the install-star-g-routes Option

By default, without this option, the device prioritizes saving resources on the Packet Forwarding Engine by not installing multicast routes until the multicast traffic arrives. In this default case:

  1. The Packet Forwarding Engine receives multicast traffic from source S for multicast group G.

  2. The Packet Forwarding Engine doesn't have forwarding next-hop information for the traffic, so it signals the Routing Engine to get that information.

    Note:

    The Packet Forwarding Engine drops multicast traffic until it gets the routing information.

  3. The Routing Engine learns about the multicast flow for (S,G) from the Packet Forwarding Engine, and installs that route on the Packet Forwarding Engine.

  4. The Packet Forwarding Engine sends the traffic on the next hop in the installed (S,G) route.

Behavior with the install-star-g-routes Option

With the install-star-g-routes option, the device prioritizes having multicast routing information available on the Packet Forwarding Engine before any traffic arrives. The device consumes extra Packet Forwarding Engine resources for routes that it isn't using yet (and might never be used). With this option:

  1. The Routing Engine receives an EVPN Type 6 route for a receiver subscribing to traffic for a multicast group G on the OISM SBD in a routing instance.

  2. The Routing Engine installs corresponding (*,G) routes on the Packet Forwarding Engine for all of the revenue bridge domains in the L3 VRF instance.

  3. At some later time, the Packet Forwarding Engine receives multicast traffic from source S for multicast group G.

  4. The Packet Forwarding Engine has forwarding next hop information for traffic for (*,G). So it forwards the traffic to receivers on any revenue bridge domains using the (*,G) route next hop.

  5. The Packet Forwarding Engine also signals the Routing Engine that it has received multicast traffic from source S for multicast group G.

  6. The Routing Engine learns about the multicast flow for (S,G) from the Packet Forwarding Engine. The Routing Engine installs the (S,G) route on the Packet Forwarding Engine.

  7. The Packet Forwarding Engine continues sending the traffic, but now uses the (S,G) route and the next hop in that more specific route.

    Note:

    The Packet Forwarding Engine still retains the (*,G) routes per revenue bridge domain that the Routing Engine installed after receiving the Type 6 route.

OISM and AR Scaling with Many VLANs

With OISM and IGMP snooping enabled in an EVPN-VXLAN fabric, OISM server leaf and border leaf devices send EVPN Type 6 SMET routes into the EVPN core when their receivers join a multicast group. When OISM-enabled devices receive the Type 6 routes, they:

  • Derive the multicast (*,G) states for IGMPv2 or multicast (S,G) states for IGMPv3 from the Type 6 routes.

  • Install the derived states on the OISM SBD and revenue bridge domain VLANs in the MAC-VRF instance for all VLANs that are part of OISM-enabled L3 tenant VRF instances.

  • Use the derived multicast routes to optimize multicast forwarding by selectively sending the traffic for a group only to other EVPN devices that have receivers subscribed to that group.

On some devices that support OISM, you can also configure the assisted replication (AR) multicast optimization feature with OISM enabled. AR replicator devices use the Type 6 routes the same way OISM devices do to forward traffic for a group only to other devices that have subscribed receivers.

QFX5130-32CD and QFX5700 switches can serve as AR replicators in a fabric with OISM, but only on devices that are not also OISM border leaf devices (AR replicator standalone mode). However, in fabrics with many VLANs, QFX5130-32CD and QFX5700 switches might have scaling issues when installing the multicast states on all the OISM VLANs. As a result, starting in Junos OS Evolved 22.2R1, when you configure these switches as AR replicators with OISM enabled, by default these switches only install the multicast (*,G) states for IGMPv2 or multicast (S,G) states for IGMPv3 on the SBD VLAN. They don’t install the multicast states on all the revenue bridge domain VLANs.

For example, consider a QFX5130-32CD device where you have a MAC-VRF instance evpn-vxlan-A with 3 VLANs—VLAN_2, VLAN_3, and VLAN_4. The show igmp snooping evpn status detail command shows that you configured VLAN_4 as the SBD (the Supplementary BD output field is Yes), and the other two VLANs are OISM revenue bridge domain VLANs:

The device received Type 6 routes from remote devices for multicast groups 233.252.0.1 and 233.252.0.2:

Due to the scaling behavior difference on QFX5130-32CD or QFX5700 switches, if you run the show multicast snooping route command on these devices, the output shows multicast group entries only on the SBD, and not on any of the revenue bridge domains. For example, with our multicast groups 233.252.0.1 and 233.252.0.2:

QFX5130-32CD or QFX5700 switches that are not running as AR replicators with OISM will install the multicast group entries on all of the revenue bridge domain VLANs and the SBD. When you run the show multicast snooping route command in that case, you see all of the revenue bridge domain VLANs and the SBD . This behavior also applies to all other platforms running OISM, whether the device is an AR replicator or not.

Configure Common OISM Elements on Border Leaf Devices and Server Leaf Devices (Symmetric Bridge Domains)

Follow these steps to configure elements common to border leaf and server leaf devices in an EVPN-VXLAN fabric running OISM with the symmetric bridge domains model. This configuration is based on an EVPN-VXLAN fabric configuration that supports OISM and has:

  • An EBGP underlay with Bidirectional Forwarding Detection (BFD) and Ethernet operations, administration and management (OAM) link detection.

  • An ERB overlay design.

  • Lean spine devices that act only as IP transit nodes in the fabric.

  • Server leaf devices configured as EVPN-VXLAN L2 gateways.

  • Border leaf devices configured as EVPN-VXLAN L2 and L3 gateways.

In the symmetric bridge domains model, you must configure all of the revenue bridge domains and the supplemental bridge domain (SBD) on all OISM devices in the fabric. See Configuration Elements for OISM Devices (Symmetric Bridge Domains) for a summary of what elements you configure on border leaf and server leaf devices, and why you configure those elements.

The sample configuration blocks we provide for these configuration steps use an OISM environment with the following elements:

  • An EVPN instance configured in either the default switch instance (no routing instance specified) OR a MAC-VRF EVPN instance. For example:

    • Default switch EVPN instance:

    • MAC-VRF EVPN instances (for each MAC-VRF instance):

    With a MAC-VRF EVPN instance configuration, you configure some elements in the MAC-VRF instances. In OISM configurations, we support either the vlan-aware or vlan-based MAC-VRF instance service type.

    To illustrate sample configuration steps here, we show a MAC-VRF instance named MAC-VRF1with the vlan-aware service type. The main differences between the two service types are:

    • vlan-aware: You can define more than one VLAN, its corresponding IRB interface, and VXLAN network identifier (VNI) mapping in the instance. As a result, you specify VLAN-related OISM or multicast configuration statements in one vlan-aware MAC-VRF instance for all of the VLANs in that instance.

    • vlan-based: You configure separate MAC-VRF instances in which you define each VLAN and its IRB interface and VNI mapping. As a result, you include similar VLAN-related OISM or multicast configuration statements for each VLAN in the corresponding vlan-based MAC-VRF instance.

  • SBD: VLAN-300

    SBD IRB interface: irb.300

    SBD IRB interface IP address: 10.0.300.1

  • Revenue bridge domains: VLAN-100 and VLAN-200

    Revenue bridge domain IRB interfaces: irb.100 and irb.200

    Revenue bridge domain IRB interface IP addresses: 10.0.100.1and 10.0.200.1

  • L3 VRF routing instance: L3VRF-1

  • If you use the M-VLAN IRB method for external multicast connectivity:

    M-VLAN: VLAN-900

    M-VLAN IRB interface: irb.900

    M-VLAN IRB interface IP address: 172.16.90.1/24

    Interface name of port connecting to external PIM router: xe-0/0/9

    Note:

    You use the same M-VLAN ID and assign IRB interface IP addresses in the same subnet across any border leaf devices to which the external PIM router is multihomed.

  • If you use the classic L3 interface method for external multicast connectivity:

    L3 interface name: xe-0/0/6

    L3 interface IP address: 172.16.10.1/24

    Note:

    You assign IP addresses in different subnets for the L3 interfaces on each border leaf device connected to the external PIM router.

  • If you use the non-EVPN IRB method for external multicast connectivity:

    Extra VLAN: VLAN-900

    Non-EVPN IRB interface: irb.900

    Non-EVN IRB interface IP address: 172.16.90.1/24

    Interface name of port connecting to external PIM router: xe-0/0/9

    Note:

    With the non-EVPN IRB method, you assign distinct extra VLAN IDs on each border leaf device. You also assign IP addresses in different subnets for the non-EVPN IRB interfaces on each border leaf device connected to the external PIM router.

Configure these OISM statements on both border leaf devices and server leaf devices:

  1. Enable OISM globally on the device.

    Configure the evpn irb oism option at the [edit forwarding-options multicast-replication] hierarchy level. This option enables OISM optimizations and external multicast capabilities in an ERB overlay fabric. This option takes the place of the evpn irb local-only option you would otherwise use in ERB overlay fabrics for multicast without OISM.

    Note:

    Some platforms, such as such as PTX10001-36MR, PTX10004, PTX10008, and PTX10016 routers, only support OISM for multicast traffic. In that case, the device posts a warning in the configuration if you set the evpn irb local-only option or the evpn irb local-remote option, and ignores those configuration items.

  2. Configure all of the revenue bridge domain with an IRB interface and a VNI mapping for each revenue bridge domain. If your configuration uses MAC-VRF EVPN instances, you configure these elements in the MAC-VRF EVPN routing instances.

    For example, if the revenue bridge domains are VLAN-100 and VLAN-200:

    With the default switch instance:

    With a vlan-aware MAC-VRF instance MAC-VRF1:

  3. Configure a VLAN for the SBD with an IRB interface and VXLAN VNI mapping. For example, for SBD VLAN-300:

    For example, if the SBD is VLAN-300:

    With the default switch instance:

    With a vlan-aware MAC-VRF instance MAC-VRF1:

  4. Configure the revenue bridge domain and the SBD IRB interface IP addresses. For example:
  5. Extend the revenue bridge domains and the SBD VNIs in the EVPN-VXLAN overlay. If your configuration uses MAC-VRF EVPN instances, you do this in the MAC-VRF EVPN routing instances.

    For example, if the revenue bridge domains are VLAN-100 and VLAN-200, and the SBD is VLAN-300:

    With the default switch instance:

    With a vlan-aware MAC-VRF instance MAC-VRF1:

  6. Enable IGMPv2 or IGMPv3 on the device. Here for simplicity we show how to enable either IGMP version globally on the device.

    You can alternatively enable IGMPv2 or IGMPv3 on the specific IRB interfaces included in the tenant L3 VRF instances that handle the multicast traffic.

    In general, on all OISM devices, enable IGMP on the SBD IRB interface and the revenue bridge domain IRB interfaces. On border leaf devices, also enable IGMP on the external multicast interfaces (depending on the external multicast method used).

    See IGMPv2 and IGMPv3 with IGMP Snooping and OISM in the Same EVPN-VXLAN Fabric for more on how to configure IGMP snooping with both IGMPv2 andIGMPv2 together on a device. To enable IGMP on individual IRB interfaces that handle multicast traffic, include multipleigmp statements, one for each IRB interface. For example, set protocols igmp interface irb-interface-name <version 3>.

    1. For IGMPv2, which is the default IGMP version, enable IGMP globally. You can also optionally specify version 2 for clarity if you have both IGMP versions in your configuration. The configuration is the same if you use a default switch EVPN instance or MAC-VRF EVPN instances.
    2. For IGMPv3, enable IGMP globally with the version 3 option. The configuration is the same if you use a default switch EVPN instance or MAC-VRF EVPN instances.
  7. Enable IGMP snooping for IGMPv2 or IGMPv3 on all the configured OISM VLANs. Here in the common configuration for all OISM leaf devices, we include configuration only for the revenue bridge domains and the SBD. In the configuration steps specific to border leaf devices, we include the IGMP snooping configuration specific to the external multicast method you use on those devices.

    When you enable IGMP snooping, you also automatically enable advertising SMET Type 6 routes in the EVPN core based on received IGMP reports. If you use MAC-VRF EVPN instances, you enable IGMP snooping in the MAC-VRF instances.

    1. With IGMPv2, enable igmp-snooping in proxy mode.
      Note:

      You can use individual igmp-snooping commands for each VLAN, or one command with the vlan all option.

      With the default switch instance:

      With a vlan-aware MAC-VRF instance MAC-VRF1:

    2. With IGMPv3, enable igmp-snooping for IGMPv3 source-specific multicast mode (SSM) reports. Include the evpn-ssm-reports-only option for all the configured VLANs. You can also configure proxy mode if needed (we don't include that option in this case).

      With the default switch instance:

      With a vlan-aware MAC-VRF instance MAC-VRF1:

  8. Configure a Layer 3 VRF instance (instance-type vrf) that you associate with OISM routing functions. Include the revenue bridge domain IRB interfaces and the SBD IRB interface in the routing instance. For example:
  9. In the L3 VRF instance, specify IRB interface that you configured as the OISM SBD. For example:
  10. (Required on the QFX10000 line of switches and PTX10001-36MR, PTX10004, PTX10008, and PTX10016 routers; required on QFX5130-32CD and QFX5700 switches only when you configure these switches with OISM and AR in the AR replicator role; optional but not recommended on EX4300-48MP, EX4400, EX4650, QFX5110 or QFX5120 switches, or QFX5130-32CD and QFX5700 switches when the AR replicator role is not collocated on the device.) To avoid traffic loss at the onset of multicast flows, enable the install-star-g-routes option at the [edit <routing-instances intance-name> multicast-snooping-options oism] hierarchy level on all OISM devices.

    If you use the default switch EVPN instance, install-star-g-routes is a global option. If you use MAC-VRF EVPN instances, you set this option in each EVPN MAC-VRF routing instance. With this option, upon receiving an EVPN Type 6 route on the SBD, the device immediately installs corresponding (*,G) routes on the Packet Forwarding Engine for the revenue bridge domain VLANs in the L3 VRF instance.

    For full details on OISM device requirements, recommendations, and behavior with or without this option, see Latency and Scaling Trade-Offs for Installing Multicast Routes with OISM (install-star-g-routes Option).

    With the default switch instance:

    With a vlan-aware or vlan-based MAC-VRF instance called MAC-VRF1:

    You might choose to not enable this option if space on the Packet Forwarding Engine is at a premium and you don't have stringent latency requirements.

Configure Server Leaf Device OISM Elements (Symmetric Bridge Domains)

First configure the OISM elements described in Configure Common OISM Elements on Border Leaf Devices and Server Leaf Devices (Symmetric Bridge Domains) in an EVPN-VXLAN fabric.

Then follow these steps to configure the additional required OISM elements on the server leaf devices. The same EVPN-VXLAN fabric base and sample OISM environment apply to the additional server leaf configuration steps here.

You configure the elements specific to the server leaf functions (like PIM) in the tenant L3 VRF instances.

See Configuration Elements for OISM Devices (Symmetric Bridge Domains) for more information about why OISM server leaf devices require these settings.

  1. Configure PIM passive mode on server leaf devices. For example:
  2. Enable the server leaf device to accept multicast traffic from the SBD IRB interface as the source interface using the accept-remote-source statement at the [edit routing-instances name protocols pim interface irb-interface-name] hierarchy level. For example, for our sample SBD, VLAN-300, the IRB interface is irb.300:
  3. Configure an OSPF area in the L3 VRF instance for peer connectivity to support external multicast traffic on the SBD for server leaf devices.

    The server leaf devices use OSPF to learn routes to external sources when multicast traffic from outside the fabric arrives on the SBD. The device creates the PIM (S,G) entries it requires to forward the traffic from the SBD to the revenue bridge domains.

    On the server leaf devices, configure OSPF on the device loopback interface and the SBD IRB interface. For example:

Configure Border Leaf Device OISM Elements with M-VLAN IRB Method (Symmetric Bridge Domains)

This section covers how to configure border leaf devices that use the OISM M-VLAN IRB method to exchange multicast data with external sources and receivers. See Supported Methods for Multicast Data Transfer to or from an External PIM Domain for more on the available external multicast methods.

First configure the OISM elements described in Configure Common OISM Elements on Border Leaf Devices and Server Leaf Devices (Symmetric Bridge Domains) in an EVPN-VXLAN fabric.

Then follow these steps to configure the additional required OISM elements on the border leaf devices. The same EVPN-VXLAN fabric base and the sample OISM environment in that section apply to the additional border leaf configuration steps here.

You configure different elements that are specific to the border leaf functions either globally, in the EVPN instances, or in the tenant L3 VRF instances.

See Configuration Elements for OISM Devices (Symmetric Bridge Domains) for more information about why OISM border leaf devices require these settings.

  1. Configure the M-VLAN similarly to how you configure the revenue bridge domains and the SBD.
    1. Configure the M-VLAN with an IRB interface and a VXLAN network identifier (VNI) mapping. For example, if the M-VLAN is VLAN-900:

      With the default switch instance:

      With a vlan-aware MAC-VRF instance MAC-VRF1:

    2. Configure the M-VLAN IRB interface IP address. For example:
    3. Extend the M-VLAN in the EVPN-VXLAN overlay. For example:

      With the default switch instance:

      With a vlan-aware MAC-VRF instance MAC-VRF1:

  2. Enable IGMP snooping for the M-VLAN, and configure the multicast-router-interface option on the Layer 2 port that connects to the external multicast PIM router. For example, if xe-0/0/9.0 is the Layer 2 interface that connects the EVPN fabric to the external multicast router on the M-VLAN:
    1. With IGMPv2, include the proxy option:

      In the default switch instance:

      In a vlan-aware MAC-VRF instance MAC-VRF1:

    2. With IGMPv3, include the evpn-ssm-reports-only option. You can also enable IGMP snooping in proxy mode with IGMPv3 if needed, although we don't show that option here:

      In the default switch instance:

      In a vlan-aware MAC-VRF instance MAC-VRF1:

  3. In the common leaf device configuration steps, you configure a Layer 3 VRF instance associated with OISM routing functions. On the border leaf devices, also include the M-VLAN IRB interface in that Layer 3 VRF. The following configuration block shows the common Layer 3 VRF instance configuration with the additional M-VLAN IRB interface configuration statement highlighted:
  4. In the Layer 3 VRF instance, set the OISM PEG role on the M-VLAN IRB interface. For example:
  5. Configure PIM in the L3 VRF instance for a border leaf device.
    1. In the Layer 3 VRF instance, configure PIM in distributed DR mode on the revenue bridge domain IRB interfaces using the distributed-dr option at the [edit routing-instances name protocols pim interface irb-interface-name] hierarchy level.

      Configure PIM in standard mode on the SBD IRB interface.

      Configure PIM on the M-VLAN IRB interface in either standard mode or distributed DR mode. A border leaf device works well in standard PIM mode if the external PIM router is single-homed to one border leaf device. However, we strongly recommend to use distributed DR mode in any case, but especially if the external PIM router is multihomed to multiple border leaf devices. Distributed DR mode also helps the device to efficiently do local routing on the M-VLAN to local receivers on border leaf devices. As a result, in the sample configuration here, we show setting PIM with the distributed-dr option on the M-VLAN IRB.

      You also configure a PIM static RP that corresponds to the external PIM RP router. In the sample use cases in this documentation, the external PIM router serves as the PIM RP.

      For example, if the revenue bridge domains are VLAN-100 and VLAN-200, the SBD is VLAN-300, and the M-VLAN is VLAN-900:

    2. In the Layer 3 VRF instance, set the accept-join-always-from option at the [edit routing-instances name protocols pim interface irb-interface-name] hierarchy level on the M-VLAN IRB interface.

      Configure a policy option along with this statement so that the device always installs PIM joins from the external PIM router. See Configuration Elements for OISM Devices (Symmetric Bridge Domains) for more information about why OISM border leaf devices require this configuration.

      This sample configuration block represents the external PIM router as an MX Series router. For the policy prefix list, include the IP address of the M-VLAN interface on the MX Series router that connects to the border leaf device. For example:

  6. Configure an OSPF area in the L3 VRF instance for external multicast peer interface connectivity.

    The border leaf device uses OSPF to learn routes to multicast sources to forward traffic from external sources toward internal receivers, and from internal sources toward external receivers. The device needs these routes to create the PIM (S,G) entries to forward traffic on the revenue bridge domains, the SBD, and the external multicast interfaces.

    On a border leaf device with the M-VLAN IRB method for external multicast, configure the OSPF area to include the device loopback interface, the SBD IRB interface, and the M-VLAN IRB interface. For example, with our sample SBD VLAN-300 and M-VLAN VLAN-900:

Configure Border Leaf Device OISM Elements with Classic L3 Interface Method (Symmetric Bridge Domains)

This section covers how to configure border leaf devices that use the OISM classic L3 interface method to exchange multicast data with external sources and receivers. See Supported Methods for Multicast Data Transfer to or from an External PIM Domain for more on the available external multicast methods.

First configure the OISM elements described in Configure Common OISM Elements on Border Leaf Devices and Server Leaf Devices (Symmetric Bridge Domains) in an EVPN-VXLAN fabric.

Then follow these steps to configure the additional required OISM elements on border leaf devices. The same EVPN-VXLAN fabric base and the sample OISM environment in that section apply to the additional border leaf configuration steps here.

You configure most of the elements that are specific to the border leaf functions at L3 in the tenant L3 VRF instances.

See Configuration Elements for OISM Devices (Symmetric Bridge Domains) for more information about why OISM border leaf devices require these settings.

  1. Configure a physical L3 interface with an IP address for external multicast. For example, for an L3 interface xe-0/0/6 with IP address 172.16.10.1/24:
  2. Include the L3 logical interface in the L3 VRF instance that you configured in the common leaf device configuration steps. For example:
  3. In the L3 VRF instance, set the OISM PEG role on the border leaf device. With the classic L3 interface method, you don't need to specify an external IRB interface for external multicast:
  4. Configure PIM in the L3 VRF instance for a border leaf device.

    For the revenue bridge domain IRB interfaces, configure PIM in distributed DR mode using the distributed-dr option at the [edit routing-instances name protocols pim interface irb-interface-name] hierarchy level.

    Configure PIM in standard mode on the SBD IRB interface and the external multicast L3 logical interface.

    We also configure a PIM static RP that corresponds to the external PIM RP router. In the sample use cases in this documentation, the external PIM router serves as the PIM RP.

    For example, if the revenue bridge domains are VLAN-100 and VLAN-200, the SBD is VLAN-300, and the L3 interface is xe-0/0/6:

  5. Configure an OSPF area in the L3 VRF instance for external multicast peer interface connectivity.

    The border leaf device uses OSPF to learn routes to multicast sources to forward traffic from external sources toward internal receivers, and from internal sources toward external receivers. The device needs these routes to create the PIM (S,G) entries to forward traffic on the revenue bridge domains, the SBD, and the external multicast interfaces.

    On a border leaf device with the classic L3 interface method for external multicast, configure the OSPF area to include the device loopback interface, the SBD IRB interface, and the external multicast logical L3 interface. For example, with our sample SBD IRB interface, irb.300, and L3 interface xe-0/0/6:

Configure Border Leaf Device OISM Elements with Non-EVPN IRB Method (Symmetric Bridge Domains)

This section covers how to configure border leaf devices that use the OISM non-EVPN IRB method to exchange multicast data with external sources and receivers. See Supported Methods for Multicast Data Transfer to or from an External PIM Domain for more on the available external multicast methods.

First configure the OISM elements described in Configure Common OISM Elements on Border Leaf Devices and Server Leaf Devices (Symmetric Bridge Domains) in an EVPN-VXLAN fabric.

Then follow these steps to configure the additional required OISM elements on border leaf devices. The same EVPN-VXLAN fabric base and the sample OISM environment in that section apply to the additional border leaf configuration steps here.

You configure most of the elements that are specific to the border leaf functions (like PIM) in the tenant L3 VRF instances. With this method, you don't extend the extra VLAN in the EVPN instance, so you don't configure related elements in the EVPN instance. The external multicast configuration elements are the same whether you use the default switch instance or MAC-VRF EVPN instances.

See Configuration Elements for OISM Devices (Symmetric Bridge Domains) for more information about why OISM border leaf devices require these settings.

  1. Configure the extra VLAN with an IRB interface for external multicast. For example:
  2. Enable IGMP snooping for the extra VLAN, and configure the multicast-router-interface option on the port that connects to the external multicast PIM router. For example, if xe-0/0/9.0 is the interface on the border leaf device that connects to the external multicast router on the extra VLAN:
    1. With IGMPv2, include the proxy option:
    2. With IGMPv3, you don't need the evpn-ssm-reports only option because you don't extend the extra VLAN in the EVPN instance. You can use proxy mode if needed, although we don't show that option here:
  3. Include the extra VLAN IRB interface in the L3 VRF instance that you configured in the common leaf device configuration steps.

    The following configuration block shows the common L3 VRF configuration and highlights the additional statement:

  4. In the L3 VRF instance, set the OISM PEG role on the border leaf device. For example:
  5. Configure PIM in the L3 VRF instance for a border leaf device.

    For the revenue bridge domain IRB interfaces, configure PIM in distributed DR mode using the distributed-dr option at the [edit routing-instances name protocols pim interface irb-interface-name] hierarchy level.

    Configure PIM in standard mode on the SBD IRB interface and the extra VLAN IRB interface.

    We also configure a PIM static RP that corresponds to the external PIM RP router. In the sample use cases in this documentation, the external PIM router serves as the PIM RP.

    For example, if the revenue bridge domains are VLAN-100 and VLAN-200, the SBD is VLAN-300, and the extra VLAN is VLAN-900:
  6. Configure an OSPF area in the L3 VRF instance for external multicast peer interface connectivity.

    The border leaf device uses OSPF to learn routes to multicast sources to forward traffic from external sources toward internal receivers, and from internal sources toward external receivers. The device needs these routes to create the PIM (S,G) entries to forward traffic on the revenue bridge domains, the SBD, and the external multicast interfaces.

    On a border leaf device with the non-EVPN IRB method for external multicast, configure the OSPF area to include the device loopback interface, the SBD IRB interface, and the non-EVPN IRB interface. For example, with our sample SBD IRB interface (irb.300) and extra non-EVPN IRB interface (irb.900):

CLI Commands to Verify the OISM Configuration

To verify your OISM configuration:
  1. Use the show evpn oism command to view the SBD IRB interface in each Layer 3 VRF instance that you configured on the device for OISM. For example:

    To view information for a specific routing instance or to see more details about the configuration, use the extensive option with this show command. For example:

  2. Use the show route table bgp.evpn.0 ...extensive command to see the OISM capabilities you have enabled on an OISM leaf device. These capabilities are in the EVPN multicast flags extended community in EVPN Type 3 inclusive multicast Ethernet tag (IMET) routes. The OISM-related flags include:
    • igmp-snooping—You enabled IGMP snooping. The flags value for IGMP snooping without OISM or PEG configuration = 0x1.

    • oism—You globally enabled OISM. The flags value for OISM and IGMP snooping without PEG configuration = 0x9.

    • peg—You configured PEG mode on the interface. The flags value for OISM and IGMP snooping with PEG configuration = 0x19.

    For example, a route advertised for an M-VLAN IRB interface on a border leaf device has the peg flag set:

    You don't see the peg flag set for a revenue bridge domain interface on a border leaf device or a server leaf device:

  3. Enter the show igmp snooping evpn status <vlan name><detail> command to see the EVPN-VXLAN L2 multicast context for the OISM bridge domains (VLANs) on the device. The default output includes the VXLAN VNI mappings of the VLANs. For example:

    The detail option also displays whether a VLAN is the OISM SBD (Supplementary BD) or the M-VLAN (External VLAN). Both of those output fields display No for revenue bridge domains (VLANs) or if you don't enable OISM.

    Here is an example on a server leaf device in a fabric where the SBD is VLAN-300:

    Here is another example on a border leaf device in a fabric where the M-VLAN is VLAN-900:

  4. Other show commands display multicast information that isn't specific to OISM elements but can help check configured multicast options and traffic flow.

    For example, use the show evpn multicast-snooping status to see if you enabled IGMP snooping on VLANs you configured for OISM elements.

    In the following sample command, if the revenue bridge domains on a server leaf device are VLAN-100 and VLAN-200, the SBD is VLAN-300, and you enabled IGMP snooping (but not MLD snooping):

    • show evpn multicast-snooping status—See if you enabled IGMP snooping on VLANs you configured for OISM elements.

      In the following case, if the revenue bridge domains on a server leaf device are VLAN-100 and VLAN-200, the SBD is VLAN-300, and you enabled IGMP snooping (but not MLD snooping):

Release History Table
Release
Description
22.3R1-EVO
Starting in Junos OS Evolved Release 22.3R1, PTX10001-36MR, PTX10004, PTX10008, and PTX10016 routers support OISM with IGMPv2 or IGMPv3, IGMP snooping, and SMET route optimization. These devices support OISM using MAC-VRF EVPN instances with vlan-aware and vlan-based service types. You can configure any of these devices in OISM server leaf, border leaf, or lean spine roles. In the border leaf role, these devices support any of the available OISM methods to connect to an external multicast PIM domain: M-VLAN IRB method, classic L3 interface method, or non-EVPN IRB method.
22.3R1
Starting in Junos OS Release 22.3R1, EX4300-48MP and EX4400 switches support forwarding and routing multicast traffic using OISM. These devices support OISM with IGMPv2 in the default switch instance with the VLAN-aware service model. You can configure these devices only in the OISM server leaf role, and not as an OISM border leaf device.
22.2R1-EVO
Starting in Junos OS Evolved Release 22.2R1, you can enable AR with OISM in MAC-VRF EVPN instance configurations on QFX5130-32CD and QFX5700 switches. You can configure the AR leaf role or AR replicator role on these devices. The AR replicator role operates only in standalone mode (the AR replicator role can't be collocated with the OISM border leaf role on the same device). We support AR and OISM with IGMPv2 or IGMPv3, and IGMP snooping.
22.2R1-EVO
Starting in Junos OS Evolved Release 22.2R1, QFX5130-32CD and QFX5700 switches configured as AR replicators with OISM install multicast (*,G) states with IGMPv2 or multicast (S,G) states with IGMPv3 for EVPN Type 6 routes only on the SBD VLAN. On these devices, you only see multicast group routes on the SBD in show multicast snooping route command output.
22.2R1
Starting in Junos OS Release 22.2R1, we support OISM with IGMPv2 or IGMPv3 with the default switch instance or MAC-VRF EVPN instances (vlan-aware and vlan-based service types) on EX4650, QFX5110, QFX5120, QFX10002 (except QFX10002-60C), QFX10008, and QFX10016 switches. You can configure any of these devices in the OISM server leaf role. All of these devices except EX4650 and QFX5110 switches can be OISM border leaf devices. On QFX10000 Series border leaf devices, you can use either the OISM M-VLAN IRB interface method or the classic L3 interface method to connect to an external multicast PIM domain. On EX4650 and QFX5120 border leaf devices, you can use only the classic L3 interface method.
22.2R1
Starting in Junos OS Release 22.2R1, you can enable AR with OISM in the default switch instance or MAC-VRF EVPN instances on EX4650, QFX5110, QFX5120, QFX10002 (except QFX10002-60C), QFX10008, and QFX10016 switches. The AR replicator role can be collocated with the OISM border leaf role on the same device, or you can configure the AR replicator role in standalone mode on a lean spine device in the fabric. (Only switches in the QFX10000 line can be AR replicators.) We support AR and OISM with IGMPv2 or IGMPv3, and IGMP snooping.
22.1R1EVO
Starting in Junos OS Evolved Release 22.1R1, QFX5130-32CD and QFX5700 switches support OISM with IGMPv2 or IGMPv3 in MAC-VRF EVPN instances (vlan-aware and vlan-based service types). These devices can be OISM server leaf or border leaf devices. The border leaf devices support the classic L3 interface model or the non-EVPN IRB model to connect to an external multicast PIM domain.
21.2R1
Starting in Junos OS Release 21.2R1, QFX5110, QFX5120, and QFX10002 (except QFX10002-60C) switches support OISM with IGMPv2 in the default switch instance with the VLAN-aware service model. You can configure any of these devices in the OISM server leaf role, but only QFX10002 switches can be OISM border leaf devices. The border leaf devices support either the OISM M-VLAN IRB model or the classic L3 interface model to connect to an external multicast PIM domain.