Configuring 100-Gigabit Ethernet MICs/PICs

• MX Series 100-Gigabit Ethernet Interfaces
• PTX Series 100-Gigabit Ethernet Interfaces
• T Series 100-Gigabit Ethernet Interfaces

The MPC3E supports two separate slots for MICs. MICs provide the physical interface and are installed into the MPCs.

The MPC3E supports these MICs as field replaceable units (FRUs):

• 100-Gigabit Ethernet MIC with CFP (model number MIC3-3D-1X100GE-CFP)

• 100-Gigabit Ethernet MIC with CXP (model number MIC3-3D-1X100GE-CXP)

• 10-port 10-Gigabit Ethernet MIC with SFPP (model number MIC3-3D-10XGE-SFPP)

• 2-port 40-Gigabit Ethernet MIC with QSFP+ (model number MIC3-3D-2X40GE-QSFPP)

The MPC3E has two separate configurable MIC slots. Each MIC corresponds to a single PIC and the mapping between the MIC and PIC is 1 to 1 (one MIC is treated as one PIC). The MIC plugged into slot 0 corresponds to PIC 0 and the MIC plugged into slot 1 corresponds to PIC 2.

The MPC3E also supports these legacy MICs:

• 20-port Gigabit Ethernet MIC with SFP (model number MIC-3D-20GE-SFP)

• 2-port 10-Gigabit Ethernet MICs with XFP (model number MIC-3D-2XGE-XFP)

The 100-Gigabit Ethernet CFP MIC supports the IEEE standards—compliant 100BASE-LR4 interface, using the 100G CFP optical transceiver modules for connectivity. The 100-Gigabit Ethernet CXP MIC supports the 100BASE-SR10 interface, using 100-Gigabit CXP optical transceiver modules for connectivity. The 2-port 40-Gigabit Ethernet QSFPP MIC supports the 40BASE-SR4 interface and uses quad small form-factor pluggable (QSFPP) optical transceivers for connectivity. The 10-port 10-Gigabit Ethernet SFPP MIC uses SFP+ optical transceiver modules for connectivity.

For detailed information about each MIC, see 100-Gigabit Ethernet MIC with CFP, 100-Gigabit Ethernet MIC with CXP, 40-Gigabit Ethernet MIC with QSFP+. For information about supported hardware and transceivers, see MPC3E.

The MPC3E supports these features:

• Optical diagnostics and related alarms

• Virtual Router Redundancy Protocol (VRRP) support

• IEEE 802.1Q virtual LANs (VLANs) support

• Synchronous Ethernet

• Remote monitoring (RMON) and Ethernet statistics (EtherStats)

• Source MAC learning

• MAC accounting and policing—Dynamic local address learning of source MAC addresses

• Flexible Ethernet encapsulation

• Multiple Tag Protocol Identifiers (TPIDs)

For information about the supported and unsupported Junos OS features for this MPC, see “Protocols and Applications Supported by the MPC3E (MX-MPC3E)” in the MX Series Interface Module Reference.

The 100-Gigabit Ethernet PIC (model number PD-1CE-CFP-FPC4) is a 1-port 100-Gigabit Ethernet Type 4 PIC with 100-gigabit small form-factor pluggable (CFP) transceiver. This PIC is available only as packaged in an assembly with the T1600-FPC4-ES FPC. The 100-Gigabit Ethernet PIC occupies PIC slots 0 and 1 in the T1600-FPC4-ES FPC. For information about supported transceivers and hardware, see 100-Gigabit Ethernet PIC with CFP (T1600 Router).

The 100-Gigabit Ethernet PIC supports flexible encapsulation and MAC accounting.

MAC learning, MAC policing, and Layer 2 rewrite functionality are not supported.

The ingress flow can be filtered based on the VLAN source and destination addresses. Ingress frames can also be classified according to VLAN, stacked VLAN, source address, VLAN source address, and stacked VLAN source address. VLAN manipulation on egress frames are supported on both outer and inner VLAN tags.

The following features are supported:

• The following encapsulation protocols are supported:

  • Layer 2 protocols

    • Ethernet CCC, Ethernet TCC, Ethernet VPLS

    • VLAN CCC

    • Extended VLAN TCC

    • VLAN VPLS

    • Flexible Ethernet service

  • Layer 3 protocols

    • IPv4

    • Ipv6

    • MPLS

• CFP MSA compliant MDIO control features (transceiver dependent).

• Graceful Routing Engine switchover (GRES) is supported in all PIC and chassis configurations.

• Interface creation:

  • When the PIC, is brought online, the router creates two 50 gigabit capable interfaces, et-x/0/0:0 and et-x/0/0:1, where x represents the FPC slot number. Each physical interface represents two internal 50 gigabit Ethernet Packet Forwarding Engines. Two logical interfaces are configured under each physical interface.

  • Packet Forwarding Engine 0 is physical interface 0, Packet Forwarding Engine 1 is physical interface 1

• 802.3 link aggregation:

  Same rate or same mode link aggregation:

  • Two logical interfaces are created for each 100-Gigabit Ethernet PIC. To utilize bandwidth beyond 50 gigabits per second, an aggregate interface must be explicitly configured on the 100-Gigabit Ethernet PIC that includes the two 50 gigabit interfaces.

  • Each 100 gigabit Ethernet aggregate consumes one of the router-wide aggregated Ethernet device pools. The number of 100-Gigabit Ethernet PICs cannot exceed the router-wide limit, which is 128 for Ethernet.

  • In each aggregate bundle, each 100-Gigabit Ethernet PIC consumes two members. Hence, an aggregate bundle that consists purely of 100-Gigabit Ethernet PICs supports a maximum of half of the software limit for the number of members. Therefore, with a maximum of 16 links, up to 8 100-Gigabit Ethernet links are supported.

  • Combining 100-Gigabit Ethernet PICs into aggregate interfaces with other Ethernet PICs is not permitted. However, other Ethernet PICs can also be configured within the same T1600 with 100-Gigabit Ethernet PICs, and used in separate aggregate interfaces.

  • Multiple (Juniper Networks) Type 4 100-Gigabit Ethernet PICs on a T1600 router can be combined into a static aggregated Ethernet bundle to connect to a different type of 100 gigabit Ethernet PIC on a remote router (Juniper Networks or other vendors). LACP is not supported in this configuration.

  Mixed rate or mixed mode link aggregation:

  • Starting with Junos OS Release 13.2, aggregated Ethernet supports mixed rates and mixed modes on 100-Gigabit Ethernet PIC.

  • Static link protection and Link Aggregation Control Protocol (LACP) is supported on mixed aggregated Ethernet link configured on a 100-Gigabit Ethernet PIC.

  • When configuring a mixed aggregated Ethernet link on a 100-Gigabit Ethernet PIC, ensure that you add both the 50-Gigabit Ethernet interfaces of the 100-Gigabit Ethernet PIC to the aggregated Ethernet bundle. Moreover, both these 50-Gigabit Ethernet interfaces must be included in the same aggregated Ethernet bundle.

  • For a single physical link event of an aggregated Ethernet link configured on a 100-Gigabit Ethernet PIC, the packet loss performance value is twice the original value because of the two 50-Gigabit Ethernet interfaces of the 100-Gigabit Ethernet PIC.

• Software Packet Forwarding Engine—Supports all Gigabit Ethernet PIC classification, firewall filter, queuing model, and rewrite functionality.

• Egress traffic performance—Maximum egress throughput is 100 gigabits per second on the physical interface, with 50 gigabits per second on the two assigned logical interfaces.

• Ingress traffic performance—Maximum ingress throughput is 100 gigabits per second on the physical interface, with 50 gigabits per second on the two assigned logical interfaces. To achieve 100 gigabits per second ingress traffic performance, use one of the interoperability modes described below. For example, if VLAN steering mode is not used when connecting to a remote 100 gigabits per second interface (that is on a different 100 gigabits per second PIC on a Juniper Networks router or a different vendor’s equipment), then all ingress traffic will try to use one of the 50 gigabits per second Packet Forwarding Engines, rather than be distributed among the two 50 gigabits per second Packet Forwarding Engines, resulting in a total of 50 gigabits per second ingress performance.

• Interoperability modes—The 100-Gigabit Ethernet PIC supports interoperability with through configuration in one of the following two forwarding option modes:

  • SA multicast mode—In this mode, the 100-Gigabit Ethernet PIC supports interconnection with other Juniper Networks 100-Gigabit Ethernet PICs (Model: PD-1CE-CFP) interfaces only.

  • VLAN steering mode—In this mode, the 100-Gigabit Ethernet Type 4 PIC with CFP supports interoperability with 100 gigabit Ethernet interfaces from other vendors only.

The following features are supported on 100-Gigabit Ethernet Type 5 PIC with CFP:

• Access to all 100-Gigabit Ethernet port counters through SNMP.

• Logical interface–level MAC filtering, accounting, policing, and learning for source media access control (MAC).

• Channels defined by two stacked VLAN tags.

• Channels defined by flex-vlan-tagging.

• IP service for stacked VLAN tags.

• Defining the rewrite operation to be applied to the incoming and outgoing frames on logical interfaces on this PIC.

  Note:

  Only the Tag Protocol Identifier (TPID) 0x8100 is supported.

• Interface encapsulations, such as the following:

  • untagged—Default encapsulation, when other encapsulation is not configured.

    • You can configure only one logical interface (unit 0) on the port.

    • You cannot include the vlan-id statement in the configuration of the logical interface.

  • vlan-tagging—Enable VLAN tagging for all logical interfaces on the physical interface.

  • stacked-vlan-tagging—Enable stacked VLAN tagging for all logical interfaces on the physical interface.

  • ethernet-ccc—Ethernet cross-connect.

  • ethernet-tcc—Ethernet translational cross-connect.

  • vlan-ccc—802.1Q tagging for a cross-connect.

  • vlan-tcc—Virtual LAN (VLAN) translational cross-connect.

  • extended-vlan-ccc—Standard TPID tagging for an Ethernet cross-connect.

  • extended-vlan-tcc—Standard TPID tagging for an Ethernet translational cross-connect.

  • flexible-ethernet-services—Allows per-unit Ethernet encapsulation configuration.

  • ethernet-vpls—Ethernet virtual private LAN service.

  • vlan-vpls—VLAN virtual private LAN service.

• The following Layer 3 protocols are also supported:

  • IPv4

  • IPv6

  • MPLS

• CFP Multi-Source Agreement (MSA) compliant Management Data Input/Output (MDIO) control features (transceiver dependent).

• 802.3 link aggregation:

  • The configuration of the 100-Gigabit Ethernet Type 5 PIC with CFP complies with that of the existing 1-Gigabit or 10-Gigabit Ethernet PIC and aggregated Ethernet interfaces.

• Interoperability mode—Interoperability with the 100-Gigabit Ethernet Type 4 PIC with CFP through configuration in sa-multicast forwarding mode.

• Juniper Networks enterprise-specific Ethernet Media Access Control (MAC) MIB

• The 100-Gigabit Ethernet Type 5 PIC with CFP supports all Gigabit Ethernet PIC classification, firewall filters, queuing model, and Layer 2 rewrite functionality features of the Gigabit Ethernet PICs. To configure these parameters, see Configuring Gigabit Ethernet Policers, Configuring Gigabit Ethernet Policers, and Stacking and Rewriting Gigabit Ethernet VLAN Tags Overview.

• A Type 5 FPC can support up to two 100-Gigabit Ethernet PICs. Both the PICs (that is, PIC 0 and PIC 1) can be offline or online independently.

The following features are not supported on the 100-Gigabit Ethernet Type 5 PIC with CFP:

• MAC filtering, accounting, and policing for destination MAC at the logical interface level.

  Note:

  Because destination MAC filtering is not supported, the hardware is configured to accept all the multicast packets. This configuration enables the OSPF protocol to work.

• Premium MAC policers at the logical interface level.

• MAC filtering, accounting, and policing at the physical interface level.

• Multiple TPIDs.

• IP service for nonstandard TPID.

Table 4 lists the capabilities of 100-Gigabit Ethernet Type 5 PIC with CFP.

Juniper Networks Junos operating system (Junos OS) supports a variety of 100-Gigabit Ethernet interfaces. The 100-Gigabit Ethernet standard, introduced by IEEE 802.3ba-2010, enables transmission of Ethernet frames at the rate of 100 gigabits per second (Gbps). It is used for very high speed transmission of voice and data signals across the numerous world-wide fiber-optic networks.

Interface interoperability refers to the ability of an interface to interoperate with other router interfaces. You can enable interoperability between different 100-Gigabit Ethernet interfaces by performing specific configuration tasks. The following sections list the 100-Gigabit Ethernet interfaces, corresponding interoperable interfaces, and links to the interoperability tasks and reference information.

• Interoperability of the MIC-3D-1X100GE-CFP MIC with PICs on Other Routers
• Interoperability of the MPC4E-3D-2CGE-8XGE MPC with PICs on Other Routers
• Interoperability of the P1-PTX-2-100GE-CFP PIC with PICs on Other Routers
• Interoperability of the PD-1CE-CFP-FPC4 PIC with PICs or MICs on Other Routers

You can enable interoperability between the 100-Gigabit Ethernet PICs PD-1CE-CFP-FPC4 and PF-1CGE-CFP by:

• Enabling source address (SA) multicast bit steering mode on the 100-Gigabit Ethernet PIC PF-1CGE-CFP.

• Configuring the two 50-Gigabit Ethernet physical interfaces on the 100-Gigabit Ethernet PIC PD-1CE-CFP-FPC4 as one aggregated Ethernet physical interface.

SA multicast mode uses the multicast bit in the source MAC address for packet steering. By default, the SA multicast bit is set to 0 for all packets sent by the 100-Gigabit Ethernet PIC PF-1CGE-CFP. The 100-Gigabit Ethernet PIC PD-1CE-CFP-FPC4 looks at the bit and forwards the packets to either Packet Forwarding Engine 0 or Packet Forwarding Engine 1. When the PIC sends out a packet, the multicast bit is set based on the egress Packet Forwarding Engine number (0 or 1).

The default packet steering mode for PD-1CE-CFP-FPC4 is SA multicast bit mode. No SA multicast configuration is required to enable this mode.

PD-1CE-CFP-FPC4 uses two 50 Gpbs Packet Forwarding Engines to achieve 100 Gbps throughput. The 50-Gigabit Ethernet physical interfaces are created when the 100-Gigabit Ethernet PIC is plugged in. The two physical interfaces are visible and configuration is allowed on both the physical interfaces. You must configure the physical interfaces on PD-1CE-CFP-FPC4 in static link aggregation group (LAG) mode without enabling Link Aggregation Control Protocol (LACP). This ensures that a single 100-Gigabit aggregated interface is visible on the link connecting to the 100-Gigabit Ethernet PIC PF-1CGE-CFP instead of two independent 50-Gigabit Ethernet interfaces.

You can enable interoperability between the 100-Gigabit Ethernet PIC PD-1CE-CFP-FPC4 and the 100-Gigabit Ethernet PIC P1-PTX-2-100GE-CFP by:

• Configuring the two 50-Gigabit Ethernet physical interfaces on the 100-Gigabit Ethernet PIC PD-1CE-CFP-FPC4 as one aggregated Ethernet physical interface.

• Configuring source address (SA) multicast bit steering mode on the 100-Gigabit Ethernet PIC P1-PTX-2-100GE-CFP.

SA multicast bit steering mode uses the multicast bit in the source MAC address for packet steering.

The 100-Gigabit Ethernet PIC PD-1CE-CFP-FPC4 uses two 50-Gpbs Packet Forwarding Engines to achieve 100-Gbps throughput. The 50-Gigabit Ethernet physical interfaces are created when the 100-Gigabit Ethernet PIC is plugged in. The two physical interfaces are visible and configuration is allowed on both the physical interfaces. You must configure the physical interfaces on the 100-Gigabit Ethernet PIC PD-1CE-CFP-FPC4 in static link aggregation group (LAG) mode without enabling Link Aggregation Control Protocol (LACP). This ensures that a single 100-Gigabit aggregated interface is visible on the link connecting to the 100-Gigabit Ethernet PIC P1-PTX-2-100GE-CFP.

On the 100-Gigabit Ethernet PIC PD-1CE-CFP-FPC4, ingress packets are forwarded to either Packet Forwarding Engine number 0 or 1 based on the SA multicast bit in the received packet. The SA multicast bit of egress packets is set based on whether the packet is forwarded from Packet Forwarding Engine number 0 or 1. As the default packet steering mode is SA multicast bit steering mode, no configuration is necessary to enable this mode.

On the 100-Gigabit Ethernet PIC P1-PTX-2-100GE-CFP, the SA multicast bit is ignored in ingress packets. When SA multicast bit steering mode is enabled, the SA multicast bit in the egress packets is set to 0 or 1 based on the flow hash value that is computed internally by the Packet Forwarding Engine complex for each packet. No CLI configuration is required to generate the flow hash value as this computation is done automatically. The flow hash algorithm uses fields in the packet header to compute the flow hash value. By default, the SA multicast bit is set to 0 in egress packets. You must configure SA multicast bit steering mode to enable interoperability with the 100-Gigabit Ethernet PIC PD-1CE-CFP-FPC4 .