Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
 

Example: Configure PFC Across L3 Interfaces

Priority-based flow control (PFC) helps ensure lossless transport across DCB interfaces by pausing incoming traffic when output queue buffers fill to a certain threshold. In addition to configuring PFC on L2 (bridging) interfaces, you can configure PFC on VLAN-tagged traffic that traverses L3 interfaces. This enables you to preserve the lossless characteristics that PFC provides on VLAN-tagged traffic, even when the traffic crosses L3 interfaces that connect two L2 networks.

Note:

This topic is applicable for VLAN-tagged traffic only. On supported platforms, you can also configure DSCP-based PFC for untagged traffic on L3 interfaces and L2 access interfaces. DSCP-based PFC uses a DSCP classifier to classify the traffic based on a 6-bit DSCP value that is mapped to a 3-bit PFC priority value. For details on configuring DSCP-based PFC on supporting switches, see Configuring DSCP-based PFC for Layer 3 Untagged Traffic.

Requirements

This example uses the following hardware and software components:

  • Two switches

  • Junos OS Release 13.2 or later for the QFX Series Switches

  • Two Ethernet hosts

Overview

On a network that uses two QFX Series switches to connect hosts on two different Ethernet networks across an L3 network, to configure PFC across the L2 and L3 interfaces, you must:

  • Configure the L2 and L3 interfaces on the switches

  • Configure VLANs to carry the traffic across the L2 and L3 networks

  • Configure integrated routing and bridging (IRB) interfaces on the L2 interfaces to move the L2 VLAN traffic to L3

  • Configure and apply the appropriate classifiers to the interfaces

  • Configure and apply CNPs on the interfaces to enable PFC on the traffic that you want to be lossless

    Note:

    Configuring or changing PFC on an interface blocks the entire port until the PFC change is completed. After a PFC change is complete, the port is unblocked and traffic resumes. Blocking the port stops ingress and egress traffic, and causes packet loss on all queues on the port until the port is unblocked.

  • Configure lossless forwarding classes and either hierarchical port scheduling (also known as enhanced transmission selection) or direct port scheduling, depending on your switch, on the interfaces

Note:

PFC operates at the queue level, based on the IEEE 802.1p code point in the priority code point (PCP) field of the Ethernet frame header (sometimes known as the CoS bits). For this reason, VLAN-tagged traffic on L3 interfaces on which you want to enable PFC must use an IEEE 802.1p classifier to map incoming traffic to forwarding classes and loss priorities. You cannot use a DSCP or DSCP IPv6 classifier to classify L3 traffic if you want to enable PFC on VLAN-tagged traffic flows.

Topology

Figure 1 shows the topology for this example.

Figure 1: Enabling PFC Across Layer 3 Interface HopsNetwork topology with two Juniper QFX5100 switches, SW1 and SW2, connecting to two Ethernet hosts, Host 1 and Host 2. Host 1 connects to SW1 via ports xe-0/0/20 and xe-0/0/21; Host 2 connects to SW2 via the same ports. SW1 and SW2 interconnect via ports xe-0/0/40 and xe-0/0/41. Ports xe-0/0/20 and xe-0/0/21 operate on Layer 2 with IRB and PFC, while ports xe-0/0/40 and xe-0/0/41 are on Layer 3 with IEEE classifiers and PFC.

Table 1 shows the configuration components for this example. On the two switches, the Ethernet host-facing interfaces use the same interface names and configuration, and the Layer 3 network-facing interfaces use the same interface names and configuration.

Table 1: Components of the PFC Across Layer 3 Interfaces Topology

Component

Settings

Hardware

Two switches, Switch SW1 and Switch SW2.Two Ethernet hosts

L3 interfaces (xe-0/0/40 and xe-0/0/41) and VLANs

Interface xe-0/0/40:

  • Interface family—inet

  • Interface IP address—100.103.1.2/24

  • VLAN tagging—enabled

  • Interface VLAN ID—103

Interface xe-0/0/41:

  • Interface family—inet

  • Interface IP address—100.104.1.2/24

  • VLAN tagging—enabled

  • Interface VLAN ID—104

L2 interfaces (xe-0/0/20 and xe-0/0/21) and VLAN membership

Family: Ethernet switchingInterface mode—trunkInterface xe-0/0/20 VLAN membership—vlan105Interface xe-0/0/21 VLAN membership—vlan106

VLANs for the IRB interfaces

VLAN unit 105—family inet, IP address 100.105.1.1/24VLAN unit 106—family inet, IP address 100.106.1.1/24

L2 IRB interfaces

Interface xe-0/0/20:

  • IRB interface unit—105

  • IRB interface family—inet

  • IRB interface IP address—100.105.1.1/24

  • IRB interface VLAN ID—105

  • L3 interface name—irb.105

Interface xe-0/0/21:

  • IRB interface unit—106

  • IRB interface family—inet

  • IRB interface IP address—100.106.1.1/24

  • IRB interface VLAN ID—106

  • L3 interface name—irb.106

Forwarding classes (both switches)

Name—lossless-3Queue mapping—queue 3Packet drop attribute—no-loss

Name—lossless-4Queue mapping—queue 4Packet drop attribute—no-loss

Note:

Matching the forwarding class names (lossless-3 and lossless-4) to the queue number and to the classified IEEE 802.1p code point (priority) creates a configuration that is logical and easy to map because the forwarding class, queue, and priority all use the same number.

Name—all-othersQueue mapping—queue 0Packet drop attribute—none

Note:

The forwarding class all-others is for best-effort traffic that traverses the interfaces.

L2 interface BA classifier

Name—lossless-3-4-ieeeForwarding class lossless-3—mapped to code point 011 (IEEE 802.1p priority 3) and a PLP of lowForwarding class lossless-4—mapped to code point 100 (IEEE 802.1p priority 4) and a PLP of low

Apply the L2 IEEE 802.1p classifier to both the L2 and the L3 interfaces (xe-0/0/20, xe-0/0/21, xe-0/040, and xe-0/0/41).

CNP (PFC, both switches)

Name—lossless-cnpPFC enabled on IEEE 802.1p code points—011 (lossless-3 forwarding class and priority), 100 (lossless-4 forwarding class and priority)

Apply the CNP to both the L2 and the L3 interfaces (xe-0/0/20, xe-0/0/21, xe-0/040, and xe-0/0/41) to enable PFC on IEEE 802.1p code points 011 and 100.

Enhanced transmission selection (ETS) hierarchical port scheduling (only if using ETS)

Hierarchical port scheduling (ETS) includes configuring:

  • Schedulers to assign bandwidth to traffic

  • Scheduler mapping to forwarding classes

  • Grouping of the forwarding classes (priorities) in forwarding class sets (priority groups)

  • A TCP to assign bandwidth to the forwarding class set and to associate the forwarding class set with the scheduler mapping

Hierarchical port scheduling also includes applying the hierarchical scheduler (defined in the TCP) to the interfaces.

This example focuses on configuring PFC across the L2 and L3 interfaces. To maintain this focus, this example includes the CLI statements needed to configure hierarchical port scheduling, but does not include descriptive explanations of the configuration. The Related Documentation section provides links to example documents that show how to configure hierarchical port scheduling.

Apply the scheduling configuration to both the L2 and the L3 interfaces (xe-0/0/20, xe-0/0/21, xe-0/040, and xe-0/0/41).

Direct port scheduling (only if using port scheduling instead of ETS)

Direct port scheduling includes configuring:

  • Schedulers to assign bandwidth to traffic

  • Scheduler mapping to forwarding classes

Port scheduling also includes applying the scheduler map to the interfaces.

This example focuses on configuring PFC across the L2 and L3 interfaces. To maintain this focus, this example includes the CLI statements needed to configure direct port scheduling, but does not include descriptive explanations of the configuration. The Related Documentation section provides links to example documents that show how to configure port scheduling.

Apply the scheduling configuration to both the L2 and the L3 interfaces (xe-0/0/20, xe-0/0/21, xe-0/040, and xe-0/0/41).

Configuration

CLI Quick Configuration

To configure PFC across L3 interfaces, copy the following commands, paste them in a text file, remove the line breaks, change variables and details to match your network configuration, and then copy and paste the commands into the CLI at the [edit] hierarchy level. The same configuration applies to both Switch SW1 and Switch SW2. The configuration is separated into the configuration common to ETS and direct port scheduling, and the portions of the configuration that apply only to ETS and only to port scheduling.

Common Configuration (Applies to ETS Hierarchical Scheduling and to Port Scheduling)

Configuration for ETS Hierarchical Scheduling

The ETS-specific portion of this example configures forwarding class set (priority group) membership and priority group CoS settings (TCP), and assigns the priority group and its CoS configuration to the interfaces.

Configuration for Port Scheduling

The port-scheduling-specific portion of this example assigns the scheduler maps (which set the CoS treatment of the forwarding classes in the scheduler map) to the interfaces.

Common Configuration (Applies to ETS Hierarchical Scheduling and to Port Scheduling)

Step-by-Step Procedure

The following procedure shows you how to configure the VLANs, IRB interfaces, lossless forwarding classes, classifiers, PFC settings to enable PFC across L3 interfaces, and the queue scheduling configuration common to ETS and direct port scheduling. For completeness, the ETS hierarchical port scheduling and direct port scheduling configurations are included separately, in the following procedures, but without explanatory text. See the Related Documentation links for detailed examples of the scheduling elements of the configuration.

  1. Configure the L3 interface VLANs and IP addresses:

  2. Configure the L2 interface VLAN membership and interface mode:

  3. Configure the IRB interfaces and VLANs to transport incoming L2 traffic assigned to VLANs vlan105 (of which interface xe-0/0/20 is a member) and vlan106 (of which interface xe-0/0/21 is a member) across L3:

  4. Configure the lossless forwarding classes and a BE forwarding class for any other traffic that might use the interfaces:

  5. Configure the IEEE classifier for the L2 and L3 interfaces to classify incoming traffic into the lossless forwarding classes based on the IEEE 802.1p code point of the traffic:

  6. Configure the CNP to enable PFC on the lossless priorities (the lossless forwarding classes mapped to IEEE 802.1p code points 3 and 4):

  7. Apply the L2 IEEE 802.1p classifier and the CNP to the L3 interfaces:

  8. Apply the L2 IEEE 802.1p classifier and the CNP to the L2 interfaces:

  9. Configure queue scheduling to support the lossless configuration and map the schedulers to the forwarding classes:

ETS Hierarchical Scheduling Configuration

Step-by-Step Procedure

  1. Configure hierarchical scheduling to support the lossless configuration and apply it to the L2 and L3 interfaces:

Port Scheduling Configuration

Step-by-Step Procedure

  1. Apply port scheduling to support the lossless configuration on interfaces:

Results

Display the results of the interface, VLAN, and CoS configurations. The system shows only the explicitly configured parameters; it does not show default parameters. The results are valid for both Switch SW1 and Switch SW2 because the same configuration is used on both switches. The results are from the ETS hierarchical scheduling configuration, which show the more complex configuration. Direct port scheduling results would not show the TCP or forwarding class set portions of the configuration, but would display the name of the scheduler map under each interface. Other than that, the results are the same.

Display the results of the interface configuration:

Display the results of the vlan configuration:

Display the results of the CoS configuration:

Tip:

To quickly configure the switch, issue the load merge terminal command, and then copy the hierarchies and paste them into the switch terminal window.

Verification

To verify that the PFC across L3 interfaces configuration has been created and is operating properly, perform these tasks:

Verifying the Interface Configuration

Purpose

Verify that the L2 Ethernet interfaces, L3 IP interfaces, IRB interfaces, and VLAN interfaces have been created on the switch and are correctly configured.

Action

Display the switch interface configuration using the show configuration interfaces command:

Meaning

The show configuration interfaces command displays all of the interfaces configured on the switch. The command output shows that:

  • Interfaces xe-0/0/20 and xe-0/0/21 are Ethernet interfaces (family ethernet-switching) in trunk interface mode. Interface xe-0/0/20 is a member of VLAN vlan105, and interface xe-0/0/21 is a member of VLAN vlan106.

  • Interfaces xe-0/0/40 and xe-0/0/41 are IP interfaces (family inet) with VLAN tagging enabled. Interface xe-0/0/40 has an IP address of 100.103.1.2/24 and a VLAN ID of 103. Interface xe-0/0/41 has an IP address of 100.104.1.2/24 and a VLAN ID of 104.

  • Two IRB interfaces are configured, IRB unit 105 with an IP address of 100.105.1.1/24 and IRB unit 106 with an IP address of 100.106.1.1/24.

  • Two VLAN interfaces are configured: VLAN unit 105 with an IP address of 100.105.1.1/24 (for IRB interface unit 105) and VLAN unit 106 with an IP address of 100.106.1.1/24 (for IRB interface unit 106).

Verifying the VLAN Configuration

Purpose

Verify that VLANs have been created on the switch and are correctly configured.

Action

Display the VLAN configuration using the show configuration vlans command:

Meaning

The show configuration vlans command displays all of the VLANs configured on the switch. The command output shows that:

  • VLAN vlan105 has been configured with VLAN ID 105 on IRB interface irb.105.

  • VLAN vlan106 has been configured with VLAN ID 106 on IRB interface irb.106.

Verifying the PFC Configuration (CNP)

Purpose

Verify that PFC has been enabled on the correct IEEE 802.1p code points (priorities) in the CNP.

Action

Display the PFC configuration using the show configuration class-of-service congestion-notification-profile command:

Meaning

The show configuration class-of-service congestion-notification-profile command displays all of the CNPs configured on the switch. The command output shows that:

  • The CNP named lossless-cnp is configured on the switch.

  • The CNP lossless-cnp enables PFC on IEEE 802.1p code points 100 and 100.

Verify the Forwarding Class Configuration

Purpose

Verify that the two lossless forwarding classes and the BE forwarding class have been configured on the switch.

Action

Display the forwarding class configuration using the show configuration class-of-service forwarding-classes command:

Meaning

The show configuration class-of-service forwarding-classes command displays all of the forwarding classes configured on the switch. Default forwarding classes are not displayed. The command output shows that:

  • Forwarding class lossless-3 is mapped to queue 3 and is configured as a lossless forwarding class (the no-loss attribute is applied)

  • Forwarding class lossless-4 is mapped to queue 4 and is configured as a lossless forwarding class (the no-loss attribute is applied)

  • Forwarding class all-others is mapped to queue 0. It is not a lossless forwarding class (the no-loss attribute is not applied).

Verifying the Classifier Configuration

Purpose

Verify that the IEEE 802.1p classifier has been configured on the switch.

Action

Display the classifier configuration using the show configuration class-of-service classifiers command:

Meaning

The show configuration class-of-service classifiers command displays all of the classifiers configured on the switch. The command output shows that the L2 IEEE 802.1p classifier lossless-3-4-ieee classifies traffic with the code point 011 into the lossless-3 forwarding class with a loss priority of low, and classifies traffic with the code point 100 into the lossless-4 forwarding class with a loss priority of low.

Verifying the Interface CoS Configuration (Hierarchical Scheduling, PFC, and Classifier Mapping to Interfaces)

Purpose

Verify that the interfaces have the correct hierarchical scheduling, PFC, and classifier configurations.

Note:

The results are from the ETS hierarchical scheduling configuration, which shows the more complex configuration. Direct port scheduling results do not show the TCP or forwarding class set portions of the interface configuration, but rather display the name of the scheduler map under each interface.

Action

Display the interface CoS configuration using the show configuration class-of-service interfaces command:

Meaning

The show configuration class-of-service interfaces command displays all of the CoS components configured on the switch interfaces. The command output shows that:

  • The configuration on L2 Ethernet interfaces xe-0/0/20 and xe-0/0/21 includes:

    • Hierarchical scheduling—The forwarding class set lossless_fc_set with the TCP lossless_tcp for the lossless traffic, and the forwarding class set all-others_fc_set with the TCP all-others_tcp for the best-effort traffic are applied to both interfaces.

    • PFC—The lossless-cnp CNP is applied to both interfaces.

    • Classifiers—The L2 IEEE 802.1p classifier lossless-3-4-ieee is applied to both interfaces.

  • The configuration on L3 IP interfaces xe-0/0/40 and xe-0/0/41 includes:

    • Hierarchical scheduling—The forwarding class set lossless_fc_set with the TCP lossless_tcp for the lossless traffic, and the forwarding class set all-others_fc_set with the TCP all-others_tcp for the best-effort traffic are applied to both interfaces.

    • PFC—The lossless-cnp CNP is applied to both interfaces.

    • Classifiers—The L2 IEEE 802.1p classifier lossless-3-4-ieee is applied to both interfaces. Traffic that would use a DSCP or a DSCP IPv6 classifier if it were configured uses the IEEE 802.1p classifier instead. Using the IEEE 802.1p classifier allows the interface to use PFC to pause traffic during periods of congestion to prevent packet loss.