Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
 

Example: Configuring PFC Across Layer 3 Interfaces

Priority-based flow control (PFC) helps ensure lossless transport across data center bridging interfaces by pausing incoming traffic when output queue buffers fill to a certain threshold. On a QFX5210, QFX5200, QFX5110, QFX5100, EX4600, or QFX10000 switch running the Enhanced Layer 2 Software (ELS) CLI, in addition to configuring PFC on Layer 2 (bridging) interfaces, you can configure PFC on VLAN-tagged traffic that traverses Layer 3 interfaces. This enables you to preserve the lossless characteristics that PFC provides on VLAN-tagged traffic, even when the traffic crosses Layer 3 interfaces that connect two Layer 2 networks.

Note:

This topic is applicable for VLAN-tagged traffic only. Starting in Junos OS Release 17.4R1, QFX5110, QFX5200, and QFX5210 switches also support DSCP-based PFC for untagged traffic on Layer 3 interfaces and Layer 2 access interfaces. DSCP-based PFC uses a DSCP classifier to classify the traffic based on a 6-bit DSCP value that is mapped to a 3-bit PFC priority value. For details on configuring DSCP-based PFC on supporting switches, see Configuring DSCP-based PFC for Layer 3 Untagged Traffic.

Requirements

This example uses the following hardware and software components:

  • Two switches

  • Junos OS Release 13.2 or later for the QFX Series

  • Two Ethernet hosts

Overview

On a network that uses two QFX5210, QFX5200, QFX5110, QFX5100, EX4600, or QFX10000 switches to connect hosts on two different Ethernet networks across a Layer 3 network, to configure PFC across the Layer 2 and Layer 3 interfaces, you must:

  • Configure the Layer 2 and Layer 3 interfaces on the switches

  • Configure VLANs to carry the traffic across the Layer 2 and Layer 3 networks

  • Configure integrated routing and bridging (IRB) interfaces on the Layer 2 interfaces to move the Layer 2 VLAN traffic to Layer 3

  • Configure and apply the appropriate classifiers to the interfaces

  • Configure and apply congestion notification profiles (CNPs) on the interfaces to enable PFC on the traffic that you want to be lossless

    Note:

    Configuring or changing PFC on an interface blocks the entire port until the PFC change is completed. After a PFC change is completed, the port is unblocked and traffic resumes. Blocking the port stops ingress and egress traffic, and causes packet loss on all queues on the port until the port is unblocked.

  • Configure lossless forwarding classes and either hierarchical port scheduling (also known as enhanced transmission selection) or direct port scheduling, depending on your switch, on the interfaces

Note:

PFC operates at the queue level, based on the IEEE 802.1p code point in the priority code point (PCP) field of the Ethernet frame header (sometimes known as the CoS bits). For this reason, VLAN-tagged traffic on Layer 3 interfaces on which you want to enable PFC must use an IEEE 802.1p classifier to map incoming traffic to forwarding classes (which are in turn mapped to output queues) and loss priorities. You cannot use a DSCP or DSCP IPv6 classifier to classify Layer 3 traffic if you want to enable PFC on VLAN-tagged traffic flows.

Topology

Figure 1 shows the topology for this example.

Figure 1: Enabling PFC Across Layer 3 Interface HopsEnabling PFC Across Layer 3 Interface Hops

Table 1 shows the configuration components for this example. On the two switches, the Ethernet host-facing interfaces use the same interface names and configuration, and the Layer 3 network-facing interfaces use the same interface names and configuration.

Table 1: Components of the PFC Across Layer 3 Interfaces Topology

Component

Settings

Hardware

Two switches, Switch SW1 and Switch SW2.

Two Ethernet hosts

Layer 3 interfaces (xe-0/0/40 and xe-0/0/41) and VLANs

Interface xe-0/0/40:

  • Interface family—inet

  • Interface IP address—100.103.1.2/24

  • VLAN tagging—enabled

  • Interface VLAN ID—103

Interface xe-0/0/41:

  • Interface family—inet

  • Interface IP address—100.104.1.2/24

  • VLAN tagging—enabled

  • Interface VLAN ID—104

Layer 2 interfaces (xe-0/0/20 and xe-0/0/21) and VLAN membership

Family: Ethernet switching

Interface mode—trunk

Interface xe-0/0/20 VLAN membership—vlan105

Interface xe-0/0/21 VLAN membership—vlan106

VLANs for the IRB interfaces

VLAN unit 105—family inet, IP address 100.105.1.1/24

VLAN unit 106—family inet, IP address 100.106.1.1/24

Layer 2 IRB interfaces

Interface xe-0/0/20:

  • IRB interface unit—105

  • IRB interface family—inet

  • IRB interface IP address—100.105.1.1/24

  • IRB interface VLAN ID—105

  • Layer 3 interface name—irb.105

Interface xe-0/0/21:

  • IRB interface unit—106

  • IRB interface family—inet

  • IRB interface IP address—100.106.1.1/24

  • IRB interface VLAN ID—106

  • Layer 3 interface name—irb.106

Forwarding classes (both switches)

Name—lossless-3

Queue mapping—queue 3

Packet drop attribute—no-loss

Name—lossless-4

Queue mapping—queue 4

Packet drop attribute—no-loss

Note:

Matching the forwarding class names (lossless-3 and lossless-4) to the queue number and to the classified IEEE 802.1p code point (priority) creates a configuration that is logical and easy to map because the forwarding class, queue, and priority all use the same number.

Name—all-others

Queue mapping—queue 0

Packet drop attribute—none

Note:

The forwarding class all-others is for best-effort traffic that traverses the interfaces.

Layer 2 interface behavior aggregate (BA) classifier

Name—lossless-3-4-ieee

Forwarding class lossless-3—mapped to code point 011 (IEEE 802.1p priority 3) and a packet loss priority of low

Forwarding class lossless-4—mapped to code point 100 (IEEE 802.1p priority 4) and a packet loss priority of low

Apply the Layer 2 IEEE 802.1p classifier to both the Layer 2 and the Layer 3 interfaces (xe-0/0/20, xe-0/0/21, xe-0/040, and xe-0/0/41).

Congestion notification profile (PFC, both switches)

Name—lossless-cnp

PFC enabled on IEEE 802.1p code points—011 (lossless-3 forwarding class and priority), 100 (lossless-4 forwarding class and priority)

Apply the CNP to both the Layer 2 and the Layer 3 interfaces (xe-0/0/20, xe-0/0/21, xe-0/040, and xe-0/0/41) to enable PFC on IEEE 802.1p code points 011 and 100.

Enhanced transmission selection (ETS) hierarchical port scheduling (only if using ETS)

Hierarchical port scheduling (ETS) includes configuring:

  • Schedulers to assign bandwidth to traffic

  • Scheduler mapping to forwarding classes

  • Grouping of the forwarding classes (priorities) in forwarding class sets (priority groups)

  • A traffic control profile to assign bandwidth to the forwarding class set and to associate the forwarding class set with the scheduler mapping

Hierarchical port scheduling also includes applying the hierarchical scheduler (defined in the traffic control profile) to the interfaces.

This example focuses on configuring PFC across the Layer 2 and Layer 3 interfaces. To maintain this focus, this example includes the CLI statements needed to configure hierarchical port scheduling, but does not include descriptive explanations of the configuration. The Related Documentation section provides links to example documents that show how to configure hierarchical port scheduling.

Apply the scheduling configuration to both the Layer 2 and the Layer 3 interfaces (xe-0/0/20, xe-0/0/21, xe-0/040, and xe-0/0/41).

Direct port scheduling (only if using port scheduling instead of ETS)

Direct port scheduling includes configuring:

  • Schedulers to assign bandwidth to traffic

  • Scheduler mapping to forwarding classes

Port scheduling also includes applying the scheduler map to the interfaces.

This example focuses on configuring PFC across the Layer 2 and Layer 3 interfaces. To maintain this focus, this example includes the CLI statements needed to configure direct port scheduling, but does not include descriptive explanations of the configuration. The Related Documentation section provides links to example documents that show how to configure port scheduling.

Apply the scheduling configuration to both the Layer 2 and the Layer 3 interfaces (xe-0/0/20, xe-0/0/21, xe-0/040, and xe-0/0/41).

Configuration

CLI Quick Configuration

To configure PFC across Layer 3 interfaces, copy the following commands, paste them in a text file, remove the line breaks, change variables and details to match your network configuration, and then copy and paste the commands into the CLI at the [edit] hierarchy level. The same configuration applies to both Switch SW1 and Switch SW2. The configuration is separated into the configuration common to ETS and direct port scheduling, and the portions of the configuration that apply only to ETS and only to port scheduling.

Common Configuration (Applies to ETS Hierarchical Scheduling and to Port Scheduling)

Configuration for ETS Hierarchical Scheduling

The ETS-specific portion of this example configures forwarding class set (priority group) membership and priority group CoS settings (traffic control profile), and assigns the priority group and its CoS configuration to the interfaces.

Configuration for Port Scheduling

The port-scheduling-specific portion of this example assigns the scheduler maps (which set the CoS treatment of the forwarding classes in the scheduler map) to the interfaces.

Common Configuration (Applies to ETS Hierarchical Scheduling and to Port Scheduling)

Step-by-Step Procedure

The following step-by-step procedure shows you how to configure the VLANs, IRB interfaces, lossless forwarding classes, classifiers, PFC settings to enable PFC across Layer 3 interfaces, and the queue scheduling configuration common to ETS and direct port scheduling. For completeness, the ETS hierarchical port scheduling and direct port scheduling configurations are included separately, in the following procedures, but without explanatory text. See the Related Documentation links for detailed examples of the scheduling elements of the configuration.

  1. Configure the Layer 3 interface VLANs and IP addresses:

  2. Configure the Layer 2 interface VLAN membership and interface mode:

  3. Configure the IRB interfaces and VLANs to transport incoming Layer 2 traffic assigned to VLANs vlan105 (of which interface xe-0/0/20 is a member) and vlan106 (of which interface xe-0/0/21 is a member) across Layer 3:

  4. Configure the lossless forwarding classes and a best-effort forwarding class for any other traffic that might use the interfaces:

  5. Configure the IEEE classifier for the Layer 2 and Layer 3 interfaces to classify incoming traffic into the lossless forwarding classes based on the IEEE 802.1p code point of the traffic:

  6. Configure the CNP to enable PFC on the lossless priorities (the lossless forwarding classes mapped to IEEE 802.1p code points 3 and 4):

  7. Apply the Layer 2 IEEE 802.1p classifier and the CNP to the Layer 3 interfaces:

  8. Apply the Layer 2 IEEE 802.1p classifier and the CNP to the Layer 2 interfaces:

  9. Configure queue scheduling to support the lossless configuration and map the schedulers to the forwarding classes (statements included here for completeness; see the Related Documentation links for detailed examples of scheduling configuration):

ETS Hierarchical Scheduling Configuration

Step-by-Step Procedure

  1. Configure hierarchical scheduling to support the lossless configuration (included here for completeness; see the Related Documentation links for detailed examples of scheduling configuration) and apply it to the Layer 2 and Layer 3 interfaces:

Port Scheduling Configuration

Step-by-Step Procedure

  1. Apply port scheduling to support the lossless configuration on interfaces:

Results

Display the results of the interface, VLAN, and class-of-service configurations (the system shows only the explicitly configured parameters; it does not show default parameters). The results are valid for both Switch SW1 and Switch SW2 because the same configuration is used on both switches. The results are from the ETS hierarchical scheduling configuration, which show the more complex configuration. Direct port scheduling results would not show the traffic control profile or forwarding class set portions of the configuration, but would display the name of the scheduler map under each interface (instead of the names of the forwarding class set and output traffic control profile). Other than that, the results are the same.

Display the results of the interface configuration:

Display the results of the vlan configuration:

Display the results of the class-of-service configuration:

Tip:

To quickly configure the switch, issue the load merge terminal command, and then copy the hierarchies and paste them into the switch terminal window.

Verification

To verify that the PFC across Layer 3 interfaces configuration has been created and is operating properly, perform these tasks:

Verifying the Interface Configuration

Purpose

Verify that the Layer 2 Ethernet interfaces, Layer 3 IP interfaces, IRB interfaces, and VLAN interfaces have been created on the switch and are correctly configured.

Action

Display the switch interface configuration using the show configuration interfaces command:

Meaning

The show configuration interfaces command displays all of the interfaces configured on the switch. The command output shows that:

  • Interfaces xe-0/0/20 and xe-0/0/21 are Ethernet interfaces (family ethernet-switching) in trunk interface mode. Interface xe-0/0/20 is a member of VLAN vlan105, and interface xe-0/0/21 is a member of VLAN vlan106.

  • Interfaces xe-0/0/40 and xe-0/0/41 are IP interfaces (family inet) with VLAN tagging enabled. Interface xe-0/0/40 has an IP address of 100.103.1.2/24 and a VLAN ID of 103. Interface xe-0/0/41 has an IP address of 100.104.1.2/24 and a VLAN ID of 104.

  • Two IRB interfaces are configured, IRB unit 105 with an IP address of 100.105.1.1/24 and IRB unit 106 with an IP address of 100.106.1.1/24.

  • Two VLAN interfaces are configured, VLAN unit 105 with an IP address of 100.105.1.1/24 (for IRB interface unit 105) and VLAN unit 106 with an IP address of 100.106.1.1/24 (for IRB interface unit 106).

Verifying the VLAN Configuration

Purpose

Verify that VLANs have been created on the switch and are correctly configured.

Action

Display the VLAN configuration using the show configuration vlans command:

Meaning

The show configuration vlans command displays all of the VLANs configured on the switch. The command output shows that:

  • VLAN vlan105 has been configured with VLAN ID 105 on IRB interface irb.105.

  • VLAN vlan106 has been configured with VLAN ID 106 on IRB interface irb.106.

Verifying the PFC Configuration (Congestion Notification Profile)

Purpose

Verify that PFC has been enabled on the correct IEEE 802.1p code points (priorities) in the CNP.

Action

Display the PFC configuration using the show configuration class-of-service congestion-notification-profile command:

Meaning

The show configuration class-of-service congestion-notification-profile command displays all of the CNPs configured on the switch. The command output shows that:

  • The CNP named lossless-cnp is configured on the switch.

  • The CNP lossless-cnp enables PFC on IEEE 802.1p code points 100 and 100.

Verify the Forwarding Class Configuration

Purpose

Verify that the two lossless forwarding classes and the best-effort forwarding class have been configured on the switch.

Action

Display the forwarding class configuration using the show configuration class-of-service forwarding-classes command:

Meaning

The show configuration class-of-service forwarding-classes command displays all of the forwarding classes configured on the switch (default forwarding classes are not displayed). The command output shows that:

  • Forwarding class lossless-3 is mapped to queue 3 and is configured as a lossless forwarding class (the no-loss attribute is applied)

  • Forwarding class lossless-4 is mapped to queue 4 and is configured as a lossless forwarding class (the no-loss attribute is applied)

  • Forwarding class all-others is mapped to queue 0. It is not a lossless forwarding class (the no-loss attribute is not applied).

Verifying the Classifier Configuration

Purpose

Verify that the IEEE 802.1p classifier has been configured on the switch.

Action

Display the classifier configuration using the show configuration class-of-service classifiers command:

Meaning

The show configuration class-of-service classifiers command displays all of the classifiers configured on the switch. The command output shows that the Layer 2 IEEE 802.1p classifier lossless-3-4-ieee classifiers traffic with the code point 011 into the lossless-3 forwarding class with a loss priority of low, and classifies traffic with the code point 100 into the lossless-4 forwarding class with a loss priority of low.

Verifying the Interface CoS Configuration (Hierarchical Scheduling, PFC, and Classifier Mapping to Interfaces)

Purpose

Verify that the interfaces have the correct hierarchical scheduling, PFC, and classifier configurations.

Note:

The results are from the ETS hierarchical scheduling configuration, which shows the more complex configuration. Direct port scheduling results would not show the traffic control profile or forwarding class set portions of the interface configuration, but would display the name of the scheduler map under each interface instead of the names of the forwarding class set and output traffic control profile. Other than that, they are the same.

Action

Display the interface CoS configuration using the show configuration class-of-service interfaces command:

Meaning

The show configuration class-of-service interfaces command displays all of the CoS components configured on the switch interfaces. The command output shows that:

  • The configuration on Layer 2 Ethernet interfaces xe-0/0/20 and xe-0/0/21 includes:

    • Hierarchical scheduling—The forwarding class set lossless_fc_set with the traffic control profile lossless_tcp for the lossless traffic, and the forwarding class set all-others_fc_set with the traffic control profile all-others_tcp for the best-effort traffic are applied to both interfaces.

    • PFC—The lossless-cnp congestion notification profile is applied to both interfaces.

    • Classifiers—The Layer 2 IEEE 802.1p classifier lossless-3-4-ieee is applied to both interfaces.

  • The configuration on Layer 3 IP interfaces xe-0/0/40 and xe-0/0/41 includes:

    • Hierarchical scheduling—The forwarding class set lossless_fc_set with the traffic control profile lossless_tcp for the lossless traffic, and the forwarding class set all-others_fc_set with the traffic control profile all-others_tcp for the best-effort traffic are applied to both interfaces.

    • PFC—The lossless-cnp congestion notification profile is applied to both interfaces.

    • Classifiers—The Layer 2 IEEE 802.1p classifier lossless-3-4-ieee is applied to both interfaces. Traffic that would use a DSCP or a DSCP IPv6 classifier if it were configured uses the IEEE 802.1p classifier instead. Using the IEEE 802.1p classifier allows the interface to use PFC to pause traffic during periods of congestion to prevent packet loss.