ON THIS PAGE
Example: Configuring Two or More Lossless FCoE IEEE 802.1p Priorities on Different FCoE Transit Switch Interfaces
Although the default configuration provides two lossless forwarding classes mapped to two different IEEE 802.1p priorities (code points), you can explicitly configure up to six lossless forwarding classes and map them to different priorities. You can support up to six different types of lossless traffic, and you can support the same type of traffic if it uses different priorities in different parts of your converged network.
This example shows you how to configure two lossless forwarding classes for FCoE traffic and map them to two different priorities on an FCoE transit switch.
Requirements
This example uses the following hardware and software components:
One switch used as an FCoE transit switch
Junos OS Release 12.3 or later for the QFX Series
Overview
Some network topologies support FCoE traffic on more than one IEEE 802.1p priority. For example, when the switch acts as a transit switch, it could be connected to two QFX3500 switches in FCoE-FC gateway mode. Each of the gateway switches could connect a set of FCoE clients to a different SAN, and each set of FCoE clients could use a different priority for FCoE traffic to avoid fate sharing and maintain separation of the two FCoE networks. In this case, you need to configure two forwarding classes for FCoE traffic, each mapped to a different output queue and a different priority.
Supporting lossless behavior for two FCoE traffic classes requires configuring:
At least one lossless forwarding class for FCoE traffic (this example uses the default
fcoeforwarding class as one of the two lossless FCoE forwarding classes, so we need to explicitly configure only one FCoE forwarding class)Behavior aggregate (BA) classifiers to map the FCoE forwarding classes to the appropriate IEEE 802.1p code points (priorities) on each interface
Congestion notification profiles (CNPs) for each interface to enable PFC on the FCoE code points at the interface ingress and to configure PFC flow control on the interface egress so that the interface can respond to PFC messages received from the connected peer
Note:Configuring or changing PFC on an interface blocks the entire port until the PFC change is completed. After a PFC change is completed, the port is unblocked and traffic resumes. Blocking the port stops ingress and egress traffic, and causes packet loss on all queues on the port until the port is unblocked.
DCBX applications and an application map to support DCBX application TLV exchange for the lossless FCoE traffic on the configured FCoE priorities. By default, DCBX is enabled on all Ethernet interfaces, but only on priority 3 (IEEE 802.1p code point 011). To support DCBX application TLV exchange when you are not using the default configuration, you must configure all of the applications and map them to interfaces and priorities.
The priorities specified in the BA classifiers, CNPs, and DCBX application map must match, or the configuration does not work. You must specify the same lossless FCoE forwarding class in each configuration and use the same IEEE 802.1p code point (priority) so that the FCoE traffic is properly classified into flows and so that those flows receive lossless treatment.
Topology
This example shows how to configure two lossless FCoE traffic classes, map them to two different priorities, and configure flow control to ensure lossless behavior for those priorities on the interfaces. This example uses three Ethernet interfaces, xe-0/0/20, xe-0/0/21, and xe-0/0/22:
Interface xe-0/0/20 connects to an FCoE-FC gateway that connects to Fibre Channel (FC) SAN 1. FCoE traffic to and from FC SAN 1 uses the default
fcoeforwarding class and the default mapping to priority 3 (IEEE 802.1p code point 011) and output queue 3.Interface xe-0/0/21 connects to another FCoE-FC gateway that connects to Fibre Channel (FC) SAN 2. FCoE traffic to and from FC SAN-2 uses an explicitly configured FCoE forwarding class that is mapped to priority 5 (code point 101) and output queue 5.
Interface xe-0/0/22 connects to FCoE devices on the converged Ethernet network and handles traffic destined for FC SAN 1 and FC SAN 2. Interface xe-0/0/22 must properly handle lossless FCoE traffic of both priorities (both FCoE forwarding classes), including pausing the traffic on ingress or egress as required.
Figure 1 shows the topology for this example, and Table 1 shows the configuration components for this example.
Component |
Settings |
|---|---|
Hardware |
One switch |
Forwarding classes |
Name— Note:
A lossless forwarding class can be mapped to any output
queue. However, because the Name— |
BA classifiers |
Each interface requires a different classifier because each interface handles a different subset of FCoE traffic.
|
PFC configuration (CNPs) |
Each interface requires a different CNP because each interface handles a different subset of FCoE traffic and must pause that traffic on different priorities.
Note:
When you apply a CNP with an explicit output queue flow control configuration to an interface, the explicit CNP overwrites the default output CNP. The output queues that are enabled for pause in the default configuration (queues 3 and 4) are not enabled for pause unless they are included in the explicitly configured output CNP. |
DCBX application mapping |
Interface xe-0/0/20 does not need an application map because DCBX exchanges application protocol TLVs only on the default FCoE priority (priority 3). Interface xe-0/0/21 requires an application map that enables DCBX application protocol TLV exchange on priority 5 (code point 101) for FCoE traffic. Interface xe-0/0/22 requires an application map that enables DCBX application protocol TLV exchange both on priority 3 (code point 011) and on priority 5 (code point 101) for FCoE traffic.
Note:
LLDP and DCBX must be enabled on the interface. By default, LLDP and DCBX are enabled on all Ethernet interfaces. |
This example does not include scheduling (bandwidth allocation) configuration or the FIP snooping configuration. This examples focuses only on the lossless FCoE priority configuration.
QFX10000 switches do not support FIP snooping. For this reason, QFX10000 switches cannot be used as FCoE access transit switches. QFX10000 switches can be used as intermediate or aggregation transit switches in the FCoE path, between an FCoE access transit switch that performs FIP snooping and an FCF.
Configuration
CLI Quick Configuration
To quickly configure two lossless FCoE forwarding
classes that use different priorities on an FCoE transit switch, copy
the following commands, paste them in a text file, remove line breaks,
change variables and details to match your network configuration,
and then copy and paste the commands into the CLI at the [edit] hierarchy level.
set class-of-service forwarding-classes class fcoe1 queue-num 5 no-loss set class-of-service classifiers ieee-802.1 fcoe_p3 forwarding-class fcoe loss-priority low code-points 011 set class-of-service classifiers ieee-802.1 fcoe_p5 forwarding-class fcoe1 loss-priority low code-points 101 set class-of-service classifiers ieee-802.1 fcoe_p3_p5 forwarding-class fcoe loss-priority low code-points 011 set class-of-service classifiers ieee-802.1 fcoe_p3_p5 forwarding-class fcoe1 loss-priority low code-points 101 set class-of-service interfaces xe-0/0/20 unit 0 classifiers ieee-802.1 fcoe_p3 set class-of-service interfaces xe-0/0/21 unit 0 classifiers ieee-802.1 fcoe_p5 set class-of-service interfaces xe-0/0/22 unit 0 classifiers ieee-802.1 fcoe_p3_p5 set class-of-service congestion-notification-profile fcoe_p3_cnp input ieee-802.1 code-point 011 pfc mru 2240 set class-of-service congestion-notification-profile fcoe_p3_cnp input cable-length 100 set class-of-service congestion-notification-profile fcoe_p5_cnp input ieee-802.1 code-point 101 pfc mru 2240 set class-of-service congestion-notification-profile fcoe_p5_cnp input cable-length 150 set class-of-service congestion-notification-profile fcoe_p5_cnp output ieee-802.1 code-point 101 pfc flow-control-queue 5 set class-of-service congestion-notification-profile fcoe_p3_p5_cnp input ieee-802.1 code-point 011 pfc mru 2240 set class-of-service congestion-notification-profile fcoe_p3_p5_cnp input ieee-802.1 code-point 101 pfc mru 2240 set class-of-service congestion-notification-profile fcoe_p3_p5_cnp input cable-length 100 set class-of-service congestion-notification-profile fcoe_p3_p5_cnp output ieee-802.1 code-point 011 pfc flow-control-queue 3 set class-of-service congestion-notification-profile fcoe_p3_p5_cnp output ieee-802.1 code-point 101 pfc flow-control-queue 5 set class-of-service interfaces xe-0/0/20 congestion-notification-profile fcoe_p3_cnp set class-of-service interfaces xe-0/0/21 congestion-notification-profile fcoe_p5_cnp set class-of-service interfaces xe-0/0/22 congestion-notification-profile fcoe_p3_p5_cnp set applications application fcoe_p5_app ether-type 0x8906 set applications application fcoe_all_app ether-type 0x8906 set policy-options application-maps fcoe_p5_app_map application fcoe_p5_app code-points 101 set policy-options application-maps fcoe_all_app_map application fcoe_all_app code-points [011 101] set protocols dcbx interface xe-0/0/21 application-map fcoe_p5_app_map set protocols dcbx interface xe-0/0/22 application-map fcoe_all_app_map
Procedure
Step-by-Step Procedure
To configure two lossless forwarding classes for FCoE traffic on different interfaces, classify FCoE traffic into the forwarding classes, configure congestion notification profiles to enable PFC on the FCoE priorities and output queues, and configure DCBX application protocol TLV exchange for traffic on both FCoE priorities:
Configure lossless forwarding class
fcoe1and map it to output queue5for FCoE traffic that uses IEEE 802.1p priority 5:[edit class-of-service] user@switch# set forwarding-classes class fcoe1 queue-num 5 no-loss
Note:This examples uses the default
fcoeforwarding class as the other lossless FCoE forwarding class.Configure the ingress classifier (
fcoe_p3) for interfacexe-0/0/20. The classifier maps the FCoE priority (IEEE 802.1p code point011) to lossless FCoE forwarding classfcoe:[edit class-of-service classifiers] user@switch# set ieee-802.1 fcoe_p3 forwarding-class fcoe loss-priority low code-points 011
Configure the ingress classifier (
fcoe_p5) for interfacexe-0/0/21. The classifier maps the FCoE priority (IEEE 802.1p code point101) to lossless FCoE forwarding classfcoe1:[edit class-of-service classifiers] user@switch# set ieee-802.1 fcoe_p5 forwarding-class fcoe1 loss-priority low code-points 101
Configure the ingress classifier (
fcoe_p3_p5) for interfacexe-0/0/22. The classifier maps the two FCoE priorities (IEEE 802.1p code points011and101) to the two lossless FCoE forwarding classesfcoeandfcoe1, respectively:[edit class-of-service classifiers] user@switch# set ieee-802.1 fcoe_p3_p5 forwarding-class fcoe loss-priority low code-points 011 user@switch# set ieee-802.1 fcoe_p3_p5 forwarding-class fcoe1 loss-priority low code-points 101
Apply each classifier to the appropriate interface:
[edit class-of-service] user@switch# set interfaces xe-0/0/20 unit 0 classifiers ieee-802.1 fcoe_p3 user@switch# set interfaces xe-0/0/21 unit 0 classifiers ieee-802.1 fcoe_p5 user@switch# set interfaces xe-0/0/22 unit 0 classifiers ieee-802.1 fcoe_p3_p5
Configure the CNP input stanza for interface xe-0/0/20 to enable PFC on the FCoE priority (IEEE 802.1p code point 011), set the MRU value (2240 bytes), and set the cable length value (100 meters). No output stanza is needed because queue 3 is paused by default on priority 3, and we are not explicitly configuring output queue flow control for any other queues.
[edit class-of-service] user@switch# set congestion-notification-profile fcoe_p3_cnp input ieee-802.1 code-point 011 pfc mru 2240 user@switch# set congestion-notification-profile fcoe_p3_cnp input cable-length 100
Configure the CNP for interface xe-0/0/21. The input stanza enables PFC on the FCoE priority (IEEE 802.1p code point 101), sets the MRU value (2240 bytes), and sets the cable length value (150 meters). The output stanza configures flow control on output queue 5 on the FCoE priority:
[edit class-of-service] user@switch# set congestion-notification-profile fcoe_p5_cnp input ieee-802.1 code-point 101 pfc mru 2240 user@switch# set congestion-notification-profile fcoe_p5_cnp input cable-length 150 user@switch# set congestion-notification-profile fcoe_p5_cnp output ieee-802.1 code-point 101 pfc flow-control-queue 5
Configure the CNP for interface xe-0/0/22. The input stanza enables PFC on the FCoE priorities (IEEE 802.1p code points 011 and 101), sets the MRU value (2240 bytes), and sets the cable length value (100 meters). The output stanza configures flow control on output queues 3 and 5 on the FCoE priorities:
[edit class-of-service] user@switch# set congestion-notification-profile fcoe_p3_p5_cnp input ieee-802.1 code-point 011 pfc mru 2240 user@switch# set congestion-notification-profile fcoe_p3_p5_cnp input ieee-802.1 code-point 101 pfc mru 2240 user@switch# set congestion-notification-profile fcoe_p3_p5_cnp input cable-length 100 user@switch# set congestion-notification-profile fcoe_p3_p5_cnp output ieee-802.1 code-point 011 pfc flow-control-queue 3 user@switch# set congestion-notification-profile fcoe_p3_p5_cnp output ieee-802.1 code-point 101 pfc flow-control-queue 5
Apply each CNP to the appropriate interface:
[edit class-of-service] user@switch# set interfaces xe-0/0/20 congestion-notification-profile fcoe_p3_cnp user@switch# set interfaces xe-0/0/21 congestion-notification-profile fcoe_p5_cnp user@switch# set interfaces xe-0/0/22 congestion-notification-profile fcoe_p3_p5_cnp
Configure the DCBX FCoE application and application map to apply to interface xe-0/0/21. Interface xe-0/0/21 uses priority 5 (IEEE 802.1p code point 101) for FCoE traffic, which requires DCBX to exchange FCoE application protocol TLVs on priority 5 on interface xe-0/0/21. Configure an application named
fcoe_p5_appfor FCoE traffic (EtherType0x8906) and configure an application map namedfcoe_p5_app_mapto map the application to code point 101:[edit] user@switch# set applications application fcoe_p5_app ether-type 0x8906 user@switch# set policy-options application-maps fcoe_p5_app_map application fcoe_p5_app code-points 101
Note:Interface xe-0/0/20 uses the default FCoE configuration (priority 3). DCBX exchanges protocol TLVs for the FCoE application by default, so you do not need to configure DCBX explicitly on interface xe-0/0/20.
Configure the DCBX FCoE application and application map to apply to interface xe-0/0/22. Interface xe-0/0/22 uses both priority 3 (IEEE 802.1p code point 011) and priority 5 for FCoE traffic, which requires DCBX to exchange FCoE application protocol TLVs on both priority 3 and priority 5. Configure an application named
fcoe_all_appfor FCoE traffic (EtherType0x8906) and configure an application map namedfcoe_all_app_mapto map the application to code points 011 and 101:[edit] user@switch# set applications application fcoe_all_app ether-type 0x8906 user@switch# set policy-options application-maps fcoe_all_app_map application fcoe_all_app code-points [011 101]
Apply the application maps to the interfaces xe-0/0/21 and xe-0/0/22 so that DCBX exchanges FCoE application TLVs on the correct code points on each interface:
[edit] user@switch# set protocols dcbx interface xe-0/0/21 application-map fcoe_p5_app_map user@switch# set protocols dcbx interface xe-0/0/22 application-map fcoe_all_app_map
Verification
To verify the configuration and proper operation of the lossless forwarding classes and IEEE 802.1p priorities, perform these tasks:
- Verifying the Forwarding Class Configuration
- Verifying the Behavior Aggregate Classifier Configuration
- Verifying the PFC Flow Control Configuration (CNP)
- Verifying the Interface Configuration
- Verifying the DCBX Application Configuration
- Verifying the DCBX Application Map Configuration
- Verifying the DCBX Application Protocol Exchange Interface Configuration
Verifying the Forwarding Class Configuration
Purpose
Verify that the lossless forwarding class fcoe1 has been created.
Action
Show the forwarding class configuration by using the
operational command show class-of-service forwarding class:
user@switch# show class-of-service forwarding-class Forwarding class ID Queue Policing priority No-Loss best-effort 0 0 normal Disabled fcoe 1 3 normal Enabled no-loss 2 4 normal Enabled network-control 3 7 normal Disabled fcoe1 4 5 normal Enabled mcast 8 8 normal Disabled
Meaning
The show class-of-service forwarding-class command shows all of the forwarding classes. The command output
shows that the fcoe1 forwarding class is configured on
output queue 5 with the no-loss packet drop attribute enabled.
Because we did not explicitly configure the default forwarding
classes, they remain in their default state, including the lossless
configuration of the fcoe and no-loss default
forwarding classes.
Verifying the Behavior Aggregate Classifier Configuration
Purpose
Verify that the three classifiers map the forwarding classes to the correct IEEE 802.1p code points (priorities) and packet loss priorities.
Action
List the classifiers configured to support lossless
FCoE transport using the operational mode command show class-of-service
classifier:
user@switch> show class-of-service classifier Classifier: fcoe_p3, Code point type: ieee-802.1, Index: 13913 Code point Forwarding class Loss priority 011 fcoe low Classifier: fcoe_p5, Code point type: ieee-802.1, Index: 63065 Code point Forwarding class Loss priority 101 fcoe1 low Classifier: fcoe_p3_p5, Code point type: ieee-802.1, Index: 10964 Code point Forwarding class Loss priority 011 fcoe low 101 fcoe1 low
Meaning
The show class-of-service classifier command
shows the IEEE 802.1p code points and the loss priorities that are
mapped to the forwarding classes in each classifier. The command output
shows that there are three classifiers, fcoe_p3, fcoe_p5, and fcoe_p3_p5.
Classifier fcoe_p3 maps code point 011 (priority 3) to default lossless forwarding class fcoe and a packet loss priority of low, and all other priorities
to the best-effort forwarding class with a packet loss
priority of high.
Classifier fcoe_p5 maps code point 101 (priority 5) to explicitly configured lossless forwarding class fcoe1 and a packet loss priority of low, and all
other priorities to the best-effort forwarding class with
a packet loss priority of high.
Classifier fcoe_p3_p5 maps code point 011 to default lossless forwarding class fcoe and a packet
loss priority of low, and maps code point 101 to explicitly configured lossless forwarding class fcoe1 and a packet loss priority of low. The classifier maps
all other priorities to the best-effort forwarding class
with a packet loss priority of high.
Verifying the PFC Flow Control Configuration (CNP)
Purpose
Verify that PFC is enabled on the correct input priorities and that flow control is configured on the correct output queues and priorities in each CNP.
Action
List the congestion notification profiles using the
operational mode command show class-of-service congestion-notification:
user@switch> show class-of-service congestion-notification
Name: fcoe_p3_cnp, Index: 12037
Type: Input
Cable Length: 100 m
Priority PFC MRU
000 Disabled
001 Disabled
010 Disabled
011 Enabled 2240
100 Disabled
101 Disabled
110 Disabled
111 Disabled
Type: Output
Priority Flow-Control-Queues
000
0
001
1
010
2
011
3
100
4
101
5
110
6
111
7
Name: fcoe_p3_p5_cnp, Index: 46484
Type: Input
Cable Length: 100 m
Priority PFC MRU
000 Disabled
001 Disabled
010 Disabled
011 Enabled 2240
100 Disabled
101 Enabled 2240
110 Disabled
111 Disabled
Type: Output
Priority Flow-Control-Queues
011
3
101
5
Name: fcoe_p5_cnp, Index: 12133
Type: Input
Cable Length: 150 m
Priority PFC MRU
000 Disabled
001 Disabled
010 Disabled
011 Disabled
100 Disabled
101 Enabled 2240
110 Disabled
111 Disabled
Type: Output
Priority Flow-Control-Queues
101
5
Meaning
The show class-of-service congestion-notification command shows the input and output stanzas of the three CNPs. For
CNP fcoe_p3_cnp, the input stanza shows that PFC is enabled
on IEEE 802.1p code point 011 (priority 3), the MRU
is 2240 bytes, and the cable length is 100 meters.
The CNP output stanza shows the default mapping of priorities to output
queues.
By default, only queues 3 and 4 are enabled to respond
to pause messages from the connected peer. For queue 3 to respond
to pause messages, priority 3 (code point 011) must be enabled
for PFC in the input stanza. For queue 4 to respond to pause
messages, priority 4 (code point 100) must be enabled for
PFC in the input stanza. In this example, only queue 3 responds
to pause messages from the connected peer on interfaces that use CNP fcoe_p3_cnp, because the input stanza enables PFC priority 3
only.
For CNP fcoe_p3_p5_cnp, the input stanza shows that
PFC is enabled on code points 011 and 101, the
MRU is 2240 bytes on both priorities, and the cable length
is 100 meters. The CNP output stanza shows that output
flow control is configured on queues 3 and 5 for code points 011 and 101, respectively.
For CNP fcoe_p5_cnp, the input stanza shows that
PFC is enabled on code point 101 (priority 5), the
MRU is 2240 bytes, and the cable length is 150 meters. The CNP output stanza shows that output flow control is
configured on queue 5 for code point 101 (priority 5).
Verifying the Interface Configuration
Purpose
Verify that the correct classifiers and congestion notification profiles are configured on the correct interfaces.
Action
List the ingress interfaces using the operational mode
commands show configuration class-of-service interfaces xe-0/0/20, show configuration class-of-service interfaces xe-0/0/21, and show configuration class-of-service interfaces xe-0/0/22:
user@switch> show configuration class-of-service interfaces xe-0/0/20
ccongestion-notification-profile fcoe_p3_cnp;
unit 0 {
classifiers {
ieee-802.1 fcoe_p3;
}
}
user@switch> show configuration class-of-service interfaces xe-0/0/21
congestion-notification-profile fcoe_p5_cnp;
unit 0 {
classifiers {
ieee-802.1 fcoe_p5;
}
}
user@switch> show configuration class-of-service interfaces xe-0/0/22
congestion-notification-profile fcoe_p3_p5_cnp;
unit 0 {
classifiers {
ieee-802.1 fcoe_p3_p5;
}
}
Meaning
The show configuration class-of-service interfaces
xe-0/0/20 command shows that the congestion notification profile fcoe_p3_cnp is configured on the interface, and that the IEEE
802.1p classifier associated with the interface is fcoe_p3.
The show configuration class-of-service interfaces xe-0/0/21 command shows that the congestion notification profile fcoe_p5_cnp is configured on the interface, and that the IEEE 802.1p classifier
associated with the interface is fcoe_p5.
The show configuration class-of-service interfaces xe-0/0/22 command shows that the congestion notification profile fcoe_p3_p5_cnp is configured on the interface, and that the IEEE 802.1p classifier
associated with the interface is fcoe_p3_p5.
Verifying the DCBX Application Configuration
Purpose
Verify that the two DCBX applications for FCoE are configured.
Action
List the DCBX applications by using the configuration
mode command show applications:
user@switch# show applications
application fcoe_all_app {
ether-type 0x8906;
application fcoe_p5_app {
ether-type 0x8906;
Meaning
The show applications configuration mode
command shows all of the configured applications. The output shows
that the application fcoe_all_app is configured with an
EtherType of 0x8906 (the correct EtherType for FCoE traffic)
and that the application fcoe_p5_app is also configured
with an EtherType of 0x8906.
Verifying the DCBX Application Map Configuration
Purpose
Verify that the application maps are configured.
Action
List the application maps by using the configuration
mode command show policy-options application-maps:
user@switch# show policy-options application-maps
fcoe_all_app_map {
application fcoe_all_app code-points [011 101];
}
fcoe_p5_app_map {
application fcoe_p5_app code-points 101;
}
Meaning
The show policy-options application-maps configuration mode command lists all of the configured application
maps and the applications that belong to each application map. The
output shows that there are two application maps.
Application map fcoe_all_app_map consists of the
application named fcoe_all_app mapped to IEEE 802.1p code
points 011 (priority 3) and 101 (priority 5).
Application map fcoe_p5_app_map consists of the application
named fcoe_p5_app mapped to IEEE 802.1p code point 101 (priority 5).
Verifying the DCBX Application Protocol Exchange Interface Configuration
Purpose
Verify that the application maps are applied to the correct interfaces.
Action
List the application maps on each interface using the
configuration mode command show protocols dcbx:
user@switch# show protocols dcbx
interface xe-0/0/21.0 {
application-map fcoe_p5_app_map;
}
interface xe-0/0/22.0 {
application-map fcoe_all_app_map;
}Meaning
The show protocols dcbx configuration mode
command lists the application map association with interfaces. The
output shows that interface xe-0/0/21.0 uses application
map fcoe_p5_app_map and interface xe-0/0/22.0 uses application map fcoe_all_app_map.
Because interface xe-0/0/20 uses the default lossless FCoE configuration, you do not configure application mapping to interface xe-0/0/20. The default configuration automatically exchanges application protocol TLVs for the default FCoE configuration on priority 3 (IEEE 802.1p code point 011).