Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Example: Configuring an FCoE Transit Switch

 

You can use an EX4500 CEE-enabled switch as a Fibre Channel over Ethernet (FCoE) transit switch, enabling it to transport both FCoE and Ethernet LAN traffic. Using the same switch to support both your storage network and traditional IP-based data communications reduces the costs of powering, cooling, provisioning, maintaining, and managing your network.

This example includes:

  • FIP snooping for security

  • Priority-based flow control (PFC) for lossless transport

  • The FCoE forwarding class for the DCBX application protocol type, length, value (TLV) exchange

  • A trusted port connecting to the FCoE forwarder (FCF)

  • Enlarged maximum transmission unit (MTU) size for handling FCoE traffic

This example shows how to configure an FCoE transit switch:

Requirements

This example uses the following hardware and software components:

  • One EX4500 switch (CEE-capable model)

  • Junos OS Release 12.1 or later for EX Series switches

  • One FCoE Node (ENode)

  • One FCoE forwarder (FCF)

Before you begin, be sure you have:

Overview and Topology

FCoE transmissions are vulnerable to address spoofing and man-in-the-middle attacks, because they are not actually sent through point-to-point links. This example describes how to configure the switch so that it provides security similar to that provided by traditional Fibre Channel (FC) networks. The switch is transparent to the ENode and the FCF, so the ENode and FCF communicate just as they would for a point-to-point link.

FIP snooping is disabled by default. You enable FIP snooping on a per-VLAN basis for VLANs that carry FCoE traffic. Ensure that a VLAN that carries FCoE traffic carries only FCoE traffic, because enabling FIP snooping denies access for all other Ethernet traffic.

This example shows how to configure FIP snooping on a VLAN of the EX4500 switch that is connected with one ENode, that is, a server equipped with converged network adapters (CNAs). The setup for this example includes the VLAN fcoe-vlan on the switch.

This example also shows how to configure PFC on the interfaces that are being used for FCoE traffic and how to configure an FCoE trusted port to handle traffic between the switch and the FCF gateway to the storage area network (SAN).

You must configure PFC properties for the interfaces that are carrying FCoE traffic, because flow control must be implemented on the link level for this type of traffic.

Note

Data Center Bridging Capability Exchange protocol (DCBX) is enabled by default on all 10-Gigabit Ethernet interfaces on EX4500 switches. DCBX automatically controls whether PFC is enabled or disabled on the interface. However, you must configure the PFC properties selecting the traffic class and queue. See Configuring Priority-Based Flow Control for an EX Series Switch (CLI Procedure).

You configure trunk interfaces that connect to the FCF as trusted interfaces. The switch must use the same FCoE MAC Address Prefix (FC-MAP) value that is being used by the FCF. Therefore, if the FCF is using a nondefault FC-MAP value, you must configure the FC-MAP value on the switch to match that value.

You must also enlarge the MTU size for all interfaces (both access and trunk) that are handling FCoE traffic to accommodate the maximum FC frame and Ethernet header sizes.

This example also includes configuring the fcoe forwarding class to be used for the FCoE traffic, so that it can take advantage of DCBX support for the Application Protocol TLV Exchange. See Understanding Data Center Bridging Capability Exchange Protocol for EX Series Switches for additional information.

Note

Configuring and applying PFC and a forwarding class fcoe on the DCBX interfaces automatically enables the DCBX FCoE application protocol exchange on those interfaces. Do not explicitly configure an FCoE application map, because doing that generates a commit error. See Understanding Data Center Bridging Capability Exchange Protocol for EX Series Switches for additional information.

Note

PFC is supported only on 10-Gigabit Ethernet interfaces.

Note

We recommend that you also:

  • Configure the PFC congestion notification profile for the same 802.1p code points that you are using for the fcoe forwarding class. We recommend code point 011, because this is the conventional IEEE 802.1p code point for FCoE traffic.

  • Configure at least 20 percent of the buffer for the queue that is using PFC.

  • Do not specify the exact option when configuring the buffer for the queue that is using PFC.

  • Configure the loss-priority statement to low for a traffic class that is using PFC.

  • Configure an appropriate percent of the buffer for any other forwarding classes (default forwarding classes and the user-defined forwarding classes) that you are using

The components of the topology for this example are shown in Table 1.

Table 1: Components of the FCoE Security Topology

PropertiesSettings

Switch hardware

One EX4500 CEE-enabled switch

VLAN name and ID

fcoe-vlan, tag 20

Forwarding class for FCoE traffic

fcoe, code point 011

Interfaces in fcoe-vlan

xe-0/0/1

xe-0/0/2

xe-0/0/3

xe-0/0/30

FCoE trusted port to the FCF

xe-0/0/30

PFC interfaces

xe-0/0/1

xe-0/0/2

xe-0/0/3

xe-0/0/30

CoS forwarding-class interface

xe-0/0/30

CoS scheduler-map interface

xe-0/0/30

Interfaces configured with MTU of 2500

xe-0/0/1

xe-0/0/2

xe-0/0/3

xe-0/0/30

In this example, the switch has already been configured as follows:

  • All access ports are untrusted, which is the default setting.

  • DCBX is enabled by default on all 10-Gigabit Ethernet interfaces.

  • The port connecting the switch to the FCF is configured as a trunk port.

Configuration

To configure an FCoE transit switch, perform these tasks:

CLI Quick Configuration

To quickly configure an FCoE transit switch, copy the following commands and paste them into the switch terminal window:

[edit]

set ethernet-switching-options secure-access-port vlan fcoe-vlan examine-fip fc-map 0x0EFC03

set ethernet-switching-options secure-access-port interface xe-0/0/30 fcoe-trusted

set interfaces xe-0/0/1 ether-options no-flow-control

set interfaces xe-0/0/2 ether-options no-flow-control

set interfaces xe-0/0/3 ether-options no-flow-control

set interfaces xe-0/0/30 ether-options no-flow-control
set class-of-service congestion-notification-profile cn-profile input ieee-802.1 code-point 011 pfc

set class-of-service interfaces xe-0/0/1 congestion-notification-profile cn-profile

set class-of-service interfaces xe-0/0/2 congestion-notification-profile cn-profile

set class-of-service interfaces xe-0/0/3 congestion-notification-profile cn-profile

set class-of-service interfaces xe-0/0/30 congestion-notification-profile cn-profile

set class-of-service classifiers ieee-802.1 pfc-class import default

set class-of-service classifiers ieee-802.1 pfc-class forwarding-class fcoe loss-priority low code-points 011

set class-of-service interfaces xe-0/0/1 unit 0 classifiers ieee-802.1 pfc-class

set class-of-service interfaces xe-0/0/2 unit 0 classifiers ieee-802.1 pfc-class

set class-of-service interfaces xe-0/0/3 unit 0 classifiers ieee-802.1 pfc-class

set class-of-service interfaces xe-0/0/30 unit 0 classifiers ieee-802.1 pfc-class

set class-of-service forwarding-classes class fcoe queue-num 3

set class-of-service schedulers pfc-sched buffer-size percent 25

set class-of-service schedulers default-sched buffer-size percent 17

set class-of-service scheduler-maps pfc-map forwarding-class fcoe scheduler pfc-sched

set class-of-service scheduler-maps pfc-map forwarding-class assured-forwarding scheduler default-sched

set class-of-service scheduler-maps pfc-map forwarding-class best-effort scheduler default-sched

set class-of-service scheduler-maps pfc-map forwarding-class network-control scheduler default-sched

set class-of-service scheduler-maps pfc-map forwarding-class expedited-forwarding scheduler default-sched

set class-of-service interfaces xe-0/0/30 scheduler-map pfc-map

set interfaces xe-0/0/1 mtu 2500

set interfaces xe-0/0/2 mtu 2500

set interfaces xe-0/0/3 mtu 2500

set interfaces xe-0/0/30 mtu 2500

Step-by-Step Procedure

To configure an FCoE transit switch:

  1. Enable FIP snooping on the VLAN and modify the FC-MAP value to match the FC-MAP value being used by the FCF:
    [edit ethernet-switching-options secure-access-port]

    user@switch# set vlan fcoe-vlan examine-fip fc-map 0x0EFC03
  2. Set the FCF-facing interface (xe-0/0/30) as FCoE-trusted:
    [edit ethernet-switching-options secure-access-port]

    user@switch# set interface xe-0/0/30 fcoe-trusted
  3. Configure a congestion notification profile, specifying the name of the profile and applying it to the traffic class that is indicated by the User Priority bits in the 802.1Q tagged frame of an incoming packet:Note

    The ENode and the switch must use the same traffic class for the FCoE traffic. DCBX advertises the traffic class being used by the switch and detects the traffic class being used by the ENode. If there is a mismatch, the switch disables the PFC capability of the switch interface.

    [edit class-of-service]

    user@switch# set congestion-notification-profile cn-profile input ieee-802.1 code-point 011 pfc
    Note

    The configuration of PFC includes two different ieee-802.1 configuration statements:

  4. Disable standard flow control on the interfaces that you want to use for the FCoE VLAN. Note

    PFC and standard flow control cannot be enabled on the same interface, and you must use PFC for FCoE traffic.

    [edit interfaces]

    user@switch# set xe-0/0/1 ether-options no-flow-control

    user@switch# set xe-0/0/2 ether-options no-flow-control

    user@switch# set xe-0/0/3 ether-options no-flow-control

    user@switch# set xe-0/0/30 ether-options no-flow-control
  5. Bind the congestion notification profile to all interfaces of the FCoE VLAN:
    [edit class-of-service]

    user@switch# set interface xe-0/0/1 congestion-notification-profile cn-profile

    user@switch# set interface xe-0/0/2 congestion-notification-profile cn-profile

    user@switch# set interface xe-0/0/3 congestion-notification-profile cn-profile

    user@switch# set interface xe-0/0/30 congestion-notification-profile cn-profile
  6. Create a CoS classifier for the fcoe forwarding class:
    [edit class-of-service]

    user@switch# set forwarding-classes fcoe queue-num 3
  7. Configure this forwarding class (fcoe) to use a low loss priority value and to use the same code point that is used for PFC:Note

    We recommend that you use code point 011, because this is the conventional IEEE 802.1p code point for FCoE traffic.

    [edit class-of-service]

    user@switch# set classifiers ieee-802.1 pfc-class forwarding-class fcoe loss-priority low code-points 011
  8. Bind the pfc-class classifier to all interfaces of the FCoE VLAN:
    [edit class-of-service]

    user@switch# set interfaces xe-0/0/1 unit 0 classifiers ieee-802.1 pfc-class

    user@switch# set interfaces xe-0/0/2 unit 0 classifiers ieee-802.1 pfc-class

    user@switch# set interfaces xe-0/0/3 unit 0 classifiers ieee-802.1 pfc-class

    user@switch# set interfaces xe-0/0/30 unit 0 classifiers ieee-802.1 pfc-class
  9. Assign forwarding-class fcoe to an egress queue:
    [edit class-of-service]

    user@switch# set forwarding-classes fcoe queue-num 3
  10. Set a scheduler for this queue, allocating at least 20 percent of the buffer to pfc-sched:
    [edit class-of-service]

    user@switch# set schedulers pfc-sched buffer-size percent 25

  11. Set a scheduler for the default queue, allocating 17 percent of the buffer to that queue:
    [edit class-of-service]

    uuser@switch# set schedulers default-sched buffer-size percent 17
  12. Configure a scheduler map (pfc-map) that associates the scheduler (pfc-sched) with the fcoe forwarding class and associates the default forwarding classes (assured-forwarding, best-effort and network-control) with the default schedule:
    [edit class-of-service]

    user@switch# set scheduler-maps pfc-map forwarding-class fcoe scheduler pfc-sched

    user@switch# set scheduler-maps pfc-map forwarding-class assured-forwarding schedulerdefault-sched

    user@switch# set scheduler-maps pfc-map forwarding-class best-effort scheduler default-sched

    user@switch# set scheduler-maps pfc-map forwarding-class network-control scheduler default-sched

    user@switch# set scheduler-maps pfc-map forwarding-class expedited-forwarding scheduler default-sched
  13. Assign the scheduler map (pfc-map) to the FCF-facing interface (xe-0/0/30):
    [edit class-of-service]

    user@switch# set interfaces xe-0/0/30 scheduler-map pfc-map
  14. Enlarge the MTU size to 2500 bytes for all the interfaces (both access and trunk) that are handling FCoE traffic:
    [edit interfaces]

    user@switch# set xe-0/0/1 mtu 2500

    user@switch# set xe-0/0/2 mtu 2500

    user@switch# set xe-0/0/3 mtu 2500

    user@switch# set xe-0/0/30 mtu 2500

Results

Display the results of the configuration:

[edit]

user@switch#show

Verification

Confirm that the configuration of the FCoE transit switch is working properly:

Verifying That FIP Snooping Is Working Correctly on the Switch

Purpose

Verify that FIP snooping is being implemented on the appropriate VLAN.

Action

Send some requests from ENodes to the switch.

Display the FIP snooping information :

user@switch> show fip snooping vlan detail fcoe-vlan

Meaning

The output for this VLAN (fcoe-vlan) includes the FC MAP value that you configured. It shows the MAC addresses of the FCF and the ENode that are transmitting FCoE traffic through the switch.

Verifying That PFC is Enabled, That the FCoE Application Is Advertised, and That the Switch Interface and DCB Peer Are Using the Same 802.1p Code Points

Purpose

Verify that PFC is enabled on the local switch interface and on the peer interface, and that the local interface and the peer interface are using the same code point.

Action

Send some requests from ENodes to the switch.

Display the DCBX information advertised by the configured CoS forwarding class interface (xe-0/0/30) and detected by the switch:

user@switch> show dcbx neighbors interface xe-0/0/30

Meaning

PFC is a requirement for transmitting FCoE traffic and PFC works only when the local and peer devices are both enabled for PFC and are both using the same traffic class (code point) for transmitting the PFC traffic.

In the output for Feature: PFC, check the status of Local-Advertisement to verify that PFC is enabled. If DCBX detects a misconfiguration with the DCB peer, it disables the PFC capability. In this example, the PFC Operational State is enabled, because PFC is configured symmetrically on the switch and the DCB peer. Both devices are using code point 011 for forwarding the traffic.

If the results show that PFC is disabled, you van use the information provided by this command to reconfigure the congestion notification profile to match the code point being used for PFC by the peer device. See Configuring Priority-Based Flow Control for an EX Series Switch (CLI Procedure).

Appl-Name shows the default FCoE application. The FCoE application always indicates Ethernet-Type 0x8906. The Priority-Map for the FCoE application shows the 8-bit format of the code-point setting that was specified for the PFC congestion notification profile. In this case, the three bit code point is 3, 011. So the Priority-Map for the default FCoe application is 00001000.

The fcoe forwarding-class and PFC were configured; and the configuration of the application on the switch and on the DCB are synchronized. Therefore, the Status of the FCoE application is Enabled.

If the configuration of the FCoE application on the switch did not match the FCoE application of the DCB peer, the status of the application would appear as Disabled.