Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Example: Setting Up Fibre Channel and FCoE VLAN Interfaces in an FCoE-FC Gateway Fabric

 

To transmit Fibre Channel (FC) traffic between FCoE devices and a storage area network (SAN) FC switch, you configure a local FC fabric on the gateway. The gateway FC fabric includes FCoE and native FC interfaces, and a VLAN to carry FCoE traffic from FCoE-capable devices. The gateway FC fabric creates the path between the FCoE devices and the SAN.

This example describes how to configure the interfaces, VLAN, and FC fabric to connect FCoE devices to the FC switch and route traffic between the VLAN and FC interfaces:

Requirements

This example uses the following hardware and software components:

  • A configured and provisioned Juniper Networks QFX3500 Switch to act as an FCoE-FC gateway

  • FCoE-capable devices in an Ethernet network equipped with converged network adapters (CNAs)

  • An FC switch to transmit and receive native FC traffic

  • FC storage devices in the SAN

  • Junos OS Release 11.1 or later for the QFX Series

Overview

No interfaces are configured for FC network connectivity by default. You need to configure the FC fabric and its interfaces explicitly. Each FC fabric consists of a combination of at least one FCoE VLAN interface between the FCoE-FC gateway and the FCoE devices, and one or more native FC interfaces between the FCoE-FC gateway and the FC switch.

An FCoE VLAN interface connects the FCoE-FC gateway to FCoE devices. FCoE traffic between the devices and the FCoE-FC gateway requires a dedicated VLAN used only for FCoE traffic. You cannot mix standard Ethernet traffic and FCoE traffic on the FCoE VLAN.

Note

IGMP snooping is not supported on FCoE VLANs. IGMP snooping is enabled by default on all VLANs in all software versions before Junos OS Release 13.2. Disable IGMP snooping on FCoE VLANs if you are using software that is older than 13.2.

Storm control is not supported on Ethernet interfaces that belong to the FCoE VLAN. Ensure that storm control is disabled on all Ethernet interfaces that belong to the FCoE VLAN to prevent FCoE traffic from being dropped.

When FCoE frames enter the FCoE-FC gateway, the gateway:

  1. Strips the Ethernet encapsulation from the FCoE frames.
  2. Sends the resulting native FC frames to the FC switch through the gateway’s native FC interfaces.

Each FC interface and FCoE VLAN interface can belong to only one FC fabric. Different FC fabrics must use different native FC interfaces and different FCoE VLAN interfaces. Multiple FC fabrics on the FCoE-FC gateway can connect to the same FC switch, but they must use different FC interfaces and different FCoE VLAN interfaces.

The Ethernet interfaces that belong to the FCoE VLAN should be configured in tagged-access port mode and must include the native VLAN because FIP VLAN discovery and notification frames are exchanged as untagged packets. These Ethernet interfaces require a maximum transmission unit (MTU) size of at least 2180 bytes to accommodate the FC payload and FCoE encapsulation. (Sometimes the MTU is rounded up to 2500 bytes. If larger frames are expected on the interface, set the MTU size accordingly.)

This example shows a simple configuration to illustrate the basic steps for creating:

  • The FCoE-device-facing VLAN and its 10-Gigabit Ethernet interfaces

  • The VLAN interface

  • The FC-switch-facing native FC interfaces

  • One FC fabric on the FCoE-FC gateway

Configuring these elements results in traffic being routed between the VLAN and FC interfaces, thus connecting the FCoE devices to the FC switch through the FCoE-FC gateway.

A VLAN called blue transports FCoE traffic between FCoE devices and the FCoE-FC gateway using an FCoE VLAN interface called vlan.100. The FCoE-FC gateway’s vlan.100 interface presents an F_Port interface to the FCoE devices on the VLAN. For each FCoE device ENode that logs in to the FCoE-FC gateway, the gateway instantiates a virtual F_Port (VF_Port) interface. This creates a virtual link between the ENode VN_Port and the FCoE-FC gateway. The FCoE-FC gateway’s native FC interfaces transport FC traffic between the gateway and the FC switch.

Configuring both the FCoE VLAN interface and the native FC interfaces as part of a gateway fabric associates them in the switch and makes the connection between the FCoE servers and the FC switch.

Topology

The topology for this example consists of one QFX3500 switch with FC-capable ports to connect to the FC switch and with Ethernet ports in tagged-access mode to connect to the FCoE devices. Table 1 and Figure 1 show the configuration components of this example.

Table 1: Components of the Fibre Channel Interface Configuration Topology

Property

Settings

Switch hardware

QFX3500 switch in gateway mode

FCoE VLAN name and tag ID

blue, tag 100

IGMP snooping disabled on the FCoE VLAN.

Interfaces in VLAN blue

Interfaces: xe-0/0/6, xe-0/0/7, xe-0/0/8, xe-0/0/9, xe-0/0/10, xe-0/0/11

Port mode: tagged-access

MTU: 2180

Native VLAN: 1

Note: FCoE VLANs (any VLAN that carries FCoE traffic) support only Spanning Tree Protocol (STP) and link aggregation group (LAG) Layer 2 features.

FCoE traffic cannot use a standard LAG because traffic might be hashed to different physical LAG links on different transmissions. This breaks the (virtual) point-to-point link that Fibre Channel traffic requires. If you configure a standard LAG interface for FCoE traffic, FCoE traffic might be rejected by the FC SAN.

QFabric systems support a special LAG called an FCoE LAG, which enables you to transport FCoE traffic and regular Ethernet traffic across the same link aggregation bundle. An FCoE LAG ensures that FCoE traffic uses the same physical link in the LAG for requests and replies in order to preserve the virtual point-to-point link between the FCoE device converged network adapter (CNA) and the FC SAN switch across the QFabric system Node device. An FCoE LAG does not provide load balancing or link redundancy for FCoE traffic. However, regular Ethernet traffic receives the usual LAG benefits of load balancing and link redundancy in an FCoE LAG.

FCoE VLAN interface

vlan.100

Port mode: f-port

Native Fibre Channel interfaces

Interfaces: fc-0/0/0, fc-0/0/1, fc-0/0/2, fc-0/0/3, fc-0/0/4, fc-0/0/5

Port mode: np-port

Speed: 4 Gbps

Fibre Channel fabric fcproxy1

Fabric type: proxy

Fabric ID: 1

FC interfaces: fc-0/0/0, fc-0/0/1, fc-0/0/2, fc-0/0/3, fc-0/0/4, fc-0/0/5

Figure 1: Fibre Channel Interface Configuration Topology
Fibre Channel Interface Configuration Topology

This configuration example creates a VLAN for FCoE traffic and routes its traffic to an FCoE VLAN interface that is part of the FC fabric. It also creates the FC interfaces needed to connect to the FC switch.

To set up FC interfaces and FCoE VLAN interfaces:

  • Configure a VLAN to use as a dedicated FCoE VLAN:

    • Configure the interfaces the FCoE VLAN uses as Ethernet switching interfaces in tagged-access port mode.

    • If storm control is enabled, disable it on the interfaces.

    • Configure the interfaces the FCoE VLAN uses with the native VLAN.

    • Configure the FCoE VLAN to use the desired Ethernet interfaces.

    • Disable IGMP snooping on the FCoE VLAN. (Before Junos OS Release 13.2, IGMP snooping was enabled by default on all VLANs, but is not supported on FCoE VLANs. Starting with Junos OS Release 13.2, IGMP snooping is enabled by default only on the default VLAN.)

  • Configure the FCoE VLAN interface.

  • Define the interface for the FCoE VLAN (associate the VLAN with the FCoE VLAN interface).

  • Configure the physical FC interfaces (either one or two 6-port blocks) that connect to the FC switch.

  • Configure the logical FC interfaces that connect to the FC switch.

  • Configure the FCoE-FC gateway fabric:

    • Configure the fabric ID.

    • Configure the fabric as a proxy fabric.

    • Add the FCoE VLAN interface and the native FC interfaces to the fabric.

To keep the example simple, the configuration steps show six Ethernet interfaces in the FCoE VLAN and six native FC interfaces in the FC fabric. Use the same configuration procedure to add more interfaces to the FCoE VLAN or to the FC fabric.

Configuration

CLI Quick Configuration

To quickly configure FCoE and native FC interfaces on an FCoE-FC gateway and route traffic between the FCoE VLAN and FC interfaces, copy the following commands and paste them into the switch terminal window:

[edit]


set vlans blue vlan-id 100


set vlans native vlan-id 1


set interfaces xe-0/0/6 unit 0 family ethernet-switching port-mode tagged-access vlan members blue


set interfaces xe-0/0/7 unit 0 family ethernet-switching port-mode tagged-access vlan members blue


set interfaces xe-0/0/8 unit 0 family ethernet-switching port-mode tagged-access vlan members blue


set interfaces xe-0/0/9 unit 0 family ethernet-switching port-mode tagged-access vlan members blue


set interfaces xe-0/0/10 unit 0 family ethernet-switching port-mode tagged-access vlan members blue


set interfaces xe-0/0/11 unit 0 family ethernet-switching port-mode tagged-access vlan members blue


set interfaces xe-0/0/6 unit 0 family ethernet-switching native-vlan-id 1


set interfaces xe-0/0/7 unit 0 family ethernet-switching native-vlan-id 1


set interfaces xe-0/0/8 unit 0 family ethernet-switching native-vlan-id 1


set interfaces xe-0/0/9 unit 0 family ethernet-switching native-vlan-id 1


set interfaces xe-0/0/10 unit 0 family ethernet-switching native-vlan-id 1


set interfaces xe-0/0/11 unit 0 family ethernet-switching native-vlan-id 1


set interfaces xe-0/0/6 mtu 2180


set interfaces xe-0/0/7 mtu 2180


set interfaces xe-0/0/8 mtu 2180


set interfaces xe-0/0/9 mtu 2180


set interfaces xe-0/0/10 mtu 2180


set interfaces xe-0/0/11 mtu 2180


set vlans blue interface xe-0/0/6.0


set vlans blue interface xe-0/0/7.0


set vlans blue interface xe-0/0/8.0


set vlans blue interface xe-0/0/9.0


set vlans blue interface xe-0/0/10.0


set vlans blue interface xe-0/0/11.0


set protocols igmp-snooping vlan blue disable


set interfaces vlan unit 100 family fibre-channel port-mode f-port


set vlans blue l3-interface vlan.100


set chassis fpc 0 pic 0 fibre-channel port-range 0 5


set interfaces fc-0/0/0 unit 0 family fibre-channel port-mode np-port


set interfaces fc-0/0/1 unit 0 family fibre-channel port-mode np-port


set interfaces fc-0/0/2 unit 0 family fibre-channel port-mode np-port


set interfaces fc-0/0/3 unit 0 family fibre-channel port-mode np-port


set interfaces fc-0/0/4 unit 0 family fibre-channel port-mode np-port


set interfaces fc-0/0/5 unit 0 family fibre-channel port-mode np-port


set interfaces fc-0/0/0 fibrechannel-options speed 4g


set interfaces fc-0/0/1 fibrechannel-options speed 4g


set interfaces fc-0/0/2 fibrechannel-options speed 4g


set interfaces fc-0/0/3 fibrechannel-options speed 4g


set interfaces fc-0/0/4 fibrechannel-options speed 4g


set interfaces fc-0/0/5 fibrechannel-options speed 4g


set fc-fabrics fcproxy1 fabric-id 1


set fc-fabrics fcproxy1 fabric-type proxy


set fc-fabrics fcproxy1 interface vlan.100


set fc-fabrics fcproxy1 interface fc-0/0/0.0


set fc-fabrics fcproxy1 interface fc-0/0/1.0


set fc-fabrics fcproxy1 interface fc-0/0/2.0


set fc-fabrics fcproxy1 interface fc-0/0/3.0


set fc-fabrics fcproxy1 interface fc-0/0/4.0


set fc-fabrics fcproxy1 interface fc-0/0/5.0


Step-by-Step Procedure

Configure FCoE and FC interfaces in an FCoE-FC gateway FC fabric and set up traffic routing between the FCoE VLAN and FC interfaces:

  1. Configure the VLAN for FCoE traffic:
    [edit vlans]

    user@switch# set blue vlan-id 100



  2. Configure the native VLAN:
    [edit vlans]

    user@switch# set native vlan-id 1



  3. Configure the Ethernet interfaces for the FCoE VLAN in tagged-access mode and as members of the FCoE VLAN (VLAN blue):
    [edit interfaces]

    user@switch# set xe-0/0/6 unit 0 family ethernet-switching port-mode tagged-access vlan members blue


    user@switch# set xe-0/0/7 unit 0 family ethernet-switching port-mode tagged-access vlan members blue


    user@switch# set xe-0/0/8 unit 0 family ethernet-switching port-mode tagged-access vlan members blue


    user@switch# set xe-0/0/9 unit 0 family ethernet-switching port-mode tagged-access vlan members blue


    user@switch# set xe-0/0/10 unit 0 family ethernet-switching port-mode tagged-access vlan members blue


    user@switch# set xe-0/0/11 unit 0 family ethernet-switching port-mode tagged-access vlan members blue



  4. Configure the native VLAN on the Ethernet interfaces in the FCoE VLAN:
    [edit interfaces]

    user@switch# set xe-0/0/6 unit 0 family ethernet-switching native-vlan-id 1


    user@switch# set xe-0/0/7 unit 0 family ethernet-switching native-vlan-id 1


    user@switch# set xe-0/0/8 unit 0 family ethernet-switching native-vlan-id 1


    user@switch# set xe-0/0/9 unit 0 family ethernet-switching native-vlan-id 1


    user@switch# set xe-0/0/10 unit 0 family ethernet-switching native-vlan-id 1


    user@switch# set xe-0/0/11 unit 0 family ethernet-switching native-vlan-id 1



  5. Set the MTU to 2180 for each Ethernet interface:
    [edit interfaces]

    user@switch# set xe-0/0/6 mtu 2180


    user@switch# set xe-0/0/7 mtu 2180


    user@switch# set xe-0/0/8 mtu 2180


    user@switch# set xe-0/0/9 mtu 2180


    user@switch# set xe-0/0/10 mtu 2180


    user@switch# set xe-0/0/11 mtu 2180




  6. Assign the Ethernet interfaces to the FCoE VLAN:
    [edit vlans blue interface]

    user@switch# set xe-0/0/6.0


    user@switch# set xe-0/0/7.0


    user@switch# set xe-0/0/8.0


    user@switch# set xe-0/0/9.0


    user@switch# set xe-0/0/10.0


    user@switch# set xe-0/0/11.0




  7. Disable IGMP snooping on the FCoE VLAN:
    [edit protocols]

    user@switch# set igmp-snooping vlan blue disable



  8. Configure the FCoE VLAN interface and port mode for the FCoE traffic:
    [edit interfaces]

    user@switch# set vlan unit 100 family fibre-channel port-mode f-port



  9. Define the FCoE VLAN interface as the interface for the FCoE VLAN:
    [edit vlans]

    user@switch# set blue l3-interface vlan.100



  10. Configure the physical FC interfaces the fabric uses to connect to the FC switch:
    [edit chassis fpc 0 pic 0]

    user@switch# set fibre-channel port-range 0 5
    Note

    When you configure ports as FC ports, the port designation changes from xe-n/n/n.n format to fc-n/n/n.n format to indicate that the interface is an FC interface. FC interfaces do not support 10-Gbps interface speed but instead conform to FC interface speeds of 2 Gbps, 4 Gbps, or 8 Gbps.



  11. Configure the native FC interfaces and port mode:
    [edit interfaces]


    user@switch# set fc-0/0/0 unit 0 family fibre-channel port-mode np-port

    user@switch# set fc-0/0/1 unit 0 family fibre-channel port-mode np-port

    user@switch# set fc-0/0/2 unit 0 family fibre-channel port-mode np-port

    user@switch# set fc-0/0/3 unit 0 family fibre-channel port-mode np-port

    user@switch# set fc-0/0/4 unit 0 family fibre-channel port-mode np-port

    user@switch# set fc-0/0/5 unit 0 family fibre-channel port-mode np-port



  12. Configure the native FC interface port speed:
    [edit interfaces]


    user@switch# set fc-0/0/0 fibrechannel-options speed 4g

    user@switch# set fc-0/0/1 fibrechannel-options speed 4g

    user@switch# set fc-0/0/2 fibrechannel-options speed 4g

    user@switch# set fc-0/0/3 fibrechannel-options speed 4g

    user@switch# set fc-0/0/4 fibrechannel-options speed 4g

    user@switch# set fc-0/0/5 fibrechannel-options speed 4g



  13. Configure the FC fabric name and unique ID:
    [edit fc-fabrics]

    user@switch# set fcproxy1 fabric-id 1



  14. Define the FC fabric as an FCoE-FC gateway:
    [edit fc-fabrics]

    user@switch# set fcproxy1 fabric-type proxy



  15. Assign the FCoE VLAN interface to the fabric:
    [edit fc-fabrics]

    user@switch# set fcproxy1 interface vlan.100



  16. Assign the native FC interfaces to the fabric:
    [edit fc-fabrics]

    user@switch# set fcproxy1 interface fc-0/0/0.0

    user@switch# set fcproxy1 interface fc-0/0/1.0

    user@switch# set fcproxy1 interface fc-0/0/2.0

    user@switch# set fcproxy1 interface fc-0/0/3.0

    user@switch# set fcproxy1 interface fc-0/0/4.0

    user@switch# set fcproxy1 interface fc-0/0/5.0



Results

Display the results of the configuration:





Tip

To quickly configure the interfaces, issue the load merge terminal command and then copy the hierarchy and paste it into the switch terminal window.

Verification

To verify that the native FC interfaces and FCoE VLAN interface have been created, added to the FC fabric, and are operating properly, perform these tasks:

Verifying That the Native FC Interfaces and the FCoE VLAN Interface Have Been Created

Purpose

Verify that the six native FC interfaces and the FCoE VLAN interface have been created on the switch and are configured in the correct mode.

Action

List all of the FC interfaces configured on the switch using the show fibre-channel interfaces command:

Meaning

The show fibre-channel interfaces command lists all native FC interfaces and FCoE VLAN interfaces configured on the switch. The command output shows that the FC interfaces fc-0/0/0.0, fc-0/0/1.0, fc-0/0/2.0, fc-0/0/3.0, fc-0/0/4.0, and fc-0/0/5.0 have been created and that those six interfaces:

  • Are native Fibre Channel interfaces (type FC).

  • Belong to the FC fabric with a configured fabric ID of 1.

  • Are capable of N_Port ID virtualization (NPIV).

  • Have a configured mode and an operational mode of proxy N_Port (NP), which means that they should be connected to an FCF or an FC switch, not to an FCoE device, and that they carry native FC traffic.

  • Show an operational state of up.

The command output also shows that the FCoE VLAN interface vlan.100 has been created and that interface:

  • Is an FCoE VLAN interface (type FCOE).

  • Belongs to the FC fabric with a configured fabric ID of 1.

  • Is capable of N_Port ID virtualization (NPIV).

  • Has a configured mode and an operational mode of F_Port (F), which means that its interfaces connect to FCoE devices and carry FCoE traffic.

  • Shows an operational state of up.

Verifying That the FCoE VLAN Includes the Correct Ethernet Interfaces

Purpose

Verify that the FCoE VLAN blue has been created with the correct VLAN tag (100) and with the correct Ethernet interfaces.

Action

List all of the interfaces configured on the switch in VLAN blue using the show vlans command:

user@switch> show vlans blue

Meaning

The show vlans blue command lists the interfaces that are members of the FCoE VLAN blue. The command output shows that the blue VLAN has a tag ID of 100 and includes the interfaces xe-0/0/6.0, xe-0/0/7.0, xe-0/0/8.0, xe-0/0/9.0, xe-0/0/10.0, and xe-0/0/11.0.

Verifying That the FC Fabric Includes the Correct Interfaces

Purpose

Verify that the FC fabric configuration is configured on the switch with the correct native FC and FCoE VLAN interfaces.

Action

List all of the interfaces configured on FC fabrics on the switch using the show fibre-channel fabric command:

Meaning

The show fibre-channel fabric command lists the interfaces that are members of each FC fabric. The command output shows that the only fabric configured on the switch is named fcproxy1, has a fabric-id of 1, and is a proxy fabric in an FCoE-FC gateway. The command output also shows that the native FC interfaces fc-0/0/0.0, fc-0/0/1.0, fc-0/0/2.0, fc-0/0/3.0, fc-0/0/4.0, and fc-0/0/5.0, and the FCoE VLAN interface vlan.100 belong to fcproxy1.

Verifying Native FC Interface Operation

Purpose

Verify that the native FC interfaces are online and display the number of FC sessions on each interface.

Action

List all of the native FC NP_Port interface states and sessions by FC fabric using the show fibre-channel proxy np-port command:

Meaning

The show fibre-channel proxy np-port command lists the interfaces that are configured as native FC proxy N_Port interfaces. The command output shows:

  • The fabric name is fcproxy1 and its fabric ID is 1.

  • The interfaces are online.

  • The number of FC sessions (virtual links) running on each interface.

  • The load-balancing (LB) state is ON for all of the interfaces.

  • The LB weight reflects the port speed of each interface, which is 4 Gbps.

Verifying That IGMP Snooping Has Been Disabled on the FCoE VLAN

Purpose

Verify that IGMP snooping is disabled on the FCoE VLAN.

Action

List the IGMP snooping protocol information for the FCoE VLAN using the show configuration protocols igmp-snooping vlan blue command:

user@switch> show configuration protocols igmp-snooping vlan blue

Meaning

The show configuration protocols igmp-snooping vlan blue command lists the IGMP snooping configuration for the FCoE VLAN. The command output shows that IGMP snooping is disabled on the FCoE VLAN.