Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Example: Configuring an FCoE LAG on a Redundant Server Node Group

 

This example shows how to configure a Fibre Channel over Ethernet (FCoE) link aggregation group (LAG) on a redundant server Node group (RSNG) to transport FCoE traffic and regular Ethernet traffic across the same link aggregation bundle. The FCoE servers have converged network adapters (CNAs) and communicate with the Fibre Channel (FC) storage area network (SAN). FCoE servers are usually connected to a switch that performs FIP snooping, such as an FCoE transit switch or an FCoE-FC gateway switch that performs FIP snooping. This example provides a common FCoE LAG configuration for an FCoE transit switch and an FCoE-FC gateway, and shows how to disable FIP snooping scaling on an FCoE untrusted FCoE-FC gateway fabric (fc-fabric).

Requirements

This example uses the following hardware and software components:

  • Two Juniper Networks QFabric System Node devices configured as an RSNG. The Node devices can be configured as FCoE transit switches or as FCoE-FC gateways. (A configuration with one Node device as an FCoE transit switch and the other Node device as an FCoE-FC gateway is possible providing that the transit switch and the FCoE-FC gateway use different FCoE VLANs.)

  • Junos OS Release 13.2X52-D10 or later for the QFX Series

  • One FCoE server with two CNA ports

Overview

Standard LAGs use a hashing algorithm to determine which physical link in the LAG is used for a transmission, so a series of communications between two devices might use different physical links in the LAG for different transmissions. However, FCoE traffic requires a point-to-point link (or a virtual point-to-point link) between the FCoE device and the Fibre Channel (FC) storage area network (SAN) switch.

An FCoE LAG solves this problem by ensuring that the same LAG link is used for communication between a given FCoE device and the QFabric system Node device, preserving point-to-point link emulation. At the same time, regular Ethernet traffic (traffic that is not FCoE traffic) on the LAG is distributed across member interfaces in the same way as on a standard LAG. FCoE traffic is treated properly in terms of maintaining a virtual point-to-point link with the FC SAN, and regular Ethernet traffic enjoys the usual LAG benefits of load balancing and link redundancy.

Note

Configuring a LAG as an FCoE LAG does not provide link redundancy for FCoE traffic, and does not load balance FCoE traffic.

On FCoE-FC gateway untrusted Fibre Channel fabrics (fc-fabrics), if you configure an FCoE LAG, you must also disable enhanced FIP snooping scaling (scaling up to 2,500 sessions), which reduces the number of supported FIP snooping sessions to 376 sessions. On an FCoE-FC gateway, disabling enhanced FIP snooping scaling is global to the Node device. Trusted fc-fabrics on an FCoE-FC gateway support enhanced FIP snooping scaling.

This example shows you how to:

  • Configure the RSNG and its Node devices

  • Configure the FCoE LAG on the RSNG

  • Configure a dedicated VLAN for FCoE traffic (an FCoE VLAN) and a native VLAN for untagged FCoE initialization protocol (FIP) traffic

  • Enable VN2VF_Port FIP snooping on the FCoE VLAN

  • Disable FIP snooping scaling on an untrusted FCoE-FC gateway fabric

Note

FCoE traffic requires lossless transport across the Ethernet network to comply with the requirements for transporting storage traffic. This example describes how to configure an FCoE LAG to provide redundancy for FCoE traffic. See Example: Configuring CoS PFC for FCoE Traffic for how to configure lossless transport for FCoE traffic.

Note

On a Node device that is configured as an FCoE-FC gateway, you must create a Fibre Channel fabric, configure native FC interfaces, configure an FCoE VLAN interface (a Layer 3 RVI) for the FCoE VLAN (which includes the FCoE LAG as a member interface), and add the native FC interfaces and FCoE VLAN interface to the FC fabric. For an example of FCoE-FC gateway interface configuration, see Example: Setting Up Fibre Channel and FCoE VLAN Interfaces in an FCoE-FC Gateway Fabric.

Topology

Table 1 shows the configuration components for this example.

Table 1: Components of the FCoE LAG Configuration Example

Component

Settings

Hardware

Two QFabric system Node devices configured as an RSNG (the Node devices can be configured as FCoE transit switches or as FCoE-FC gateways; this example is valid for both modes):

  • RSNG name—RSNG1

  • First Node device—Serial number ABCD1234, alias name row1-rack1

  • Second Node device—Serial number ABCD1235, alias name row1-rack2

Note: The alias names chosen for this example indicate the physical locations of the Node devices. You can use any aliasing system you want to make identifying Node devices easier, or you can use the default Node device names (the Node device serial numbers).

One FCoE server with two CNA ports.

LAG configuration

RSNG device count—48

FCoE LAG name—RSNG1:ae20

FCoE LAG member interfaces—row1rack1:xe-0/0/20 and row1rack2:xe-0/0/20

FCoE LAG LACP—active

FCoE LAG port mode—trunk

MTU—2180

FCoE LAG VLAN memberships—FCoE VLAN (fcoe-vlan1) and native VLAN

FCoE VLAN

Name—fcoe-vlan1

VLAN ID—2000

Member interfaces—RSNG1:ae20

Native VLAN

Name—native

VLAN ID—1

Member interfaces—RSNG1:ae20

VN2VF_Port FIP snooping

Enabled on the FCoE VLAN (fcoe-vlan1)

FIP snooping scaling

Enabled for FCoE transit switch portion of the example.

Disabled for the FCoE-FC gateway portion of the example (gateway FC fabric is FCoE untrusted).

Figure 1 shows the network topology for this example.

Figure 1: FCoE LAG Example Topology
FCoE LAG Example Topology

Configuration

To configure an FCoE LAG between an FCoE server with two CNA ports and the two Node device members of an RSNG, perform these tasks:

CLI Quick Configuration

In this example, the enhanced FIP snooping scaling is disabled (376 sessions) on the FCoE-FC gateway because the gateway fabric is an untrusted fc-fabric.

Most of the FCoE LAG configuration is common to both the FCoE transit switch and FCoE-FC gateway modes of operation. The CLI Quick Configuration shows the common configuration statements first, followed by the additional configuration statement that disables FIP snooping scaling on the FCoE-FC gateway. Disabling FIP snooping scaling on an FCoE-FC gateway is a global configuration that affects all of the fc-fabrics on the gateway. (On an FCoE transit switch, you can disable FIP snooping scaling on an individual FCoE VLAN without affecting other FCoE VLANs.)

Note

This example does not include configuring the FC fabric, the native FC fabric ports, and the Layer 3 FCoE VLAN interface.

To quickly configure an FCoE LAG , copy the following commands, paste them in a text file, remove line breaks, change variables and details to match your network configuration, and then copy and paste the commands into the CLI at the [edit] hierarchy level.

Common configuration:

set fabric aliases node-device ABCD1234 row1-rack1

set fabric aliases node-device ABCD1235 row1-rack2

set fabric resources node-group RSNG1 node-device row1-rack1

set fabric resources node-group RSNG1 node-device row1-rack2

set chassis node-group RSNG1 aggregated-devices ethernet device-count 48

set interfaces RSNG1:ae20 unit 0 family ethernet-switching port-mode trunk vlan members fcoe-vlan1

set interfaces RSNG1:ae20 unit 0 family ethernet-switching native-vlan-id 1

set interfaces RSNG1:ae20 mtu 2180

set interfaces RSNG1:ae20 aggregated-ether-options fcoe-lag

set interfaces RSNG1:ae20 aggregated-ether-options lacp active

set interfaces row1-rack1:xe-0/0/20 ether-options 802.3ad RSNG1:ae20

set interfaces row1-rack2:xe-0/0/20 ether-options 802.3ad RSNG1:ae20

set vlans fcoe-vlan1 vlan-id 2000

set vlans native vlan-id 1

set vlans fcoe-vlan1 interface RSNG1:ae20

set ethernet-switching-options secure-access-port vlan fcoe-vlan1 examine-fip

Note

If you want to configure an FCoE-FC gateway fabric as a trusted fabric so that you can leave enhanced FIP snooping scaling enabled on the gateway, add the following statement to the configuration, replacing the variable fc-fabric-name with the name of the FC fabric (if you do this, do not disable FIP snooping scaling as shown in the FCoE-FC Gateway Additional Configuration):

set fc-fabrics fc-fabric-name protocols fip fcoe-trusted

Additional configuration to disable enhanced FIP snooping scaling on an FCoE-FC gateway untrusted FC fabric:

set fc-options no-fip-snooping-scaling

Configuring an FCoE LAG on an RSNG (FCoE Transit Switch or FCoE-FC Gateway)

Step-by-Step Procedure

To configure the RSNG member Node devices, the FCoE LAG, the FCoE VLAN, and VN2VF_Port FIP snooping on an FCoE transit switch or an FCoE-FC gateway:

  1. Define aliases for the two Node devices that will be in the RSNG (aliases are easier to remember and more descriptive than the Node device serial number). Name the Node device with serial number ABCD1234 as row1-rack1 and the Node device with the serial number ABCD1235 as row1-rack2:
    admin@qfabric# set fabric aliases node-device ABCD1234 row1-rack1

    admin@qfabric# set fabric aliases node-device ABCD1235 row1-rack2



  2. Configure the Node device membership for row1-rack1 and row1-rack2 in the RSNG RSNG1:
    admin@qfabric# set fabric resources node-group RSNG1 node-device row1-rack1

    admin@qfabric# set fabric resources node-group RSNG1 node-device row1-rack2



  3. Configure the number of LAG interfaces that RSNG RSNG1 can support. (Each Node device in the RSNG has 48 server-facing ports. If we used one port from each Node device to provide Node device redundancy for each LAG, we might need to support a maximum of 48 LAGs, so we set the device count to 48 LAGs.)
    admin@qfabric# set chassis node-group RSNG1 aggregated-devices ethernet device-count 48



  4. Configure the LAG interface (ae20) on RSNG1 and set the port mode to trunk mode. In the same statement, configure the LAG interface membership in the dedicated FCoE VLAN fcoe-vlan1:
    admin@qfabric# set interfaces RSNG1:ae20 unit 0 family ethernet-switching port-mode trunk vlan members fcoe-vlan1



  5. Configure the LAG interface membership in the native VLAN:
    admin@qfabric# set interfaces RSNG1:ae20 unit 0 family ethernet-switching native-vlan-id 1



  6. Configure the LAG interface with an MTU of 2180 to accommodate the size of the FCoE frame and headers.
    admin@qfabric# set interfaces RSNG1:ae20 mtu 2180



  7. Configure the LAG RSNG1:ae20 as an FCoE LAG:
    admin@qfabric# set interfaces RSNG1:ae20 aggregated-ether-options fcoe-lag



  8. Enable LACP on the FCoE LAG:
    admin@qfabric# set interfaces RSNG1:ae20 aggregated-ether-options lacp active



  9. Assign one Ethernet interface on each RSNG Node device to the FCoE LAG:
    admin@qfabric# set interfaces row1-rack1:xe-0/0/20 ether-options 802.3ad RSNG1:ae20

    admin@qfabric# set interfaces row1-rack2:xe-0/0/20 ether-options 802.3ad RSNG1:ae20



  10. Configure a dedicated VLAN for FCoE traffic (an FCoE VLAN) named fcoe-vlan1 with the VLAN ID 2000:
    admin@qfabric# set vlans fcoe-vlan1 vlan-id 2000



  11. Configure a native VLAN with the VLAN ID 1 to carry untagged FIP traffic:
    admin@qfabric# set vlans native vlan-id 1



  12. Assign the FCoE LAG interface to the FCoE VLAN:
    admin@qfabric# set vlans fcoe-vlan1 interface RSNG:ae20



  13. Assign the FCoE LAG interface to the native VLAN:
    admin@qfabric# set vlans native interface RSNG:ae20



  14. Enable VN2VF_Port FIP snooping on the FCoE VLAN:
    admin@qfabric# set ethernet-switching-options secure-access-port vlan fcoe-vlan1 examine-fip



Disabling Enhanced FIP Snooping Scaling on an FCoE-FC Gateway

Step-by-Step Procedure

To disable enhanced FIP snooping scaling on an FCoE-FC gateway:

  1. Disable FIP snooping scaling on the gateway fabrics. Disabling FIP snooping scaling on an FCoE-FC gateway is global to the gateway, so every FC fabric on the gateway reverts to supporting 376 sessions (instead of 2,500 sessions as with FIP snooping scaling enabled).
    admin@qfabric# set fc-options no-fip-snooping-scaling



Results

Display the results of the configuration. The results below show the configuration on an FCoE transit switch and have been edited to include only the components configured in the example:

admin@qfabric> show configuration

Verification

To verify the configuration of the QFabric system Node device resources, FCoE LAG, FCoE VLAN, native VLAN, and FIP snooping, perform these tasks:

Verifying the Node Device Aliases (Names)

Purpose

Verify that the Node device alias names are configured.

Action

List the Node device inventory on the QFabric system using the show fabric administration inventory node-devices command:

admin@qfabric> show fabric administration inventory node-devices

Meaning

The show fabric administration inventory node-devices command lists the Node device names in the Node device column and lists the Node device serial numbers in the Identifier column. The Connection column shows if the Director device has detected the Node device, and the Model column lists QFX switch model type.

The command output shows that Node device ABCD1234 is configured with the name (alias) row1-rack1, and the Node device ABCD1235 is configured with the name row1-rack2.

Verifying the Node device Assignment to the Node Group

Purpose

Verify that the redundant server Node group includes the two Node devices.

Action

Verify that the QFabric system Node group RSNG1 is configured with the correct Node devices using the show configuration fabric resources command:

admin@qfabric> show configuration fabric resources

Meaning

The show configuration fabric resources command lists the Node groups and the Node devices in the Node groups. The command output shows that Node group RSNG1 consists of the Node devices row1-rack1 and row1-rack2.

Verifying the Number of Aggregated Ethernet Logical Devices (LAG Interfaces) That the Node Group Can Support

Purpose

Verify the number of LAG interfaces that the redundant server node group supports.

Action

List the LAG interface device count using the show configuration chassis command:

admin@qfabric> show configuration chassis

Meaning

The show configuration chassis command displays the Ethernet device count (the number of LAG interfaces supported) as 48 devices.

Verifying the FCoE LAG Interface Configuration

Purpose

Verify that the FCoE LAG interface, port mode, interface VLAN membership, and Node device interface membership in the FCoE LAG are correctly configured.

Action

List the FCoE LAG interface and Node device interface information using the show configuration interfaces command:

admin@qfabric> show configuration interfaces

Meaning

The show configuration interfaces command lists both the LAG interfaces and the individual Node device interfaces, and their configuration.

The command output shows a lot of information about the interfaces:

  • The LAG interface name is RSNG1:ae20

  • fcoe-lag confirms the LAG is an FCoE LAG

  • lacp is configured in active mode

  • Port mode is trunk

  • The LAG has membership in the fcoe-vlan1 VLAN and in the native VLAN with the VLAN ID 1.

  • Interface row1-rack1:xe-0/0/20 is a member of FCoE LAG RSNG1:ae20

  • Interface row1-rack2:xe-0/0/20 is a member of FCoE LAG RSNG1:ae20

Verifying the FCoE VLAN and Native VLAN Configuration

Purpose

Verify that the FCoE VLAN fcoe-vlan1 and the native VLAN native are configured with the correct VLAN tags (2000 and 1, respectively) and that the FCoE LAG interface RSNG1:ae20 is assigned to the VLANs.

Action

List the VLAN information using the show configuration vlans command:

admin@qfabric> show configuration vlans

Meaning

The show configuration vlans command lists the configured VLANs, their VLAN IDs, and the interfaces assigned to the VLANs.

The command output shows that the FCoE VLAN fcoe-vlan1 is configured with the VLAN ID 2000 and is assigned to the FCoE LAG interface RSNG1:ae20.

The command output also shows that the native VLAN native is configured with the VLAN ID 1 and is assigned to the FCoE LAG interface RSNG1:ae20.

Verifying the FIP Snooping Configuration

Purpose

Verify that VN2VF_Port FIP snooping is enabled on the FCoE VLAN (fcoe-vlan1).

Action

List the FIP snooping information using the show configuration ethernet-switching-options command:

admin@qfabric> show configuration ethernet-switching-options

Meaning

The show configuration ethernet-switching-options command lists the security options configured on VLANs. The command output shows that on VLAN fcoe-vlan1, VN2VF_Port FIP snooping is enabled (examine-fip output).