Configuring an FCoE LAG
A Fibre Channel over Ethernet (FCoE) link aggregation group (LAG) is a special LAG that enables you to transport FCoE traffic and regular Ethernet traffic across the same link aggregation bundle. Standard LAGs use a hashing algorithm to determine which physical link in the LAG is used for a transmission, so a series of communications between two devices might use different physical links in the LAG for different transmissions.
However, FCoE traffic requires a point-to-point link (or a virtual point-to-point link) between the FCoE device and the Fibre Channel (FC) storage area network (SAN) switch. This requirement means that communication between an FCoE device and a QFabric system Node device must use the same physical link in a LAG to maintain the virtual point-to-point connection.
An FCoE LAG solves the problem by ensuring that the same LAG link is used for communication between an FC SAN switch and a given FCoE device across a QFabric system Node device, preserving point-to-point link emulation. At the same time, regular Ethernet traffic (traffic that is not FCoE traffic) on the LAG is distributed across member interfaces in the same way as on a standard LAG. FCoE traffic is treated properly in terms of maintaining a virtual point-to-point link with the FC SAN, and regular Ethernet traffic enjoys the usual LAG benefits of load balancing and link redundancy.
Configuring a LAG as an FCoE LAG does not provide link redundancy for FCoE traffic, and does not load balance FCoE traffic.
On FCoE-FC gateway untrusted Fibre Channel fabrics (fc-fabrics), if you configure an FCoE LAG, you must also disable enhanced FIP snooping scaling (scaling up to 2,500 sessions), which reduces the number of supported FIP snooping sessions to 376 sessions. On an FCoE-FC gateway, disabling enhanced FIP snooping scaling is global to the Node device. Trusted fc-fabrics on an FCoE-FC gateway support enhanced FIP snooping scaling.
This example shows how to configure an FCoE LAG with enhanced FIP snooping scaling enabled and also with enhanced FIP snooping scaling disabled.
The steps required to create the FCoE LAG are:
Configuring an FCoE LAG interface.
Assigning the Ethernet interfaces connected to the FCoE device to the FCoE LAG.
Configuring FIP snooping.
In addition to configuring the FCoE LAG and FIP snooping scaling, you also need to configure:
Configure a dedicated FCoE VLAN for the FCoE traffic.
Configure a native VLAN for the untagged FIP traffic.
Enable FIP snooping on the FCoE VLAN.
Configure the FCoE LAG interface membership in the FCoE VLAN and the native VLAN.
For FCoE-FC gateway switches, configure a Layer 3 FCoE VLAN interface, and add the FCoE VLAN interface to the Fibre Channel fabric.
For FCoE-FC gateway switches, configure the fc-fabric as an FCoE trusted fabric if you are using enhanced FIP snooping scaling (and if the FCoE traffic is trusted).
Example: Configuring an FCoE LAG on a Redundant Server Node Group includes an example of this configuration.
Configuring an FCoE LAG When Enhanced FIP Snooping Scaling
This example shows how to configure an FCoE LAG when you can use enhanced FIP snooping scaling, such as when the FCoE-FC gateway fabrics are trusted, or on an FCoE transit switch.
- Specify the number of LAGs (Ethernet devices) the QFabric
system Node group will support:
admin@qfabric# set chassis node-group node-group-name aggregated-devices ethernet device-count device-count
For example, to configure the Node group RSNG1 to allow up to ten LAGs:
admin@qfabric# set chassis node-group RSNG1 aggregated-devices ethernet device-count 10
- Configure the LAG interface on the RSNG:
admin@qfabric# set interfaces lag-interface-name unit unit family ethernet-switching port-mode trunk
For example, to configure a LAG interface named ae3 on Node group RSNG1:
admin@qfabric# set interfaces RSNG1:ae3 unit 0 family ethernet-switching port-mode trunk
- Configure the LAG interface as an FCoE LAG:
admin@qfabric# set interfaces lag-interface-name aggregated-ether-options fcoe-lag
For example, to configure LAG ae3 on a Node group named RSNG1 as an FCoE LAG:
admin@qfabric# set interfaces RSNG1:ae3 aggregated-ether-options fcoe-lag
- Enable LACP on the FCoE LAG:
admin@qfabric# set interfaces fcoe-lag-interface-name aggregated-ether-options lacp active
For example, to configure LACP on FCoE LAG RSNG1:ae3:
admin@qfabric# set interfaces RSNG1:ae3 aggregated-ether-options lacp active
- Assign the Ethernet interfaces connected to the FCoE device
converged network adapter (CNA) to the FCoE LAG:
admin@qfabric# set interfaces interface-name ether-options 802.3ad fcoe-lag-name
For example, to assign interfaces xe-0/0/20 and xe-0/0/21 on Node device row1-rack1 (which is part of the Node group RSNG1) to the FCoE LAG ae3 (on Node group RSNG1):
admin@qfabric# set interfaces row1-rack1:xe-0/0/20 ether-options 802.3ad RSNG1:ae3
admin@qfabric# set interfaces row1-rack1:xe-0/0/21 ether-options 802.3ad RSNG1:ae3
On QFabric system Node groups that have two or more member nodes, you can assign interfaces from any Node in the Node group to the FCoE LAG. Adding to the example, if Node device row2-rack1 is part of Node group RSNG1, then you can add interfaces from row2-rack1 to the FCoE LAG. For example, set interfaces row2-rack1:xe-0/0/20 ether-options 802.3ad RSNG1:ae3 adds an interface on a second Node device to the FCoE LAG.
- Enable FIP snooping on the FCoE VLAN:
admin@qfabric# set ethernet-switching-options secure-access-port vlan fcoe-vlan-name examine-fip
For example, to enable FIP snooping on an FCoE VLAN named fcoe-vlan-blue:
admin@qfabric# set ethernet-switching-options secure-access-port vlan fcoe-vlan-blue examine-fip
- On an FCoE-FC gateway only, enable FCoE trusted mode on
admin@qfabric# set fc-fabrics fc-fabric-name protocols fip fcoe-trusted
For example, to configure an fc-fabric named sanfab1 as an FCoE trusted fabric:
admin@qfabric# set fc-fabrics sanfab1 protocols fip fcoe-trusted
Configuring an FCoE LAG When Enhanced FIP Snooping Scaling
Must be Disabled
This example shows how to configure an FCoE LAG when you need to disable enhanced FIP snooping scaling, for example, when an FCoE-FC gateway fabric is untrusted.
Follow steps 1-6 of the preceding example to configure the FCoE LAG and enable FIP snooping on the FCoE VLAN. Next, disable enhanced FIP snooping scaling:
- On an FCoE-FC gateway switch, disable FIP snooping scaling
on all FCoE LAGs in the Fibre Channel fabric options configuration:
admin@qfabric# set fc-options no-fip-snooping-scaling
This global statement disables FIP snooping scaling on all FCoE LAGs associated with all FC fabrics on the switch.