ON THIS PAGE
Understanding Virtual Chassis Fabric Components
This topic describes the components of a Virtual Chassis Fabric (VCF).
Spine-and-Leaf Topology
The VCF uses a spine-and-leaf architecture where each device in the fabric is either a spine device or a leaf device.
A VCF can have up to four spine devices, and up to twenty total devices. Each spine device has at least one direct Virtual Chassis port (VCP) connection to each leaf device in the VCF.
All traffic entering a leaf device can, therefore, be forwarded to any directly connected spine device and is always two hops away from any other leaf device—leaf device to leaf device traffic travels from the source leaf device to a spine device to the destination leaf device—within the VCF.
See Figure 1 for an illustration of the VCF spine-and-leaf architecture:
A VCF topology should include at least four members—two spine devices and at least two leaf devices. For topologies with three or fewer members, use a Virtual Chassis configuration instead.
Traffic is forwarded through a VCF using a weighted algorithm designed to avoid congestion. Traffic travelling across the VCF from one leaf device to another leaf device is forwarded using the best path available at the time, so any connection to a spine device can be used to transport traffic from one leaf device to another leaf device.
Supported VCF Configurations
A VCF can be configured using QFX5100 switches as the spine devices, which is referred to as a QFX5100 VCF. Starting in Junos OS Release 17.3R1, a VCF can also be configured using QFX5110-32Q switches as the spine devices, which is referred to as a QFX5110 VCF. The following VCF configurations are supported based on either QFX5110 or QFX5100 switches as the spine members, as indicated:
A non-mixed QFX5100 VCF has QFX5100 switches as spine members, and supports only QFX5100 switches as leaf members.
A mixed QFX5100 VCF has QFX5100 switches as spine members, and supports any combination of EX4300, QFX3500, and QFX3600 switches, possibly with additional QFX5100 switches, as leaf members.
Note:EX4300 multigigabit model (EX4300-48MP) switches are not supported in a VCF.
A non-mixed QFX5110 VCF or simply a QFX5110 VCF has QFX5110-32Q switches as spine members, and supports either only QFX5110 switches or any combination of supported QFX5100 switches and QFX5110 switches as leaf members. (Both QFX5110 and QFX5100 switches run the same software image in a VCF, and do not need to operate in mixed mode.)
Spine Devices
A spine device:
Must be either a QFX5100 switch in a QFX5100 VCF, or a QFX5110-32Q switch in a QFX5110 VCF.
Can be configured into the Routing Engine or linecard role.
In a VCF, at least two spine devices must be configured into the Routing Engine role. The remaining spine devices can be configured as alternate Routing Engine devices or configured into the linecard role.
Has a direct connection to each leaf device.
Typically connects a router, firewall, or other data center networking device to the VCF.
A VCF should always have at least two active spine devices interconnected with at least two leaf devices. A VCF supports up to four spine devices.
In a QFX5110 VCF, you must use only QFX5110-32Q switches as the spine devices.
In a QFX5100 VCF, we recommend using the following QFX5100 switches as spine devices:
QFX5100-24Q switches, in deployments where devices are connecting to the VCF using the 10-Gbps Ethernet interfaces on the leaf devices, or using a mix of 10-Gbps and 1-Gbps Ethernet interfaces on the leaf devices.
QFX5100-96S or QFX5100-48S, in deployments where devices are connecting to the VCF using 1-Gbps Ethernet interfaces only on the leaf devices.
QFX5100-48T switches are not supported as spine devices.
Leaf Devices
A leaf device:
Can be a QFX5100, QFX3500, QFX3600, or EX4300 switch in a QFX5100 VCF.
Can be a QF5110 or QFX5100 switch in a QFX5110 VCF.
Has a direct connection to each spine device.
Usually operates in the linecard role.
For resiliency, if you don’t have additional spine devices to take on the primary or backup Routing Engine role in case a spine device Routing Engine member fails, you can configure a leaf device into the Routing Engine role.
Typically connects an endpoint device—for instance, a server or other storage device in a data center—to the VCF.
A VCF can have up to twenty total devices and up to four devices can be configured as spine devices. The devices that are not spine devices in a VCF operate as leaf devices.
A VCF should include at least four members—two spine devices and at least two leaf devices. For topologies with three or fewer members, use a Virtual Chassis configuration instead.
Routing Engine Role
A VCF has two spine devices operating in the Routing Engine role—a primary Routing Engine and a backup Routing Engine.
The device that functions as the primary Routing Engine:
Is usually a spine device.
Manages the member devices.
Runs the chassis management processes and control protocols.
Represents all the member devices interconnected within the VCF configuration. (The hostname and other parameters that you assign to this device during setup apply to all members of the VCF.)
The device that functions as the backup Routing Engine:
Is usually a spine device.
Maintains a state of readiness to take over the primary role if the primary fails.
Synchronizes with the primary in terms of protocol states, forwarding tables, and so forth, so that it preserves routing information and maintains network connectivity without disruption when the primary is unavailable.
In a VCF, you usually configure at least two spine devices into the Routing Engine role. You can either explicitly configure the remaining spine devices into the linecard role, or any devices not elected as the primary and backup Routing Engines will automatically operate in the linecard role (see Primary Routing Engine Election Process).
A spine device operating in the linecard role can complete all spine-related functions with no limitations within a VCF.
Linecard Role
A spine or a leaf device can be configured into the linecard role in a VCF.
In a VCF, you configure at least two spine devices into the Routing Engine role. You can either explicitly configure the remaining spine devices into the linecard role, or any devices not elected as the primary and backup Routing Engines will automatically operate in the linecard role (see Primary Routing Engine Election Process). A spine device that you don’t explicitly configure into Routing Engine role or that you explicitly configure into linecard role will never take on a primary role if the primary or backup Routing Engine fails.
A spine device operating in the linecard role can complete all spine-related functions with no limitations within a VCF.
Leaf devices in a VCF usually operate in the linecard role.
In autoprovisioned configurations, the VCF assigns the linecard role to leaf devices when they are cabled into the VCF.
In preprovisioned configurations, you must manually configure linecard role on the leaf devices and any spine devices that you don’t want to participate in the primary-role election process (see Primary Routing Engine Election Process).
In nonprovisioned configurations, leaf devices are assigned the linecard role according to the primary-role election algorithm, which uses the primary-role priority values to set the roles of each device in the VCF (among other criteria).
A member that functions in the linecard role in a VCF:
Runs only a subset of Junos OS.
Detects certain error conditions (such as an unplugged cable) on any interfaces that have been configured on it through the device functioning as the primary Routing Engine.
Primary Routing Engine Election Process
The device in the primary Routing Engine role in a VCF is usually a spine device.
In a preprovisioned or autoprovisioned VCF, you configure at least two spine devices into the Routing Engine role. In the primary-role election process, the spine device that has been powered on the longest assumes the primary Routing Engine role; the spine device that has been powered on the second longest assumes the backup Routing Engine role.
In a nonprovisioned VCF, the primary and backup Routing Engines are selected using the following algorithm:
Choose the spine device with the highest user-configured primary-role priority (255 is the highest possible value) as the primary Routing Engine, and the spine device with the second highest primary-role priority value as the backup Routing Engine.
A spine device with a primary-role priority of 0 will always stay in the linecard role.
Choose the spine device that was primary the last time the VCF booted.
Choose the spine device that has been included in the VCF configuration for the longest period of time.
Choose the spine device with the lowest MAC address.
In a QFX5100 VCF, QFX3500, QFX3600, and EX4300 devices never assume the primary or backup Routing Engine role.
In a QFX5110 VCF, the spine devices should be QFX5110-32Q switches, so those will always be the switches in the primary or backup Routing Engine role.
We strongly recommend that you configure the primary-role priority of the spine devices in your VCF to ensure that the correct devices assume their intended roles when you configure your VCF using a nonprovisioned configuration.
Virtual Chassis Ports (VCPs)
Virtual Chassis ports (VCPs) are used in a VCF to interconnect leaf devices to spine devices. All control and data traffic in a VCF is transported over VCPs.
You can configure the following ports into VCPs in a VCF to interconnect switch models that support the indicated port types and speeds:
10-Gbps SFP+ ports in a QFX5100 VCF or a QFX5110 VCF
40-Gbps QSFP+ ports in either a QFX5100 VCF or a QFX5110 VCF
100-Gbps or 40-Gbps QSFP28 ports in a QFX5110 VCF
Channelized interfaces cannot be configured into VCPs.
You can manually configure VCP ports, or all the ports listed above can also be automatically converted into VCPs when a new device is added to an autoprovisioned or preprovisioned VCF under certain conditions. Automatic VCP conversion is discussed in more detail in the following section.
Automatic Virtual Chassis Port (VCP) Conversion
Ports that can be VCPs are automatically converted into VCPs when:
Link Layer Discovery Protocol (LLDP) is enabled on the interfaces on both ends of the VCP link. LLDP is enabled by default.
The device being added to the VCF is configured into fabric mode.
One of the devices is already part of a VCF that was autoprovisioned or preprovisioned.
The interfaces for the ports on both ends of the link are not already configured as VCPs.
For interfaces with any of the following specifications, you must use the
request virtual-chassis vc-port deletecommand to change the interface into a network interface for it to be eligible for automatic VCP conversion:A 40-Gbps QSFP+ port on an EX4300 switch, which is configured into a VCP by default.
Any interface in the VCF that was a VCP that has not yet been reconfigured. If a device is removed from a VCF, the interface that was interconnected to the removed device remains configured as a VCP until it is reconfigured into a network port.
Any interface that has been configured into a VCP using the
request virtual-chassis vc-port setcommand.
Automatic VCP conversion does not work in nonprovisioned VCFs.
Automatic VCP conversion does not automatically convert a VCP
interface into a network interface when a device is removed from a
VCF. If automatic VCP conversion has converted an interface into a
VCP (on either side of the VCP link) and you want the interface to
function as a network interface, you must manually delete the VCP
setting using the request virtual-chassis vc-port delete command.
VCF Configuration Options
You can configure a VCF using autoprovisioned, preprovisioned, or nonprovisioned configuration.
Autoprovisioned configuration allows you to plug and play leaf devices into a VCF after completing a minimal initial configuration procedure.
In a preprovisioned configuration, you deterministically control the devices in your VCF by associating each device’s serial number to a member ID and role.
Nonprovisioned configuration is possible, but not recommended for most VCF installations. Nonprovisioned configuration is a highly manual procedure that should only be performed by expert users.
See Understanding Virtual Chassis Fabric Configuration for additional information on the VCF configuration options.
Fabric Mode
Devices must be configured in fabric mode to properly participate as a member of a VCF. As a best practice, you should configure a device into fabric mode before interconnecting it into the VCF.
Devices are not in fabric mode by default. A device that is in a Virtual Chassis or a standalone device that is not part of a VCF should never be configured into fabric mode.
In autoprovisioned or preprovisioned VCF configurations, you can manually configure a spine device into fabric mode first and then interconnect it into a VCF. Alternatively, a spine device that has been zeroized or has the factory default configuration is automatically configured into fabric mode when it is interconnected into a VCF, during the automatic VCP conversion process. In either case, the process of configuring the device into fabric mode requires a device reboot as the final step for the spine device to join the VCF. If configured into fabric mode automatically, the VCF reboots the device automatically; if configured into fabric mode manually, you must manually reboot the device.
In an autoprovisioned or preprovisioned VCF, when adding a leaf device that has been zeroized or has the factory default configuration, the VCF automatically sets fabric mode and reboots the leaf device when it is interconnected into the VCF, during the automatic VCP conversion process. You can avoid the downtime that accompanies the automatic reboot by setting the device into fabric mode and rebooting the leaf device before interconnecting it into the VCF.
For best results, we strongly recommend manually configuring a spine or leaf device into fabric mode and then manually rebooting the device before interconnecting it into the VCF, rather than having the reboot happen automatically, which can be perceived as an unexpected action during VCF configuration and operation.
Mixed Mode
A mixed VCF is a VCF that includes two or more types of member switches in supported combinations that run different software images. In a mixed VCF, you must configure all devices in the VCF into mixed mode, and the switch must be rebooted after changing the mode for the change to take effect.
A VCF can be based on either QFX5110 or QFX5100 switches as the spine members, but only a QFX5100 VCF can be a mixed VCF, when the VCF contains QFX5100 spine members and also includes EX4300, QFX3500, QFX3600, or QFX5100 switches as leaf members. A QFX5110 VCF, which must have QFX5110-32Q spine members and can have any combination of QFX5100 and QFX5110 switches as leaf members, is always considered a non-mixed VCF; both types of switches run the same software image when interconnected into a VCF, and you do not need to configure the members into mixed mode. See Understanding Mixed EX Series and QFX Series Virtual Chassis for more details on which switches can be combined into a mixed VCF.
The optimal QFX5110 VCF topology is to use QFX5110 switches only, and the optimal QFX5100 VCF topology is to use a non-mixed QFX5100 with QFX5100 switches only. In each of these topologies, a VCF composed entirely of the base VCF devices supports the largest breadth of features at the highest scalability while also supporting the highest number of high-speed interfaces.
Devices are not configured into mixed mode by default. A device that is not part of a Virtual Chassis or a VCF with other devices should never be configured into mixed mode.
Virtual Management Ethernet Interface
VCF configuration can be managed remotely using a global management interface called the virtual management Ethernet (VME) interface. The VME interface is a logical interface representing all of the out-of-band management ports on the member devices. When you connect to the VCF using the VME interface’s IP address, the connection is always redirected to the device acting in the primary Routing Engine role.
A VME interface should always be used to configure a VCF. The VME interface is not tied to a device, so it can always be used to log in to the VCF even after the primary Routing Engine changes.
We strongly recommend cabling the management port on all devices acting as Routing Engines to the network to ensure that you always have a direct connection to the primary Routing Engine through the VME interface, regardless of which device assumes the primary Routing Engine role. The management ports on leaf devices can also be used by the VME interface to access the VCF, so you can also cable leaf device management ports to the network, if desired.
Virtual Chassis Fabric Port Link Aggregation Group Bundles
You can increase the bandwidth on links configured as VCPs within a VCF between two devices by configuring multiple same-speed links between two devices into VCPs. If, for instance, you configure two 40-Gbps QSFP+ links that are connecting the same devices in a VCF into VCPs , the two VCP links form one LAG bundle with two member links and 80-Gbps of total available bandwidth.
A VCP LAG bundle provides more bandwidth than a single VCP link can provide. A VCP LAG bundle also improves performance by load-sharing traffic across links within the bundle, and provides redundancy because traffic can be forwarded across another member link in the VCP LAG bundle when one VCP link fails.
VCP LAG bundling occurs automatically when same-speed VCP links are configured between two devices. No user configuration is required. VCP LAG bundling works only on same-speed VCP links, so, for example, 10-Gbps and 40-Gbps links cannot be in the same VCP LAG bundle.
Virtual Chassis Fabric License Requirements
A feature license is required to configure a VCF. The VCF feature license is an independent feature license; the enhanced feature licenses (EFLs) or advanced feature licenses (AFLs) that must be purchased to enable some features on some Juniper switches cannot be purchased to enable VCF.
For a VCF deployment, two license keys are recommended for redundancy—one for the device in the primary Routing Engine role and the other for the device in the backup Routing Engine role.
Feature licenses are also required to configure advanced features on a Virtual Chassis Fabric. For a Virtual Chassis Fabric deployment, two license keys are recommended for redundancy—one for the device in the primary Routing Engine role and the other for the device in the backup Routing Engine role. See Software Features That Require Licenses on the QFX Series.
To purchase feature licenses for VCF, contact your Juniper Networks
sales representative (https://www.juniper.net/us/en/contact-us/sales-offices). The Juniper sales representative will provide you with the feature
license files and license keys. You will be asked to supply the chassis
serial number of your switch; you can obtain the serial number by
running the show virtual-chassis command.
Hardware Requirements for a Virtual Chassis Fabric
A VCF can contain up to four devices configured as spines and up to twenty total devices. A VCF should contain a minimum of four members—two spine devices and at least two leaf devices interconnected in a spine-and-leaf topology.
For topologies with three or fewer members, use a Virtual Chassis configuration instead.
Spine devices in a mixed QFX5100 VCF must be QFX5100 switches, with any combination of QFX5100, QFX3600, QFX3500, or EX4300 switches as leaf devices.
We recommend using the following QFX5100 switches as spine devices:
QFX5100-24Q switches, in deployments where devices are connecting to the VCF using the 10-Gbps Ethernet interfaces on the leaf devices, or using a mix of 10-Gbps and 1-Gbps Ethernet interfaces on the leaf devices.
QFX5100-96S or QFX5100-48S, in deployments where devices are connecting to the VCF using 1-Gbps Ethernet interfaces only on the leaf devices.
QFX5100-48T switches are not supported as spine devices.
Spine devices in a QFX5110 VCF must be QFX5110-32Q switches. Leaf devices in a QFX5110 VCF can be any combination of supported QFX5100 switches and QFX5110 switches.
The following QFX5110 switches are supported as leaf devices in a QFX5110 VCF:
QFX5110-32Q
QFX5110-48S
The following QFX5100 switches are supported as leaf devices in a QFX5110 VCF:
QFX5100-24Q
QFX5100-48S
QFX5100-48T
Starting in Junos OS Release 17.3R2, QFX5100-48T switches can be included in a QFX5110 VCF.
QFX5100-96S
Software Requirements in a Virtual Chassis Fabric
All devices in a VCF must be running the same version of Junos OS software that supports VCF.
The devices in the VCF must be using the version of software for standalone switches.
The flex software bundle is supported on non-mixed QFX5100 VCFs using QFX5100 member switches only. You cannot use the flex software bundle in mixed VCFs. The flex software bundle is the software that includes “jinstall-qfx-5-flex” text in the filename when it is downloaded from the Software Center.
A QFX5110 VCF can only be set up using QFX5110 and QFX5100 switches that are running the same Junos OS image, which must be an image that includes “-qfx-5e-” in the software package filename when the Junos OS package is downloaded from the Software Center.
QFX5100 switches running a Junos OS image that includes “-qfx-5-” in the software package filename must be upgraded to a package filename that includes “-qfx-5e-” before being added to a QFX5110 VCF. See Upgrading a QFX5100 Switch with a USB Device to Join a QFX5110 Virtual Chassis or Virtual Chassis Fabric.
For any VCF, we recommend updating a device to the Junos OS release running on the VCF before interconnecting it into the VCF. For additional information on VCF software upgrades, see Understanding Software Upgrades in a Virtual Chassis Fabric.