Use Case and Reference Architecture
The Collapsed Fabric with Access Switches and Juniper Apstra JVDE topology is ERB-based and created using Juniper Apstra.
This Collapsed Fabric with Access Switches and Juniper Apstra JVDE is the Collapsed Data Center Fabric with Juniper Astra JVD with the addition of an access switch layer. The Collapsed Data Center Fabric with Juniper Astra JVD is a two-switch network fabric designed for small network deployments. Switches in a collapsed fabric perform the roles of spine, leaf, and border leaf switches. This topology allows for high availability network deployments with a minimum of switch hardware; however, resource constraints limit the real-world expandability of this design. The addition of access switches provides the ability to add additional ports, especially 1 gigabit ethernet ports.
For the purposes of this JVDE, only the EX4400-48MP switch platform was tested and validated for use as an access switch. While it is technically possible to use any number of different switch platforms in an access switch role, the choice of restricting validation to the EX4400-48MP is deliberate. The EX4400-48MP is a budget-conscious choice for adding 1 gigabit ethernet ports. For customers seeking to add additional 10 gigabit or larger ports, Juniper Networks recommends using a collapsed fabric with switches that have a higher port count or using a 3-stage design such as 3-Stage Data Center Design with Juniper Apstra JVD instead.
Switches ranging from the QFX5120-48Y to the QFX 5700 are validated for use in the collapsed spine layer within the collapsed fabric data center JVDs. These switches provide connectivity options ranging from 1 gigabit SFP ports all the way to 400 gigabit QSFP56-DD ports.
This JVDE expands the topology in the Collapsed Data Center Fabric with Juniper Astra JVD by providing an access switch layer that allows you to add a modest number of 1 gigabit or 2.5 gigabit ports to your topology. Juniper recommends the 3-Stage Data Center Design with Juniper Apstra JVD for customers that require more 10 gigabit or higher-speed ports than can be provided by this validated collapsed fabric design. The 3-Stage Data Center Design with Juniper Apstra JVD is validated to handle the higher resource demands that come with an expanded number of high-throughput ports.
The Collapsed Fabric with Access Switches and Juniper Apstra JVDE uses EVPN for the control plane, VXLAN for the data plane, and eBGP for both underlay and overlay signaling. This means leaf switches can discover all the “remote” hosts without flooding the overlay with ARP/ND requests. Because the switches in the Collapsed Fabric with Access Switches and Juniper Apstra JVDE serve all fabric roles, including the border leaf role, the collapsed fabric switches are tested to serve as anycast gateways as well as gateways to external networks. These anycast and external network gateways require Data Center Interconnect (DCI) features.
This JVDE demonstrates the addition of one high-availability pair of switches in the access switch role, although multiple pairs of access switches can be connected to the collapsed fabric. Each pair of access switches operates as an independent layer 2 EVPN-VXLAN collapsed fabric connected to the Layer 3 EVPN-VXLAN collapsed fabric using an Ethernet Segment Identifier Link Aggregation Group (ESI LAG). For more details on how Juniper Apstra handles access switches, see the Access Switchsection of the Juniper Apstra User Guide.
In this design two generic systems are created to represent servers. Another generic system is created to represent an external router which is outside of the collapsed fabric. Neither the servers nor the router are directly managed by Apstra. Apstra, however, has to be aware of the servers and the router to assign the right virtual networks, routing groups, and other related configuration to the appropriate switch interfaces. This design connects one generic server to the collapsed fabric layer and another generic server to the access switch layer. These server connections to switches in multiple layers connections are configured intentionally to demonstrate that devices can be connected to either layer as needed.
In the Apstra collapsed fabric design used in this JVDE, the collapsed spine switches operate with a layer 3 VXLAN while access switches operate with a layer 2 VXLAN. As a result, the collapsed spine switches have IRB interfaces while the access switches do not have IRB interfaces. Therefore, all inter-VLAN routing happens at the collapsed spine layer and no inter-VLAN routing happens at the access switch layer. Individual access switch ports can be configured in either trunk or access mode, allowing devices connected to the access switches to connect to different VLANs; however, routing between those VLANs will always happen on the collapsed spine switches and not on the access switches.
Prerequisites
This JVD assumes that the Apstra server virtual machine (VM) and Apstra ZTP server VM are already deployed with the appropriate version, and you know how to access the console of these VMs. To implement the topology provided in this JVDE, the virtual network of both VMs must be on the same subnet as the physical management network interface of the switches. For additional information on deploying the Apstra server and ZTP VMs, see the Juniper Apstra Installation and Upgrade Guide .
This JVD assumes that you have a basic knowledge of Apstra terminology and processes and that you are familiar with provisioning a data center reference architecture with a blueprint. For more information on Juniper Apstra, see the Juniper Apstra User Guide on the Juniper Apstra Techlibrary homepage.
Juniper Hardware and Software Components
For this JVDE, the Juniper products and software versions are listed below. The listed architecture is the recommended base representation for the validated design. As part of a complete solutions suite, we routinely swap hardware devices with other models during iterative use case testing. Each platform also goes through the same tests for each specified version of Junos OS.
Juniper Hardware Components
The following switches are tested and validated to work with this Collapsed Fabric with Access Switches and Juniper Apstra JVDE, and are the same ones included in its parent JVD.
Collapsed Spine role:
- QFX5130-32CD
- QFX5120-48Y
- QFX5700
- ACX7100-48L
- PTX10001-36MR
- Access Switch Role:
- EX4400-48MP
For the purposes of this document, the following switches are used in the configuration walkthrough:
|
Juniper Hardware |
|||
|---|---|---|---|
| Platform | Role | Hostname | Junos OS Release |
|
QFX5120-48Y |
Collapsed Spine |
DC3-LEAF-1 and DC3-LEAF-2 |
22.2R3-S2 |
|
EX4400-48MP |
Access Switch |
DC3-ACCESS-1-1 and DC3-ACCESS-1-2 |
22.2R3-S2 |
|
Juniper Software |
|
|---|---|
|
Product |
Version |
|
QFX5120-48Y |
Collapsed Spine |
|
EX4400-48MP |
Access Switch |
Juniper Apstra Overview
Juniper Apstra is a multivendor, intent-based network software (IBNS) solution that orchestrates data center deployments and manages small to large-scale data centers through Day-0 to Day-2 operations. It is an ideal tool for building data centers for AI clusters, providing invaluable Day-2 insights through monitoring and telemetry services.
Provisioning a data center fabric through Juniper Apstra is a modular function that leverages various building blocks to instantiate a fabric. These basic building blocks are as follows:
- A logical device is a logical representation of a switch's port density, speed, and possible breakout combinations. Since this is a logical representation, any hardware specifics are abstracted.
- Device profiles provide hardware specifications of a switch that describe the hardware (such as CPU, RAM, type of ASIC, and so on) and port organization. Juniper Apstra has several pre-defined device profiles that exist for common data center switches from different vendors.
- Interface maps bind together a logical device and a device profile, generating a port schema that is applied to the specific hardware and network operating system, which is represented by the device profile. By default, Juniper Apstra provides several pre-defined interface maps with the ability to create user-defined interface maps as needed.
- Rack types define logical racks in Juniper Apstra, the same way a physical rack in a data center is constructed. However, in Juniper Apstra, this is an abstracted view of it, with links to logical devices that are used as leaf switches, the kind and number of systems connected to each leaf, any redundancy requirements (such as MLAG or ESI LAG), and how many links, per spine, for each leaf.
- Templates take one or more rack types as inputs and define the overall schema/design of the fabric. You can choose between a 3-stage Clos fabric, a 5-stage Clos fabric, or a collapsed spine design. You can also choose to build an IP fabric (with static VXLAN endpoints, if needed) or a BGP EVPN-based fabric (with BGP EVPN as the control plane).
- The blueprint instantiates the fabric, taking a template as its only input. A blueprint requires additional user input to bring the fabric to life, including resources such as IP pools, ASN pools, and interface maps. Additional virtual configuration is done, such as defining new virtual networks (VLANs/VNIs), building new VRFs, defining connectivity to systems such as hosts or WAN devices, and so on.