Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
 

Campus Fabric Core Distribution High-Level Architecture

The campus fabric, with an EVPN-VXLAN architecture, decouples the overlay network from the underlay network. This approach addresses the needs of the modern Enterprise network by allowing network administrators to create logical L2 networks across one or more L3 networks. By configuring different routing instances, you can enforce the separation of virtual networks because each routing instance has its own separate routing and switching table.

The Mist UI workflow makes it easy to create campus fabrics.

Diagram Description automatically generated

Campus Fabric Core Distribution ERB Components

This configuration example uses the following devices:

  • Two EX9204 switches as core devices, software version: Junos OS Release 21.4R1.12 or later
  • Two QFX5120 switches as distribution devices, software version: Junos OS Release 21.4R1.12 or later
  • Two access layer EX4400 Switches, software version: Junos OS Release 22.1R1.10 or later
  • One SRX345 WAN router, software version: Junos OS Release 20.2R3-S2.5 or later
  • Juniper Access Points
  • Two Linux desktops that act as wired clients
Note:

Juniper’s recommended software version for Campus Fabric IP Core Distributed ERB is available under the EVPN/VXLAN ERB section at: https://supportportal.juniper.net/s/article/Junos-Software-Versions-Suggested-Releases-to-Consider-and-Evaluate?language=en_US .

Figure 1: Topology Diagram Description automatically generated

Juniper Mist Wired Assurance

Wired Assurance, through the Mist UI, can be used to centrally manage all Juniper switches. Juniper Mist Wired Assurance gives you full visibility on the devices that comprise your network’s access layer. The Juniper Mist portal provides a user interface to access your architecture through the AI-driven cloud services with your Juniper Mist account. You can monitor, measure, and get alerts on key compliance metrics on the wired network. This includes switch version and Power Over Ethernet (PoE) compliance, switch-AP affinity, and Virtual LAN (VLAN) insights.

Juniper Switch Onboarding to the Mist Cloud:

https://www.juniper.net/documentation/us/en/software/nce/nce-214-midsize-branch-mist-pwp/topics/topic-map/nce-214-midsize-ranch-mist-example_part2.html

Wired Assurance, through the Mist UI, is used to build a Campus Fabric Core Distribution ERB from ground up. This includes the following:

  • Assignment of p2p links between the core and distribution layers.
  • Assignment of unique BGP AS numbers per device participating in the underlay and overlay.
  • Creation of Virtual Routing and Forwarding (VRF) instances allow you to logically segment traffic. This also includes the assignment of new or existing VLANs to each representative VRF.
  • IP addressing of each L3 gateway Integrated Routing and Bridging (IRB) assigned to the distribution layer.
  • IP addressing of each lo0.0 loopback.
  • Configuration of routing policies for underlay and overlay connectivity.
  • Optimized Maximum Transmission Unit (MTU) settings for p2p underlay, L3 IRB, and ESI-LAG bundles.
  • Downloadable connection table (.csv format) that can be used by those involved in the physical buildout of the campus fabric.
  • Graphical interface depicting all devices with BGP peering and physical link status.

For more information on Juniper Mist Wired Assurance, see: https://www.mist.com/documentation/category/wired-assurance/

Juniper Mist Wired Assurance Switches

You must validate that each device participating in the campus fabric has been adopted or claimed and assigned to a site. The switches are named for respective layers in the fabric to facilitate building and operating the fabric.

Figure 2: Switch Inventory Graphical user interface, table Description automatically generated

Templates Overview

A key feature of switch management through the Juniper Mist cloud is to use templates and a hierarchical model to group the switches and make bulk updates. Templates provide uniformity and convenience, while the hierarchy (Site and Switch) provides both scale and granularity.

Templates and the hierarchical model means that you can create a template configuration and then all the devices in each group inherit the template settings. When a conflict occurs, for example, when there are settings at both the Site and Organizational levels that apply to the same device, the narrower settings (in this case, Site) override the broader settings defined at the Organization level.

Individual switches, at the bottom of the hierarchy, can inherit all or part of the configuration defined at the Organization level, and again at the Site level. Of course, individual switches can also have their own unique configurations.

You can include individual Command Line Interface (CLI) commands at any level of the hierarchy, which are then appended to all the switches in that group on an “AND” basis– that is, individual CLI settings are appended to the existing configuration (existing setting might be replaced or appended).

Note:

If you run CLI commands for items not native to the Mist UI, this configuration data is applied last; overwriting existing configuration data within the same stanza. You can access the CLI Command option from the Switch Template or individual Switch configuration.

Graphical user interface, text, application, chat or text message Description automatically generated

Under Organization and Switch Templates, we use the following template.

Background pattern Description automatically generated

Topology

Wired Assurance provides the template for LAN and Loopback IP addressing for each core and distribution device once the device’s management IP address is reachable. Each device is provisioned with a /32 loopback address and /31 point-to-point interfaces that interconnect core and distribution devices within the Campus Fabric Core Distribution. The devices such as the access layer of switches connect to the distribution layer using standard LAG; while the distribution uses ESI-LAG in a multihoming, load balancing manner.

The WAN router can be provisioned via Mist UI but is separate from the campus fabric workflow. The WAN router has a southbound lag configured to connect to the ESI-LAG on the core switches. WAN routers can be standalone or built as a high availability cluster. In this document, a single SRX Series Firewalls is used as the WAN router.

Create the Campus Fabric

From the Organization option on the left-hand section of the Mist UI, select Wired Campus Fabric.

Graphical user interface, application Description automatically generated

Mist provides the option of deploying a campus fabric at the Organizational or Site level noted on the upper left-hand Campus Fabric menu shown below. For example, if you are building a Campus wide architecture with multiple buildings, each building housing distribution and access switches, you can consider building an Organizational level Campus Fabric. This campus fabric ties each of the sites together forming a holistic campus fabric. Otherwise, the Site build with a single set of core, distribution, and access switches is sufficient.

Campus Fabric Org Build

Graphical user interface, text, application Description automatically generated

Campus Fabric Site Build

Graphical user interface, text, application Description automatically generated

Note:

Campus Fabric Site deployment is the focus of this document.

Choose the Campus Fabric Topology

Select Campus Fabric Core-Distribution:

Graphical user interface, application Description automatically generated

Mist provides a section to name the Campus Fabric Core Distribution ERB:

  • Configuration—Provide a name in accordance with company standards
  • Topology Sub-type:
  • CRB
  • ERB
Note:

ERB uses anycast addressing which provides a shared IP addresses among all distribution layer devices participating in the L3 IRB. Deployments that require a routing protocol on the L3 IRB must use CRB with virtual-gateway addressing.NOTE: You must choose CRB if most of their traffic patterns are north-south while ERB should be selected if east-west traffic patterns exist as well as IP Multicast.

Topology Settings

  • BGP Local AS: represents the starting point of private BGP AS numbers that are automatically allocated per device. You can use whatever private BGP AS number range suits your deployment, routing policy is provisioned by Mist to ensure the AS numbers are never advertised outside of the fabric.
  • Loopback prefix: represents the range of IP addresses associated with each device’s loopback address. You can use whatever range suits your deployment. VXAN tunneling using a VTEP is associated with this address.
  • Subnet: represents the range of IP addresses used for point-to-point links between devices. You can use whatever range suits your deployment. Mist breaks this subnet into /31 subnet addressing per link. This number can be modified to suit the specific deployment scale. For example, /24 provides up to 128 p2p /31 subnets.
Note:

We recommend default settings for all options unless it conflicts with other networks attached to the campus fabric. The point-point links between core and distribution layers use /31 addressing to conserve addresses.

Select Campus Fabric Nodes

Select devices to participate in each layer of the Campus Fabric Core Distribution ERB. We recommend that you validate each device’s presence in the site switch inventory prior to the creation of the campus fabric.

The next step is to assign the switches to the layers. Since the switches were named relative to target layer functionality, they can be quickly assigned to their roles.

Services Block Router is where the campus fabric interconnects external devices such as firewalls, routers, or critical devices. For example, DHCP and Radius servers. Devices to which external services connect to the campus fabric are known as Border Leafs. If you want to connect these services/devices to the Campus Fabric Core Distribution ERB in a separate device or pair of devices, clear the Use Core as border option and select the Select Switches option to choose the devices.

Graphical user interface, application, table Description automatically generated

Note:

Placing the Services Block Router on a dedicated pair of switches (or single switch) alleviates the encapsulation and de-capsulation of VXLAN headers from the core Layer. If you want to combine this capability within the core devices, you must select the User Core as border option.

Once all layers have selected the appropriate devices, you must provide a loopback IP address for each device. This loopback is associated with a logical construct called a VTEP; used to source the VXLAN Tunnel. Campus Fabric Core Distribution ERB has VTEPs for VXLAN tunneling on the distribution switches and the core switches when enabling the Core Border option.

The loopback addresses and router-ids should be in the same address space. The router-id of the loopback can be customized to differentiate between core, distribution, and access. This can help identify devices if you are troubleshooting or following next hops. The loopback is also used as the router-id and is used for overlay eBGP peering and VXLAN tunnel termination.

Graphical user interface, application Description automatically generated

Note:

The loopback address and router-id should be in the same subnet as provided by Mist.

The loopback prefix is used for import/export policies. The subnet addresses are used for point-to-point links throughout the Fabric. Mist automatically creates policies that import, and export loopback addresses used within the campus fabric. The selection of fabric type displays with default settings, which can be adapted as required.

A picture containing chart Description automatically generated

Configure Networks

Enter Network information such as VLANs and VRF (routing instances for traffic isolation purposes) options. VLANs are mapped to VNIs and can optionally be mapped to VRFs to provide you a way to logically separate traffic patterns such as IoT devices from Corp IT.

Graphical user interface, application Description automatically generated

Networks

VLANs can be created or imported under this section including the IP subnet and Default GW per each VLAN.

The Shared Elements section of the campus-fabric template includes the Networks section mentioned above where VLANs are created.

Graphical user interface, text, application, email Description automatically generated

Back to the campus fabric build, select the existing template includes L2 VLAN information. All VLAN and IP information is inherited from the template.

Graphical user interface, text, application, email Description automatically generated Graphical user interface, text, application, email Description automatically generated

Other IP Configuration

Mist Wired Assurance provides automatic IP addressing Integrated Routing and Bridging (IRBs) for each of the VLANs. Port Profiles and Port Configuration then associate the VLAN with specified ports. In this case, we selected Campus Fabric ERB at the onset of the campus fabric build.

Graphical user interface, text, application, email Description automatically generated

This option uses anycast addressing for all devices participating in the L3 subnet. In this case, Dist1 and Dist2 switches are configured with the same IP address for each L3 subnet.

More on Anycast Gateways can be found here: https://www.juniper.net/documentation/us/en/software/junos/evpn-vxlan/topics/concept/evpn-mclag-irb-gateway-anycast-address.html

Graphical user interface, application, Teams Description automatically generated Graphical user interface, application, Teams Description automatically generated

By default, all VLANs are placed in the default VRF. The VRF option allows you to group common VLANs into the same VRF or separate VRFs depending on traffic isolation requirements. This example includes three VRFs or routing instances: corp-it | developers | guest-wifi. Here, you build the first corp-it VRF and select the pre-defined vlan 1099.

Graphical user interface, text, application, email Description automatically generated Graphical user interface, text, application, email Description automatically generated

By default, inter-VRF communications are not supported within the campus fabric. If inter-VRF communications is required, each VRF can include extra routes such as a Default Route that instructs the campus fabric to use an external router or firewall for further security inspection or routing capabilities. In this example, all traffic is trunked over the ESI-LAG and the Juniper SRX Series Firewalls handles inter-VRF routing. See Figure 1 .

Notice the SRX Series Firewalls participates in the VLANs defined within the campus fabric and is the gateway of last resort for all traffic leaving the subnet. Select Add Extra Routes to inform Mist to forward all traffic leaving 10.99.99.0/24 to use the next hop of the Juniper SRX Series Firewalls: 10.99.99.254.

Application Description automatically generated with medium confidence

Create two additional VRFs:

  • developers using vlan 1088 with 0.0.0.0/0 utilizing 10.88.88.254
  • guest-wifi using vlan 1033 with 0.0.0.0/0 utilizing 10.33.33.254

Graphical user interface, application Description automatically generated Graphical user interface, text, application, email Description automatically generated

The final step in the Configure Networks section is the Distribution/Access Port Configuration.

Graphical user interface, text, application, email Description automatically generated

The section configures the active-active ESI-LAG trunks between distribution and access switches. Here, we name the port configuration and include VLANs associated with this configuration. The advanced tab provides additional configuration options:

Graphical user interface, application Description automatically generated

Note:

We recommend default settings unless specific requirements are needed.

Now that all VLANs are configured and assigned to each VRF, and the Distribution/Access ESI-LAGs are built, click Continue button at the upper-right section of the Mist UI to move to the next step.

Configure Campus Fabric Ports

The final step is the selection of physical ports among core, distribution, and access switches.

Graphical user interface, text, application Description automatically generated

Note:

We recommend that you have the output of the show lldp neighbors command from each switch(assuming LLDP is enabled before the switches were selected). This output provides a source of truth for which ports should be selected during at each layer.

Core Switches

Core1:

Starting with Core1, select xe-1/0/5 and xe-1/0/6 terminating on Distribution Switches 1 and 2 respectively.

Graphical user interface, application Description automatically generated

Graphical user interface, application, Teams Description automatically generated

Core2:

On Core2, select xe-1/0/4 and xe-1/0/5 terminating on Distribution Switches 1 and 2 respectively.

Graphical user interface, text, application Description automatically generated

Graphical user interface, application Description automatically generated

Distribution Switches

Now moving on to the Distribution Switches, you notice two interconnect options exist:

  • Link to Core
  • Link to Access

Dist1:

Select Link to Core and choose xe-0/0/5 and xe-0/0/4 terminating on Core Switches 1 and 2 respectively.

Application Description automatically generated with low confidence

A picture containing table Description automatically generated

Select Link to Access and choose ge-0/0/36 and ge-0/0/37 terminating on Access Switches 1 and 2 respectively.

Graphical user interface, application Description automatically generated

Graphical user interface, application Description automatically generated

Next, select the following interconnects off Dist2:

  • Link to Core
  • xe-0/0/6 – Core1
  • xe-0/0/5 – Core2
  • Link to Access
  • ge-0/0/36 – Access2
  • ge-0/0/37 – Access1

Access Switches

You only need to know which interfaces are used to interconnect with the Distribution switch but do not need to know the specific mapping. The system bundles all interfaces into a single Ethernet bundle through the AE Index option. This greatly simplifies the physical port build for each access switch.

Access1/2:

Select both uplinks and interface speed, while allowing Mist to define each AE index. In this case, uplinks ge-0/0/36/37 are selected as Links to Distribution on both access switches and AE Index 0/1 (system default numbering) on Access1/2 respectively.

Graphical user interface, application Description automatically generated

Graphical user interface Description automatically generated with medium confidence

Once you have completed selecting all requisite port combinations, select the Continue button at the upper right-hand corner of the Mist UI.

Campus Fabric Configuration Confirmation

This last section provides the ability to confirm each device’s configuration as shown below:

Graphical user interface, application Description automatically generated

Once you have completed verification, select the Apply Changes option at the upper right-hand corner of the Mist UI.

You must complete the second stage confirmation to create the fabric.

Mist displays the following banner including the estimated time for the campus fabric to be built. The process includes the following:

  • Mist builds the point-to-point interfaces between distribution and core devices with IP addresses chosen from the range presented at the onset of the build.
  • Each device is configured with a loopback address from the range presented at the onset of the build.
  • eBGP is provisioned at each device with unique BGP autonomous system numbers. The primary goal of the underlay is to leverage ECMP for load balancing traffic on a per packet level for device loopback reachability. The primary goal of the eBGP overlay is support of customer traffic using EVPN-VXLAN.
  • IP addressing of each L3 gateway IRB located on Dist1 and Dist2.
  • IP addressing of each lo0.0 loopback.
  • Configuration of routing policies for underlay and overlay connectivity.
  • Optimized MTU settings for p2p underlay, L3 IRB, and ESI-LAG bundles.
  • VXLAN to VLAN mapping using VNI addresses that are automatically assigned
  • VRF creation of corp-it, developers, and guest-wifi and VLAN associated with each VRF.
  • VXLAN tunneling creation between distribution devices and distribution-core devices (in support of the northbound SRX Series Firewalls that is configured in subsequent steps).
  • Downloadable connection table (.csv format) that can be used by those involved in the physical buildout of the campus fabric.
  • Graphical interface depicting all devices with BGP peering and physical link status.

Graphical user interface, text, application, email Description automatically generated

Once you click Close Campus Fabric Configuration, you can view a summary of the newly created Campus Fabric Core Distribution ERB.

Graphical user interface, application Description automatically generated

With Juniper Mist Wired Assurance, you can download a connection table (.csv format) representing the physical layout of the campus fabric. This can be used to validate all switch interconnects for those participating in the physical campus fabric build. Once the campus fabric is built or in the process of being built, you can download the connection table.

Graphical user interface, application Description automatically generated

Connection Table spreadsheet:

Graphical user interface Description automatically generated

Apply VLANs to Access Ports

As previously discussed, Mist provides the ability to templatize well known services such as Radius, NTP, DNS, and so on that can be used across all devices within a Site. These templates can also include VLANs and port profiles that can be targeted at each device within a Site. The last step before verification is to associate VLANs with the requisite ports on each access switch.

In this case, Desktop1/2 are associated with different ports on each access switch which requires the configuration to be applied to Access1/2 respectively. See Figure 1 .

It is also noteworthy that Mist Access Points connect to the same port on Access1/2 allowing the Switch Template to be customized with this configuration. For example, the following found under the Switch Template option is customized to associate each switch with its role: Core, Distribution, and Access. Further, all access switches (defined by EX4400 Switch as an example) associated the AP port profile with ge-0/0/16 without needing to configure each independent switch.

A screenshot of a computer Description automatically generated

Using Access1 as an example, we apply vlan1099 to port ge-0/0/11 under the Port Configuration section on Access1. In this example, vlan1099 (corp-it), vlan1088 (developers), and vlan1033 (guest-wifi) are defined in the Switch Template. Here, vlan1099 is selected under the configuration profile.

A screenshot of a computer Description automatically generated with medium confidence

The Switch Template definition for vlan1099 is shown below, representing attributes associated with VLANs such as dot1x authentication, QoS, and Power over Ethernet (PoE). Vlan1088 and vlan1033 need to be configured in a similar fashion.

A screenshot of a phone Description automatically generated with medium confidence