Appendix: Test Case Example Information

Virtual Test Lab

The examples in this appendix to the JVDE are evaluated in a virtual test lab consisting of a vJunos-switch , a vMX router, and vSRX V3.0 firewalls. We did not create a pair of virtual service block switches but ensured that both types of WAN routers (router or firewall) were available as redundant pairs. This is not the same lab used for testing that required the use physical hardware. We use this as an example and something you can potentially build yourself with environments such as EVE-NG to build your own labs to test the configuration examples. The fabric, in this case, was configured as IP Clos.

L2 Exit with Stretched VLAN

Note:

Previous versions of this JVD-E included an example configuration for L2 Exit using stretched VLANs. However, this led to significant issues, as users often overlooked the fact that it was never intended for production use. To avoid further confusion, we have decided to remove the example entirely. While it may still appear in older documentation, it remains an unsupported and non-production-grade solution.

L2 Exit with Transport VLAN

Note:

When doing any VLAN or VRF creation with campus fabric remember the following best practices:Create all VLANs in a switch template and then import them in the Campus Fabric dialogue. Creating the VLANs anywhere else in the Mist GUI ultimately leads to inconsistency which makes it hard to resolve issues.With the exception of the service block functions, do not create VRFs outside of the Campus Fabric dialogue.The transport VLAN method requires you to create VRFs manually on the service block function and add the transport VLAN and routes locally to the VRFs. Do not create the VRFs or routes in the Campus Fabric dialogue.We recommend that you create port profiles within switch templates so that any changes are in sync on all switches in the fabric.

When defining the transport VLANs in the switch template, do not set the subnet information. You configure this information later as Additional IP Subnet on each service block function. See Figure 1 , Figure 2 , and Figure 3 .

The following CLI configuration shows the exported version of the switch template used in the transport VLAN fabric. This allows you to review our setup when importing. As you can see, there is a minimum of two VLANs per VRF plus an additional transport VLAN per VRF.

Within the Campus Fabric Configuration dialogue, there is a section called Configure Networks. This is where you import your six access VLANs from the switch template. When finished, the configuration should be as shown in Figure 4 and the result in our case will look as shown below. Since the three transport VLANs are not part of the access layer, they are not defined in the service block function.

Next, you create 3 VRFs and attach two of the access networks to each VRF as shown in Figure 5 .

Next, go to each VRF and confirm that you only have access networks defined with no default route. You will define the transport VLANs and default routes later in the service block function. See Figure 6, Figure 7 , and Figure 8 .

Core1 and Core2 Switch Configuration

In the transport VLAN attach example, the service block function is virtual and co-located on the core switch. Therefore, you must configure the two core switches. The following pseudocode represents the configuration you must apply to the core1 and core2 switches:

The following four images display the Mist GUI configuration that results from the previous pseudocode starting with the additional IP configuration required to assign the local IP addresses to each transport VLAN.

Next, you define the Port Profile used for the uplinks. It is critical that you only include the transport VLAN in the Trunk Networks definition since only those VLANs are used and visible to the WAN router.

Next, you assign the port profiles to each uplink port.

Figure 15 shows the configuration of the first uplink to the first WAN router.

Figure 16 shows the configuration of the second uplink to the first WAN router.

Note:

You must ensure that the AE Indexes on each service block function are in sync with each other towards the same WAN router and that you define them each as ESI-LAG. You must also ensure that you don’t reuse an AE Index that is already defined elsewhere in the fabric service block.

Next you create and modify local VRFs. Remember this is an exception made only for the transport VLAN exit method. Usually, the fabric creates the VRFs automatically. In this case we must enable the Override Site/Template Settings checkbox in the VRF configuration. Figure 17 shows the required configuration in the Mist GUI.

Next you must perform the following three configurations in each of your three VRS instances:

• Enable Override Template Defined VRF Instance checkbox
• Add your transport VLAN to the pre-populated list of access VLANs
• Add a default route where the gateway IP address is the VRRP-VIP address of your WAN router.
• Figure 18 , Figure 19 , and Figure 20 show the override configurations for each of the three VRFs.

Now you must configure additional CLI to modify the transport VLANs to use VGA configuration to help avoid excess hair-pin routing of traffic within the fabric. In the switch configuration for each of your service block function switches, locate the CLI Configuration section in the Mist GUI. You must paste the required configuration into the field indicated in Figure 21.

The example CLI configuration for your core1 switch, is shown in the following code block. We have configured the static IP address as the virtual gateway IP address + 1 (10.99.1.2).

For your core2 switch, only the static IP addresses of the transport VLAN are changed to be the virtual gateway IP address + 2 (10.88.1.3).

Note:

Keep in mind that our test lab used virtual EX9214 switches as core switches. In most production environments you will not use an EX92xx switch. Therefore, you must uncomment the two lines that are commented out in the previous configuration snippet:

Note:

Service block for each transport VLAN used per VRF you must manually set the MAC address of the virtual gateway address used on the IRB interface. In our example, we used different MAC addresses per transport VLAN because it’s easier to debug.

Juniper MX as WAN Router

The following CLI snippet example contains the configuration of the interfaces, the VRRP gateway redundancy, and the static routes for the first WAN router. You may need to add default routes and interfaces to complete the configuration.

On the second WAN router, the notable configuration changes are the AE keys and indexes, and the static IP addresses.

You may wonder about those static routes in the 172.16.19x.0 range. Remember that IP Clos is an anycast fabric. As such, you must have th static routes to prepare for when the DHCP relay will use IP addresses in the fabric overlay. See Figure 22 for an example definition

The overlay Loopbacks IPs are assigned to each VRF on a switch as a /24 range. You can figure them out by looking at a fabric access switch as shown in Figure 23 . Hence, you must map them back like any other additional VLAN attached to the VRF to achieve the required reachability.

The following commands help to debug the connections on WAN router1.

The following commands help you to debug connections on WAN router2.

L3 Exit With eBGP Routing Protocol

Note:

When you create any VLAN or VRF creation with campus fabric remember the following best practices:

• Create all VLANs in a switch template and then import them into the Campus Fabric Dialogue. Creating the VLANs anywhere else in the Mist GUI ultimately leads to inconsistency which makes it hard to resolve issues.
• If needed, the fabric creates any required VRFs. Do not create VRFs manually elsewhere in the Mist GUI.
• We recommend that you create port profiles within switch templates so that any changes are in sync on all switches in a fabric.

Before you begin, you need a plan for:

• How to implement the routing protocol and route exchange
• How to configure the P2P links
• How to distribute the VLAN assignment that is indirectly used to identify the VRF

Even if the VRF already exists elsewhere in the fabric, such as on the access switch for IP Clos, the system will automatically re-create it on all service block functions when doing an L3 exit.

For each WAN router, you must reuse a VLAN name on a VRF to help the automatic creation of VRFs on the service block function. Keep in mind that when you define the local P2P links and reuse the VLAN, those definitions are purely local, so they do not conflict with the overlay VLAN definition. Additionally, you do not need to define special transport VLANs here. However, you can still use and define special transport VLANs for the P2P links if that better suits your needs.

When defining the P2P links, you must ensure that they are outside of the address range in use by the fabric. The default range used by the fabric for these links is 10.255.240.0/20. We recommend that you a /31 netmask. With that plan, you can use the even number IP addresses for the WAN router side and the odd IP addresses for the fabric side.

The system requires that you use a VLAN for each P2P link on a physical cable. This allows you to have multiple VRFs multiplexed on a single uplink cable. Remember the VLAN internally refers back to the VRF.

For eBGP you must also define your own private ASN for peering. By hardcoded default, the fabric uses 65000 ASN for the EVPN control plane and starts allocating configurable ASN at 65001. After that, it advances one digit for each node. Therefore, we recommend using ASN values below 65000 to avoid conflict with system assigned ASN. The QFX switch only allows 16 local ASN. Therefore, we recommend that you use a shared ASN among all VRFs. However, in our example, we decided to use a different ASN per WAN router.

Figure 24 shows how the two service block functions of the fabric would connect to the first WAN router.

Figure 25 shows how the two service block functions of the fabric would connect to the second WAN router. Notice that we now use the second block of VLANs from each VRF.

Table 1 displays the full configuration between the core1 and core2 switches, as service block function, and the two WAN routers. You can also see the ASN chosen for eBGP.

Table 1: MX WAN Router and Core Switch Configuration Summary for eBGP Exit

Switch	Switch AS	VRF	Core P2P IP	Core IF	WAN Router	WAN Router P2P IP	WAN Router AS	WAN Router IF	VLAN-ID
core1	64911	customera	10.255.224.1/31	ge-0/0/3.1091	wanrouter1	10.255.224.0/31	64901	ge-0/0/1.1091	1091
core1	64911	customerb	10.255.224.3/31	ge-0/0/3.1081	wanrouter1	10.255.224.2/31	64901	ge-0/0/1.1081	1081
core1	64911	devices	10.255.224.5/31	ge-0/0/3.1031	wanrouter1	10.255.224.4/31	64901	ge-0/0/1.1031	1031
core1	64911	customera	10.255.225.1/31	ge-0/0/4.1099	wanrouter2	10.255.225.0/31	64902	ge-0/0/1.1099	1099
core1	64911	customerb	10.255.225.3/31	ge-0/0/4.1088	wanrouter2	10.255.225.2/31	64902	ge-0/0/1.1088	1088
core1	64911	devices	10.255.225.5/31	ge-0/0/4.1033	wanrouter2	10.255.225.4/31	64902	ge-0/0/1.1033	1033
core2	64911	customera	10.255.226.1/31	ge-0/0/3.1091	wanrouter1	10.255.226.0/31	64901	ge-0/0/2.1091	1091
core2	64911	customerb	10.255.226.3/31	ge-0/0/3.1081	wanrouter1	10.255.226.2/31	64901	ge-0/0/2.1081	1081
core2	64911	devices	10.255.226.5/31	ge-0/0/3.1031	wanrouter1	10.255.226.4/31	64901	ge-0/0/2.1031	1031
core2	64911	customera	10.255.227.1/31	ge-0/0/4.1099	wanrouter2	10.255.227.0/31	64902	ge-0/0/2.1099	1099
core2	64911	customerb	10.255.227.3/31	ge-0/0/4.1088	wanrouter2	10.255.227.2/31	64902	ge-0/0/2.1088	1088
core2	64911	devices	10.255.227.5/31	ge-0/0/4.1033	wanrouter2	10.255.227.4/31	64902	ge-0/0/2.1033	1033

The code block below shows the exported version of the switch template used in this fabric. This allows you to review our setup when importing. As you can see, we have a minimum of two VLANs per VRF. Remember that the L3 exit model requires one VLAN per WAN router and VRF).

Within the Campus Fabric Configuration dialogue, there is a page called Configure Networks. This is where you import your six VLAN’s from the switch template. The resulting configuration is shown in the following figures.

Then you go to each VRF and delete all manual routes you may have. Make sure each VRF has a minimum of two VLAN’s attached as those are used to identify the VRF later.

Core1 Switch Configuration

In this example, the service block function is virtual and co-located on the core switch. Therefore, you must configure the two core switches. The following block of pseudocode describes what you need to configure on the core1 switch:

The following screenshots show the previous configuration translated into the Mist GUI. We start with the additional IP configuration. Notice that the VLAN IP addresses do not match the IP addresses that these VLANs had originally in the overlay. This is by design. You must always have the VLAN as a reference back to the VRF.

In the Port Configuration window, you must enable the L3-sub-interfaces and assign the first 3 sub-interfaces defined.

In the second Port Configuration window, towards the other WAN router, assign the second 3 sub-interfaces defined.

Next, you must enter the entire eBGP configuration with all six peers (three VRFs and two WAN routers. When finished, the overview page should be as shown in Figure 33 .

First, you define two routing policies, a summary of which is shown in the above table.

In our example, the export route policies contain the following terms:

• Term=fabric-all-no-hosts — Used to automatically export all routes of a fabric from all configured VRFs and their VLANs. We strip potential P2P links and single host routes. To achieve this, we use the prefix “0.0.0.0/0-30”. Using “0-30” in the prefix is a way of defining “orlonger” in the Junos OS.
• Term=overlaylo0 — Used to export the overlay loopback IP addresses (for DHCP relay usage) down to the host IP level as they are usually individually assigned to lo0.x. To achieve this, we use 172.16.192.0/24-32 as we here need to go down towards host level which we do not do in the above statement.

If desired, you can simplify the rule set by using a single term such as “0.0.0.0/0-32,” which exports all routes—including those for individual clients in the fabric, represented by their host IP addresses. This approach can also be useful for debugging, as examining the AS-Path information on the WAN router will reveal which EVPN fabric switch is announcing a given host IP.

The downside, however, is that this approach may result in a large number of routes being exchanged with the WAN router, whereas the default setting of “0.0.0.0/0-30” limits route sharing to prefixes associated only with the configured VLANs.

The import policy imports the default route from the WAN router.

Figure 35 shows the configuration of a single BGP peering entry with the required entries called out.

You must also define the WAN router as a BGP neighbor.

You may see a warning message as shown in Figure 36 . It is safe to ignore those.

• The messages mentioned above appear because we are reusing overlay VLANs—already assigned to VRFs—for underlay BGP peering with the WAN router. This can be avoided by using dedicated VLANs for BGP peering that are not part of the fabric's VRF configuration. To do so, a local VRF setup similar to the “L2 Exit with Transport VLAN” approach is required. This involves manually adding the BGP peering VLANs to each VRF, along with the overlay VLANs configured through the standard fabric configuration dialogue.
• While this method eliminates the false-positive error messages, it comes with a trade-off: every time a new VLAN is added to the fabric, you'll need to manually update each service block to include the new VLAN in the custom local VRF configuration.
• In contrast, the method used in this JVD-E avoids such error-prone reconfiguration. You simply add the new VLAN to the switch template and configure it through the fabric dialogue. The service block configuration remains unchanged and will automatically export the new VLAN's routes based on the export filter configuration shown above.

Core2 Switch Configuration

The following pseudocode represents the configuration you must apply to the core2 switch:

Aside from the P2P subnets and the BGP neighbours, the configuration on the core2 switch is the same as the configuration on the core1 switch.

Juniper MX as a WAN Router

The following several code blocks show the Junos OS CLI configuration of the P2P interfaces, the entire eBGP config with all BGP neighbours, and all import and export route policies for each WAN router. You may need to add default routes and interfaces to complete the configuration because those need to be signalled to the fabric but we don’t know how your device gets to know those.

CLI configuration for the first WAN router:

Configuration for the second WAN router:

You can use the following CLI commands to assist with debugging on WAN router1.

You can use the following CLI commands to assist with debugging on WAN router2.

Juniper SRX Series Firewall as WAN Router

The following example table and configurations show the differences between using an SRX Series Firewall in cluster mode and an MX router as the WAN router. On the fabric side, only the interface names of the SRX cluster change from the MX router configuration. Because the SRX Series Firewall runs in active/active cluster mode, there is only a single WAN router configuration and a single ASN to consider. That single configuration also includes cluster management and trust-zone management commands that are not present in a similar MX router-based configuration.

This SRX Series Firewall-based approach is less complicated than configuring redundant ethernet (reth) interfaces and link aggregation groups (LAG) on the MX router. In addition, there is need for additional CLI on the fabric side to insert virtual gateways, and so on.

Preventing Potential Asymmetric Route Issues Using As-Path Prepending

When stateful firewalls are connected to a fabric, there's a potential risk of asymmetric routing. Consider the following scenario: a client outside the fabric initiates a TCP connection to a client inside the fabric via the WAN router. During the initial TCP three-way handshake, the SYN packet is handled by the first WAN router. However, the SYN-ACK response from the fabric client may be processed by the second WAN router, since both routers advertise the same default route into the fabric. The fabric, unaware of any preference, may choose the second WAN router due to ECMP load balancing or other internal mechanisms. If the stateful firewalls (acting as WAN routers) do not share session state, the second WAN router may drop the return packet, as it has no record of the session initiated through the first WAN router.

To prevent this issue, you can make the default route advertised by the second WAN router less preferred, ensuring that return traffic, under normal conditions, is directed to the first WAN router. In BGP, this is typically achieved through AS-Path prepending, which makes the route to the second WAN router less attractive. In the SRX cluster configuration shown below, we use two export filters toward the fabric—one of which includes AS-Path prepending for the default route. These filters are then applied to the fabric neighbours based on which WAN router should be less preferred.

Once BGP peering with the fabric is established, you can verify the default routes received by the fabric. In the example below, core1-switch serves as the service block function.

A similar approach is recommended for the fabric configuration to ensure uplink traffic consistently prefers one WAN router. This is achieved by making the routes advertised to the second WAN router less attractive using AS-Path prepending. Unlike the MX router example above, which uses a shared “export-vrfs” filter, we now use two separate filters—“export-vrfs0” and “export-vrfs1”—as shown in Figure 39 .

They may look the same but all statements in “export-vrfs1” have AP-Path prepending configured as in Figure 40.

Once BGP peering with the SRX cluster is established, verify that the AS-Path prepends in the fabric are visible, as shown in the example below from the SRX cluster. Start by checking the configuration on the fabric’s service block function—in this case, core1-switch.

Then, we also ensure that we receive those routes according to our configuration on the SRX cluster as in below example.

Table 2 shows the configuration information for the core1 and core2 switches as service block function along with the WAN router configuration for the SRX cluster. We’ve marked the changes with respect to Table 1 (for MX WAN routers in bold).

Table 2: SRX Series Firewall WAN Router and Core Switch Configuration Summary

Switch	Switch AS	VRF	Core P2P IP	Core IF	WAN Router	WAN Router P2P IP	WAN Router AS	WAN Router IF	VLAN-ID
core1	64911	customera	10.255.224.1/31	ge-0/0/5.1091	node0	10.255.224.0/31	64901	ge-0/0/2.1091	1091
core1	64911	customerb	10.255.224.3/31	ge-0/0/5.1081	node0	10.255.224.2/31	64901	ge-0/0/2.1081	1081
core1	64911	devices	10.255.224.5/31	ge-0/0/5.1031	node0	10.255.224.4/31	64901	ge-0/0/2.1031	1031
core1	64911	customera	10.255.225.1/31	ge-0/0/6.1099	node1	10.255.225.0/31	64901	ge-7/0/2.1099	1099
core1	64911	customerb	10.255.225.3/31	ge-0/0/6.1088	node1	10.255.225.2/31	64901	ge-7/0/2.1088	1088
core1	64911	devices	10.255.225.5/31	ge-0/0/6.1033	node1	10.255.225.4/31	64901	ge-7/0/2.1033	1033
core2	64911	customera	10.255.226.1/31	ge-0/0/5.1091	node0	10.255.226.0/31	64901	ge-0/0/3.1091	1091
core2	64911	customerb	10.255.226.3/31	ge-0/0/5.1081	node0	10.255.226.2/31	64901	ge-0/0/3.1081	1081
core2	64911	devices	10.255.226.5/31	ge-0/0/5.1031	node0	10.255.226.4/31	64901	ge-0/0/3.1031	1031
core2	64911	customera	10.255.227.1/31	ge-0/0/6.1099	node1	10.255.227.0/31	64901	ge-7/0/3.1099	1099
core2	64911	customerb	10.255.227.3/31	ge-0/0/6.1088	node1	10.255.227.2/31	64901	ge-7/0/3.1088	1088
core2	64911	devices	10.255.227.5/31	ge-0/0/6.1033	node1	10.255.227.4/31	64901	ge-7/0/3.1033	1033

The following pseudocode describes what you need to configure on the core1 switch for this example:

The following pseudocode describes what you need to configure on the core2 switch for this example:

When finished with configuring the individual service block functions (here, the core1 and core2 switches) your overview table should be as shown in Figure 41 .

Figure 41 shows an overview of how the BGP looks after you have configured the individual service block functions for the core1 and core2 switches.

The following Junos OS CLI represents the entire configuration needed on the Series Firewall cluster for this example.