EVPN Type 5 Routing over VXLAN Tunnels
Ethernet Virtual Private Network (EVPN) with Virtual Extensible LAN (VXLAN) Type 5 routing is designed for use in data center and cloud environments to provide efficient and scalable network connectivity for virtualized workloads. It combines the benefits of EVPN and VXLAN to enable flexible and seamless communication between virtual machines (VMs) and physical devices across different IP subnets and locations. Starting with Juniper Cloud-Native Router (JCNR) Release 23.3, JCNR supports EVPN Type 5 Routing over VXLAN tunnels.
Ethernet Virtual Private Network (EVPN) technology provides a scalable and efficient way to extend Layer 2 and Layer 3 connectivity across multiple sites. EVPN uses Border Gateway Protocol (BGP) to exchange information between Provider Edge (PE) routers, allowing them to learn the location of Ethernet segments and IP prefixes. This allows for the creation of virtual networks that can span multiple sites, while providing traffic separation and isolation through the use of virtual routing and forwarding (VRF) instances. EVPN supports several encapsulation methods, including VXLAN and MPLS, which can be used to transport traffic across the service provider network.
VXLAN is a network overlay technology that allows the creation of virtual Layer 2 networks on top of an existing Layer 3 network infrastructure. It extends the reach of Layer 2 segments beyond the confines of a single physical network, which is especially useful in large-scale virtualized environments.
EVPN supports two types of routes: MAC Advertisement Route (Type 2) and IP Prefix Route (Type 5). Type 2 routes are used to exchange MAC addresses and VLANs between PE routers, while Type 5 routes are used to exchange Layer 3 network routes. In EVPN VXLAN, Type 5 routes are used to advertise IP prefixes and their associated MAC addresses. To reach a tenant using connectivity provided by the EVPN VXLAN Type 5 IP prefix route, data packets are sent as Layer 2 Ethernet frames encapsulated in the VXLAN header over the IP network across the data centers.
EVPN VXLAN Type 5 routing allows for efficient distribution of MAC and IP routing information, enabling large-scale networks with numerous virtualized workloads to operate seamlessly. The technology supports secure isolation of tenant traffic in shared environments, providing a virtual network overlay that maintains separation between tenants.
To learn more about EVPN VXLAN Type 5 routing, see Understanding EVPN Pure Type-5 Routes.
Transit router functionality should be enabled for JCNR to support EVPN VXLAN Type 5 routing. See, JCNR as a Transit Gateway.
Enabling EVPN Type 5 Routing over VXLAN Tunnels
Enable EVPN Type 5 Routing over VXLAN tunnels using custom JCNR controller configuration via the go template. Apply the custom configuration before installing JCNR, or for an existing JCNR installation, delete the cRPD pod and respawn.
Use the following sample to configure EVPN Type 5 Routing over VXLAN tunnels in JCNR using the jcnr-cni-custom-config-cm.tmpl file located in Juniper_Cloud_Native_Router_<release-number>/cRPD_examples directory.
groups {
custom {
routing-instances {
EVPN-TYPE5-VXLAN-VRF {
instance-type vrf;
protocols {
evpn {
ip-prefix-routes {
advertise direct-nexthop;
encapsulation vxlan;
vni 1000;
export EVPN-TYPE5-VXLAN-VRF-EXPORT-POLICY;
}
}
}
interface ge-0/0/1.0;
route-distinguisher 10.255.0.1:100;
vrf-target target:100:100;
}
}
}To learn more about node annotations and custom configuration, see Customize JCNR Configuration .
To learn about EVPN Type 5 configuration in Junos, see Example: Configuring EVPN with Support for Virtual Switch.
Configuration Example and CLI Commands for EVPN Type 5 Routing over VXLAN Setup
The topology shown above describes a simple setup with two JCNRs deployed as provider edge routers PE1 and PE2. The CE1 and CE2 represent hosts behind each of the PEs. As a pre-requisite, a BGP session must exist between PE1 and PE2. Consider the following EVPN-VXLAN configuration on PE1, with the interface enp4s0 towards CE1:
groups {
custom {
routing-instances {
orange {
instance-type vrf;
routing-options {
rib orange.inet6.0 {
multipath;
}
multipath;
}
protocols {
evpn {
ip-prefix-routes {
advertise direct-nexthop;
encapsulation vxlan;
vni 10010;
}
}
}
interface enp4s0;
route-distinguisher 1.1.1.1:4;
vrf-target target:4:4;
}
}
}
A VXLAN tunnel is created between routers PE1 and PE2. The 10.10.14.0/24 network routes are locally learnt on PE1 and are advertised via EVPN Type 5 to the remote PE. Similarly, the 10.10.24.0/24 network routes are locally learnt on PE2 and advertised via EVPN Type 5 to the remote PE. All traffic between CE1 and CE2 is forwarded between PE1 and PE2 over the VXLAN tunnel.
Use the commands listed in the sections below to troubleshoot a EVPN VXLAN Type 5 routing setup.
cRPD CLI Commands
The following CLI commands can be executed on the cRPD CLI. To access the cRPD CLI, see Access cRPD CLI.
-
show bgp <summary | neighbor>: Provides a summary of the EVPN connection to the peer and the status of the connection.A sample output is shown below:
host@pe1> show bgp summary Threading mode: BGP I/0 Default eBGP mode: advertise - accept, receive - accept Groups: 1 Peers: 2 Down peers: 1 Table Tot Paths Act Paths Suppressed History Damp State Pending bgp. evpn. 0 2 2 0 0 0 0 Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped… 2.2.2.2 4 10345 10336 0 2 3d 5:32:50 Establ bgp.evpn.0: 2/2/2/0 orange.evpn.0: 2/2/2/0 3.3.3.3 4 0 0 0 0 4w4d 13:28:22 Connect
-
show route <summary | table | prefix>: Displays the active entries in the routing tables. -
show evpn instance: Displays information about the EVPN routing instance. -
show evpn l3-context: Displays the configured L3 context on the local box.A sample output is shown below:
host@pe1> show evpn l3-context L3 context Type Adv Encap VNI/Label Router MAC/GW intf orange Cfg Direct VXLAN 10010 48:5a:0d:78:78:d7
-
show evpn ip-prefix-database: Provides a list of exported and imported EVPN route prefixes and the status of these routes.A sample output is shown below:
root@evpn-pe1-node> show evpn ip-prefix-database L3 context: orange IPv4->EVPN Exported Prefixes Prefix EVPN route status 2.55.1.0/24 Created 4.1.1.4/30 Created 10.10.14.0/24 Created IPv6->EVPN Exported Prefixes Prefix EVPN route status 1234::a0a:e00/120 Created abcd::401:104/126 Created abcd::2:55:1:0/120 Created EVPN->IPv4 Imported Prefixes Prefix Etag 2.55.2.0/24 0 Route distinguisher VNI/Label Router MAC Nexthop/Overlay GW/ESI Route-Status Reject-Reason 2.2.2.2:4 10020 48:5a:0d:49:fc:63 2.2.2.2 Accepted n/a 10.10.24.0/24 0 Route distinguisher VNI/Label Router MAC Nexthop/Overlay GW/ESI Route-Status Reject-Reason 2.2.2.2:4 10020 48:5a:0d:49:fc:63 2.2.2.2 Accepted n/a EVPN->IPv6 Imported Prefixes Prefix Etag 1234::a0a:1800/120 0 Route distinguisher VNI/Label Router MAC Nexthop/Overlay GW/ESI Route-Status Reject-Reason 2.2.2.2:4 10020 48:5a:0d:49:fc:63 2.2.2.2 Accepted n/a abcd::2:55:2:0/120 0 Route distinguisher VNI/Label Router MAC Nexthop/Overlay GW/ESI Route-Status Reject-Reason 2.2.2.2:4 10020 48:5a:0d:49:fc:63 2.2.2.2 Accepted n/a
-
show route table <VRF>.evpn.0: Displays the route entries in the specified routing table.A sample output is shown below.
host@pe1> show route table orange. evpn. 0 orange.evpn.0: 4 destinations, 0 routes (4 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 5:1.1.1.1:4::0::10.10.14.0::24/248 *[EVPN/170] 4W4d 13:29:25 Fictitious 5:2.2.2.2:4::0::10.10.24.0::24/248 *[BGP/170] 3d 05:33:52, localpref 100, from 2.2.2.2 AS path: I, validation-state: unverified to 10.10.1.20 via enp2s0 5:1.1.1.1:4::0::1234::00a:000::120/248 *[EVPN/170] 4w4d 13:29:25 Fictitious 5:2.2.2.2:4::0::1234::a0a:1800::120/248 *[BGP/170] 3d 05:33:52, localpref 100, from 2.2.2.2 AS path: I, validation- state: unverified to 10.10.1.20 via enp2s0 -
show route table <VRF>.inet.0: Displays the route entries in the specified routing table. -
show route table bgp.evpn.0: Displays the route entries in the specified routing table.A sample output with a local prefix is shown below.
host@pe1> show route table bgp.evpn.0 match-prefix 5:1.1.1.1:4::0::10.10.14.0::24 bgp.evpn.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 5:1.1.1.1:4::0::10.10.14.0::24/248 *[EVPN/170] 2w1d 05:11:43 Fictitious
A sample output with a remote prefix is shown below.
host@pe1> show route table bgp.evpn.0 match-prefix 5:2.2.2.2:4::0::10.10.24.0::24 bgp.evpn.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 5:2.2.2.2:4::0::10.10.24.0::24/248 *[BGP/170] 2w1d 05:11:48, localpref 100, from 2.2.2.2 AS path: I, validation-state: unverified > to 10.10.1.20 via enp2s0 -
show krt next-hop: Displays the configured next hop.
vRouter CLI Commands
The following CLI commands can be executed on the vRouter CLI. To access the vRouter CLI, see Access vRouter CLI.
-
rt --get <prefix> --vrf <vrf-id> --family <inet4/inet6>: Provides the route which is pointing to the specified IPv4 address.A sample output is shown below.
[host@pe1 /]# rt --get 10.10.24.0/24 --vrf 1 Match 10.10.24.0/24 in vRouter inet4 table 0/1/unicast Flags: L=Label Valid, P=Proxy ARP, T=Trap ARP, F=Flood ARP, Ml=MAC-IP learnt route vRouter inet4 routing table 0/1/unicast Destination PPL Flags Label Nexthop Stitched MAC(Index) 10.10.24.0/24 0 LPT 10020 30 -
-
vxlan --dump: Provides information regarding the VNIs that are configured and the next hop.A sample output is shown below.
[host@pe1 /]# vxlan --dump VXLAN Table VNID NextHop ---------------- 10010 25
-
nh --get <nh-id>: Provides the next hop details.A sample output is shown below.
[root@evpn-pe1-node /]# nh --get 30 Id:30 Type:Tunnel Fmly: AF_INET Rid:0 Ref_cnt:5 Vrf:0 Flags:Valid, Policy, Vxlan, Etree Root, l3_vxlan, Oif:1 Len:14 Data:52 54 00 78 c8 f2 52 54 00 ee 83 cd 08 00 Sip:1.1.1.1 Dip:2.2.2.2 L3_Vxlan_SMac:48:5a:0d:78:78:d7 L3_Vxlan_DMac:48:5a:0d:49:fc:63 -
vif --list: Provides a list of enterprises configured with thevif. -
flow --l: Displays all the active flows in the system.Use this command to verify the traffic flowing between CE1 and CE2 on the vRouter. A sample output is shown below.
[host@pe1 /]# flow -l Flow table(size 161218560, entries 629760) Entries: Created 11 Added 11 Deleted 20 Changed 26Processed 11 Used Overflow entries 0 (Created FlOwS/CPU: 0 0 0 0 0 0 0 0 0 0 11 0 (oflows 0) Action: F-Forward, D=Drop N=NAT(S-SNAT, D=DNAT, PS=SPAT, Pd=DPAT, L=Link Local Port) Other: K(nh)=Key Nexthop, S(nh)=RPF Nexthop Flags: E-Evicted, Ec-Evict Candidate, N=New Flow, M-Modified Dm=Delete Marked TCP(r=reverse): S-SYN, F=FIN, R=RST, C-HalfClose, E-Established, D=Dead Stats: Packets/Bytes Index Source: Port/Destination: Port Proto(V) ------------------------------------------------------------------------------------ 95644<=>443840 10.10.24.21:30 1 (1) 10.10.14.11:0 (Gen: 1, K(nh): 8, Action:F, Flags:, 005: -1, S(nh):30, Stats: 16/1344, SPort 56932, TTL 0. Sinfo 2.2.2.2) 443840<=>95644 10.10.14.11:30 1 (1) 10.10.24.21:0 (Gen: 1, K(nh):8, Action:F, Flags:, Q0S: -1, S(nh):41, Stats: 16/1344, SPort 53983, TTL 0, Sinfo 0.0.0.0) -
vifdump <vif-number>: Displays all the packet details for the specifiedvif.A sample output is shown below.
[host@pe1 /]# vifdump 3 -nevv vif0/3 PCI: 0000:04:00.0 NH: 8 MTU: 9000 dropped privs to tcpdump tcpdump: listening on mon3, link-type EN10MB (Ethernet), snapshot length 262144 bytes 20:15:15.611827 52:54:00:2c:f6:16 > 52:54:00:ef:3c:4d, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 1764, offset 0, flags [DF], proto ICMP (1), length 84) 10.10.14.11 > 10.10.24.21: ICMP echo request, id 16, seq 25, length 64 20:15:15.612472 52:54:00:ef:3c:4d > 52:54:00:2c:f6:16, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 62, id 14142, offset 0, flags [none], proto ICMP (1), length 84) 10.10.24.21 > 10.10.14.11: ICMP echo reply, id 16, seq 25, length 64 20:15:16.626773 52:54:00:2c:f6:16 > 52:54:00:ef:3c:4d, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 1863, offset 0, flags [DF], proto ICMP (1), length 84) 10.10.14.11 > 10.10.24.21: ICMP echo request, id 16, seq 26, length 64 20:15:16.627404 52:54:00:ef:3c:4d > 52:54:00:2c:f6:16, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 62, id 14187, offset 0, flags [none], proto ICMP (1), length 84) 10.10.24.21 > 10.10.14.11: ICMP echo reply, id 16, seq 26, length 64