Example: Configuring Next-Hop-Based MPLS-Over-UDP Dynamic Tunnels
This example shows how to configure a dynamic MPLS-over-UDP tunnel that includes a tunnel composite next hop. The MPLS-over-UDP feature provides a scaling advantage on the number of IP tunnels supported on a device.
Starting in Junos OS Release 18.3R1, MPLS-over-UDP tunnels are supported on PTX Series routers and QFX Series switches. For every dynamic tunnel configured on a PTX router or a QFX switch, a tunnel composite next hop, an indirect next hop, and a forwarding next hop is created to resolve the tunnel destination route. You can also use policy control to resolve the dynamic tunnel over select prefixes by including the forwarding-rib configuration statement at the [edit routing-options dynamic-tunnels] hierarchy level.
This example uses the following hardware and software components:
Five MX Series routers with MPCs and MICs.
Junos OS Release 16.2 or later running on the PE routers.
Before you begin:
Configure the device interfaces, including the loopback interface.
Configure the router ID and autonmous system number for the device.
Establish an internal BGP (IBGP) session with the remote PE device.
Establish OSPF peering among the devices.
Starting with Junos OS Release 16.2, a dynamic UDP tunnel supports the creation of a tunnel composite next hop for every UDP tunnel configured. These next-hop-based dynamic UDP tunnels are referred to as MPLS-over-UDP tunnels. The tunnel composite next hop are enabled by default for the MPLS-over-UDP tunnels.
Starting in Junos OS Release 17.1, on MX Series routers with MPCs and MICs, the scaling limit of MPLS-over-UDP tunnels is increased.
For the same tunnel destination, the next-hop-based dynamic tunnel encapsulation can either be GRE or UDP. Having both tunnel encapsulations for the same tunnel destination causes a commit error under the dynamic-tunnel configuration.
The existing dynamic tunnel feature requires complete static configuration. Currently, the tunnel information received from peer devices in advertised routes is ignored. Starting in Junos OS Release 17.4R1, on MX Series routers, the next-hop-based dynamic MPLS-over-UDP tunnels are signaled using BGP encapsulation extended community. BGP export policy is used to specify the tunnel types, advertise the sender side tunnel information, and parse and convey the receiver side tunnel information. A tunnel is created according to the received type tunnel community.
Multiple tunnel encapsulations are supported by BGP. On receiving multiple capability, the next-hop-based dynamic tunnel is created based on the configured BGP policy and tunnel preference. The tunnel preference should be consistent across both the tunnel ends for the tunnel to be set up. By default, MPLS-over-UDP (MPLSoUDP) tunnel is preferred over GRE tunnels. If dynamic tunnel configuration exists, it takes precedence over received tunnel community.
When configuring a next-hop-based dynamic MPLS-over-UDP tunnel, be aware of the following considerations:
An IBGP session must be configured between the PE devices.
A switchover between the next-hop-based dynamic tunnel encapsulations (UDP and GRE) is allowed, and this can impact network performance in terms of the supported IP tunnel scaling values in each mode.
Having both GRE and UDP next-hop-based dynamic tunnel encapsulation types for the same tunnel destination leads to a commit failure.
Graceful Routing Engine switchover (GRES) is supported with MPLS-over-UDP, and the MPLS-over-UDP tunnel type flags are unified ISSU and NSR compliant.
MPLS-over-UDP tunnels are supported on virtual MX (vMX).
MPLS-over-UDP tunnels support dynamic GRE tunnel creation based upon new IPv4-mapped-IPv6 next hops.
MPLS-over-UDP tunnel are supported in interoperability with contrail, wherein the MPLS-over-UDP tunnels are created from the contrail vRouter to an MX gateway. To enable this, the following community is required to be advertised in the route from the MX Series router to the contrail vRouter:[edit policy-options community]udp members 0x030c:64512:13;
At a given point in time, only one tunnel type is supported on the contrail vRouter—next-hop-based dynamic GRE tunnels, MPLS-over-UDP tunnels, or VXLAN.
The following features are not supported with the next-hop-based dynamic MPLS-over-UDP tunnel configuration:
RSVP automatic mesh
Plain IPV6 GRE and UDP tunnel configuration
Figure 1 illustrates a Layer 3 VPN scenario over dynamic MPLS-over-UDP tunnels. The customer edge (CE) devices, CE1 and CE2, connect to provider edge (PE) devices, PE1 and PE2, respectively. The PE devices are connected to a provider device (Device P1), and an internal BGP (IBGP) session interconnects the two PE devices. Two dynamic next-hop-based MPL-over-UDP tunnels are configured between the PE devices.
The MPLS-over-UDP tunnel is handled as follows:
- After a MPLS-over-UDP tunnel is configured, a tunnel destination
mask route with a tunnel composite next hop is created for the tunnel
in the inet.3 routing table. This IP tunnel route is withdrawn only
when the dynamic tunnel configuration is deleted.
The tunnel composite next-hop attributes include the following:
When Layer 3 VPN composite next hop is disabled—Source and destination address, encapsulation string, and VPN label.
When Layer 3 VPN composite next hop and per-prefix VPN label allocation are enabled—Source address, destination address, and encapsulation string.
When Layer 3 VPN composite next hop is enabled and per-prefix VPN label allocation is disabled—Source address, destination address, and encapsulation string. The route in this case is added to the other virtual routing and forwarding instance table with a secondary route.
- The PE devices are interconnected using an IBGP session. The IBGP route next hop to a remote BGP neighbor is the protocol next hop, which is resolved using the tunnel mask route with the tunnel next hop.
- After the protocol next hop is resolved over the tunnel composite next hop, indirect next hops with forwarding next hops are created.
- The tunnel composite next hop is used to forward the next hops of the indirect next hops.
CLI Quick Configuration
To quickly configure this example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the  hierarchy level, and then enter commit from configuration mode.
The following example requires that you navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode in the CLI User Guide.
To configure Device PE1:
- Configure the device interfaces including the loopback
interface of the device.[edit interfaces]user@PE1# set ge-0/0/0 unit 0 family inet address 10.0.0.2/8user@PE1# set ge-0/0/1 unit 0 family inet address 192.0.2.1/24user@PE1# set ge-0/0/1 unit 0 family mplsuser@PE1# set lo0 unit 0 family inet address 127.0.0.2/8
- Configure a static route for routes from Device PE1 with
Device P1 as the next-hop destination.[edit routing-options]user@PE1# set static route 126.96.36.199/8 next-hop 192.0.2.2
- Configure the router-ID and autonomous system number for
Device PE1.[edit routing-options]user@PE1# set router-id 127.0.0.2user@PE1# set autonomous-system 100
- (PTX Series
only) Configure policy control to resolve the MPLS-over-UDP dynamic
tunnel route over select prefixes.[edit routing-options dynamic-tunnels]user@PTX-PE1# set forwarding-rib inet.0 inet-import dynamic-tunnel-fwd-route-import
- (PTX Series only) Configure the inet-import policy
for resolving dynamic tunnel destination routes over .[edit policy-options]user@PTX-PE1# set policy-statement dynamic-tunnel-fwd-route-import term 1 from route-filter 127.0.0.0/8 exactuser@PTX-PE1# set policy-statement dynamic-tunnel-fwd-route-import term 1 then acceptuser@PTX-PE1# set policy-options policy-statement dynamic-tunnel-fwd-route-import then reject
- Configure IBGP peering between the PE devices.[edit protocols]user@PE1# set bgp group IBGP type internaluser@PE1# set bgp group IBGP local-address 127.0.0.2user@PE1# set bgp group IBGP family inet-vpn unicastuser@PE1# set bgp group IBGP neighbor 127.0.0.4
- Configure OSPF on all the interfaces of Device PE1, excluding
the management interface.[edit protocols]user@PE1# set ospf area 0.0.0.0 interface ge-0/0/1.0user@PE1# set ospf area 0.0.0.0 interface lo0.0 passive
- Enable next-hop-based dynamic GRE tunnel configuration
on Device PE1.
This step is required only for illustrating the implementation difference between next-hop-based dynamic GRE tunnels and MPLS-over-UDP tunnels.[edit routing-options]user@PE1# set dynamic-tunnels gre next-hop-based-tunnel
- Configure the MPLS-over-UDP tunnel parameters from Device
PE1 to Device PE2.[edit routing-options]user@PE1# set dynamic-tunnels udp-dyn-tunnel-to-pe2 source-address 127.0.0.2user@PE1# set dynamic-tunnels udp-dyn-tunnel-to-pe2 udpuser@PE1# set dynamic-tunnels udp-dyn-tunnel-to-pe2 destination-networks 127.0.0.0/8
- Configure a VRF routing instance on Device PE1 and other
routing instance parameters.[edit routing-instances]user@PE1# set MPLS-over-UDP-PE1 instance-type vrfuser@PE1# set MPLS-over-UDP-PE1 interface ge-0/0/0.0user@PE1# set MPLS-over-UDP-PE1 route-distinguisher 127.0.0.2:1user@PE1# set MPLS-over-UDP-PE1 vrf-target target:600:1
- Enable BGP in the routing instance configuration for peering
with Device CE1.[edit routing-instances]user@PE1# set MPLS-over-UDP-PE1 protocols bgp group pe1-ce1 peer-as 200user@PE1# set MPLS-over-UDP-PE1 protocols bgp group pe1-ce1 neighbor 10.0.0.1 as-override
From configuration mode, confirm your configuration by entering the show interfaces, show routing-options, show protocols, and show routing-instances commands. If the output does not display the intended configuration, repeat the instructions in this example to correct the configuration.
If you are done configuring the device, enter commit from configuration mode.
Confirm that the configuration is working properly.
Verifying the Connection Between PE Devices
Verify the BGP peering status between Device PE1 and Device PE2, and the BGP routes received from Device PE2.
From operational mode, run the show bgp summary and show route receive-protocol bgp ip-address table bgp.l3vpn.0 commands.
user@PE1> show bgp summary
Groups: 2 Peers: 2 Down peers: 0 Table Tot Paths Act Paths Suppressed History Damp State Pending bgp.l3vpn.0 2 2 0 0 0 0 Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped... 127.0.0.4 100 139 136 0 0 58:23 Establ bgp.l3vpn.0: 2/2/2/0 MPLS-over-UDP-PE1.inet.0: 2/2/2/0 10.0.0.1 200 135 136 0 0 58:53 Establ MPLS-over-UDP-PE1.inet.0: 1/1/1/0
user@PE1> show route receive-protocol bgp 127.0.0.4 table bgp.l3vpn.0
bgp.l3vpn.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden) Prefix Nexthop MED Lclpref AS path 127.0.0.4:1:127.0.0.5/8 * 127.0.0.4 100 200 I 127.0.0.4:1:188.8.131.52/24 * 127.0.0.4 100 I
In the first output, the BGP session state is Establ, which means that the session is up and the PE devices are peered.
In the second output, Device PE1 has learned two BGP routes from Device PE2.
Verify the Dynamic Tunnel Routes on Device PE1
Verify the routes in the inet.3 routing table and the dynamic tunnel database information on Device PE1.
From operational mode, run the show route table inet.3, show dynamic-tunnels database terse, show dynamic-tunnels database, and show dynamic-tunnels database summary commands.
user@PE1> show route table inet.3
inet.3: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 127.0.0.0/8 *[Tunnel/300] 00:21:18 Tunnel 127.0.0.4/8 *[Tunnel/300] 00:21:18 Tunnel Composite
user@PE1> show dynamic-tunnels database terse
Table: inet.3 Destination-network: 127.0.0.0/8 Destination Source Next-hop Type Status 127.0.0.4/8 127.0.0.2 0xb395b10 nhid 613 udp Up
user@PE1> show dynamic-tunnels database
Table: inet.3 Destination-network: 184.108.40.206/8 Destination-network: 220.127.116.11/16 Destination-network: 18.104.22.168/24 Tunnel to: 127.0.0.4/8 Reference count: 2 Next-hop type: UDP Source address: 127.0.0.2 Tunnel Id: 2 Next hop: tunnel-composite, 0xb395b10, nhid 613 VPN Label: Push 299776 Reference count: 3 Traffic Statistics: Packets 0, Bytes 0 State: Up
user@PE1> show dynamic-tunnels database summary
Dynamic Tunnels, Total 1 displayed GRE Tunnel: Active Tunnel Mode, Next Hop Base IFL Based, Total 0 displayed, Up 0, Down 0 Nexthop Based, Total 0 displayed, Up 0, Down 0 RSVP Tunnel: Total 0 displayed UDP Tunnel: Total 1 displayed, Up 1, Down 0
In the first output, because Device PE1 is configured with the MPLS-over-UDP tunnel, a tunnel composite route is created for the inet.3 routing table route entry.
In the remaining outputs, the MPLS-over-UDP tunnel is displayed with the tunnel encapsulation type, tunnel next hop parameters, and tunnel status.
Verify the Dynamic Tunnel Routes on Device PE2
Verify the routes in the inet.3 routing table and the dynamic tunnel database information on Device PE2.
From operational mode, run the show route table inet.3, and the show dynamic-tunnels database terse commands.
user@PE2> show route table inet.3
inet.3: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 127.0.0.0/8 *[Tunnel/300] 00:39:31 Tunnel 127.0.0.2/8 *[Tunnel/300] 00:24:53 Tunnel Composite
user@PE1> show dynamic-tunnels database terse
Table: inet.3 Destination-network: 127.0.0.0/8 Destination Source Next-hop Type Status 127.0.0.2/8 127.0.0.4 0xb395450 nhid 615 udp Up
The outputs show the MPLS-over-UDP tunnel creation and the next-hop ID assigned as the next-hop interface, similar to Device PE1.
Verifying That the Routes Have the Expected Indirect-Next-Hop Flag
Verify that Device PE1 and Device PE2 are configured to maintain the indirect next hop to forwarding next-hop binding on the Packet Forwarding Engine forwarding table.
From operational mode, run the show krt indirect-next-hop command on Device PE1 and Device PE2.
user@PE1> show krt indirect-next-hop
Indirect Nexthop: Index: 1048574 Protocol next-hop address: 127.0.0.4 RIB Table: bgp.l3vpn.0 Label: Push 299776 Policy Version: 1 References: 1 Locks: 3 0xb2ab630 Flags: 0x0 INH Session ID: 0x0 INH Version ID: 0 Ref RIB Table: unknown Tunnel type: UDP, Reference count: 3, nhid: 613 Destination address: 127.0.0.4, Source address: 127.0.0.2 Tunnel id: 2, VPN Label: Push 299776, TTL action: prop-ttl IGP FRR Interesting proto count : 1 Chain IGP FRR Node Num : 1 IGP Resolver node(hex) : 0xb3c70dc IGP Route handle(hex) : 0xb1ae688 IGP rt_entry protocol : Tunnel IGP Actual Route handle(hex) : 0x0 IGP Actual rt_entry protocol : Any
user@PE2> show krt indirect-next-hop
Indirect Nexthop: Index: 1048575 Protocol next-hop address: 127.0.0.2 RIB Table: bgp.l3vpn.0 Label: Push 299776 Policy Version: 1 References: 2 Locks: 3 0xb2ab740 Flags: 0x0 INH Session ID: 0x0 INH Version ID: 0 Ref RIB Table: unknown Tunnel type: UDP, Reference count: 3, nhid: 615 Destination address: 127.0.0.2, Source address: 127.0.0.4 Tunnel id: 1, VPN Label: Push 299776, TTL action: prop-ttl IGP FRR Interesting proto count : 2 Chain IGP FRR Node Num : 1 IGP Resolver node(hex) : 0xb3d3a28 IGP Route handle(hex) : 0xb1ae634 IGP rt_entry protocol : Tunnel IGP Actual Route handle(hex) : 0x0 IGP Actual rt_entry protocol : Any
The outputs show that a next-hop-based dynamic MPLS-over-UDP tunnel is created between the PE devices.
To troubleshoot the next-hop-based dynamic tunnels, see:
The next-hop-based dynamic MPLS-over-UDP tunnel configuration is not taking effect.
To troubleshoot the next-hop-based MPLS-over-UDP tunnel configuration, use the following traceroute commands at the [edit routing-options dynamic-tunnels] statement hierarchy:
traceoptions file file-name
traceoptions file size file-size
traceoptions flag all