Example: Configuring a Next-Hop-Based Dynamic GRE Tunnels
This example shows how to configure a dynamic generic routing encapsulation (GRE) tunnel that includes a tunnel composite next hop instead of an interface next hop. The next-hop-based dynamic GRE tunnel has a scaling advantage over the interface-based dynamic GRE tunnels.
This example uses the following hardware and software components:
Five MX Series routers with MPCs and MICs.
Junos OS Release 16.2 or later running on the provider edge routers.
Before you begin:
Configure the device interfaces, including the loopback interface.
Configure the router ID and autonmous system number for the device.
Establish an internal BGP (IBGP) session with the remote PE device.
Establish OSPF peering among the devices.
Starting with Junos OS Release 16.2, a dynamic generic routing encapsulation (GRE) tunnel supports the creation of a tunnel composite next hop for every GRE tunnel configured.
By default, for every new dynamic GRE tunnel configured, a corresponding logical tunnel interface is created. This is called an interface-based dynamic tunnel and is the default mode for creating dynamic tunnels. With the interface-based dynamic tunnel, the number of tunnels that can be configured on a device is dependant on the total number of interfaces supported on the device. The next-hop-based dynamic GRE tunnel removes the dependency on physical interfaces, and the GRE tunnel encapsulation is implemented as a next-hop instruction without creating a logical tunnel interface. This provides a scaling advantage for the number of dynamic tunnels that can be created on a device.
Starting in Junos OS Release 17.1, on MX Series routers with MPCs and MICs, the scaling limit of next-hop-based dynamic GRE tunnels is increased. The increased scaling values benefits data center networks, where a gateway router is required to communicate with a number of servers over an IP infrastructure; for example, in Contrail networking.
At a given point in time, either the next-hop-based dynamic tunnel or the default interface-based dynamic GRE tunnel can exist on a device. A switchover from one tunnel mode to another deletes the existing tunnels and creates new tunnels in the new tunnel mode according to the supported scaling limits. Similarly, at a given point in time, for the same tunnel destination, the next-hop-based tunnel encapsulation type can either be GRE or UDP.
To enable the next-hop-based dynamic GRE tunnel, include the next-hop-based-tunnel statement at the [edit routing-options dynamic-tunnels gre] hierarchy level.
The existing dynamic tunnel feature requires complete static configuration. Currently, the tunnel information received from peer devices in advertised routes is ignored. Starting in Junos OS Release 17.4R1, on MX Series routers, the next-hop-based dynamic GRE tunnels are signaled using BGP encapsulation extended community. BGP export policy is used to specify the tunnel types, advertise the sender side tunnel information, and parse and convey the receiver side tunnel information. A tunnel is created according to the received type tunnel community.
Multiple tunnel encapsulations are supported by BGP. On receiving multiple capability, the next-hop-based dynamic tunnel is created based on the configured BGP policy and tunnel preference. The tunnel preference should be consistent across both the tunnel ends for the tunnel to be set up. By default, MPLS-over-UDP (MPLSoUDP) tunnel is preferred over GRE tunnels. If dynamic tunnel configuration exists, it takes precedence over received tunnel community.
When configuring a next-hop-based dynamic GRE tunnel, be aware of the following considerations:
Dynamic tunnels creation is triggered in the IBGP protocol next-hop resolution process.
A switchover from the next-hop-based tunnel mode to the interface-based tunnel mode (and vice versa) is allowed, and this can impact network performance in terms of the supported IP tunnel scaling values in each mode.
RSVP automatic mesh tunnel is not supported with the next-hop-based dynamic tunnel configuration.
Dynamic GRE tunnel creation based upon new IPv4-mapped-IPv6 next hops is supported with this feature.
The following features are not supported with the next-hop-based dynamic GRE tunnel configuration:
RSVP automatic mesh
Per-tunnel traffic statistics collection on the sender (MX Series) side
QoS enforcement, such as policing and shaping, at the tunnel interface.
Path maximum transmission unit discovery
GRE key feature
GRE Operation, Administration, and Maintenance (OAM) for GRE keepalive messages.
Figure 1 illustrates a Layer 3 VPN scenario over dynamic GRE tunnels. The customer edge (CE) devices, CE1 and CE2, connect to provider edge (PE) devices, PE1 and PE2, respectively. The PE devices are connected to a provider device ( Device P1), and an internal BGP (IBGP) session interconnects the two PE devices. Two dynamic GRE tunnels are configured between the PE devices. The dynamic tunnel from Device PE1 to Device PE2 is based on the next hop tunnel mode, and the dynamic tunnel from Device PE2 to Device PE1 is based on the interface tunnel mode.
The next-hop-based dynamic GRE tunnel is handled as follows:
- After a next-hop-based dynamic GRE tunnel is configured,
a tunnel destination mask route with a tunnel composite next hop is
created for the protocol next hops in the inet.3 routing table. This
IP tunnel route is withdrawn only when the dynamic tunnel configuration
The tunnel composite next hop attributes include the following:
When Layer 3 VPN composite next hop is disabled—Source and destination address, encapsulation string, and VPN label.
When Layer 3 VPN composite next hop and per-prefix VPN label allocation are enabled—Source address, destination address, and encapsulation string.
When Layer 3 VPN composite next hop is enabled and per-prefix VPN label allocation is disabled—Source address, destination address, and encapsulation string. The route in this case is added to the other virtual routing and forwarding instance table with a secondary route.
- The PE devices are interconnected using an IBGP session. The IBGP route next hop to a remote BGP neighbor is the protocol next hop, which is resolved using the tunnel mask route with the tunnel next hop.
- After the protocol next hop is resolved over the tunnel composite next hop, indirect next hops with forwarding next hops are created.
- The tunnel composite next hop is used to forward the next hops of the indirect next hops.
CLI Quick Configuration
To quickly configure this example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the  hierarchy level, and then enter commit from configuration mode.
The following example requires that you navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode in the CLI User Guide.
To configure Device PE1:
- Configure the device interfaces including the loopback
interface of the device.[edit interfaces]user@PE1# set ge-0/0/0 unit 0 family inet address 10.0.0.2/8user@PE1# set ge-0/0/1 unit 0 family inet address 192.0.2.1/24user@PE1# set ge-0/0/1 unit 0 family mplsuser@PE1# set lo0 unit 0 family inet address 127.0.0.2/8
- Configure a static route for routes from Device PE1 with
Device P1 as the next-hop destination.[edit routing-options]user@PE1# set static route 188.8.131.52/8 next-hop 192.0.2.2
- Configure the router ID and autonomous system number for
Device PE1.[edit routing-options]user@PE1# set router-id 127.0.0.2user@PE1# set autonomous-system 100
- Configure IBGP peering between the PE devices.[edit protocols]user@PE1# set bgp group IBGP type internaluser@PE1# set bgp group IBGP local-address 127.0.0.2user@PE1# set bgp group IBGP family inet-vpn unicastuser@PE1# set bgp group IBGP neighbor 127.0.0.4
- Configure OSPF on all the interfaces of Device PE1, excluding
the management interface.[edit protocols]user@PE1# set ospf area 0.0.0.0 interface ge-0/0/1.0user@PE1# set ospf area 0.0.0.0 interface lo0.0 passive
- Enable next-hop-based dynamic GRE tunnel configuration
on Device PE1.[edit routing-options]user@PE1# set dynamic-tunnels gre next-hop-based-tunnel
- Configure the dynamic GRE tunnel parameters from Device
PE1 to Device PE2.[edit routing-options]user@PE1# set dynamic-tunnels gre-dyn-tunnel-to-pe2 source-address 127.0.0.2user@PE1# set dynamic-tunnels gre-dyn-tunnel-to-pe2 greuser@PE1# set dynamic-tunnels gre-dyn-tunnel-to-pe2 destination-networks 184.108.40.206/24
- Export the load-balancing policy to the forwarding table.[edit routing-options]user@PE1# set forwarding-table export pplb
- Configure a VRF routing instance on Device PE1 and other
routing instance parameters.[edit routing-instances]user@PE1# set L3VPN-Over-GRE-PE1 instance-type vrfuser@PE1# set L3VPN-Over-GRE-PE1 interface ge-0/0/0.0user@PE1# set L3VPN-Over-GRE-PE1 route-distinguisher 127.0.0.2:1user@PE1# set L3VPN-Over-GRE-PE1 vrf-target target:600:1
- Enable BGP in the routing instance configuration for peering
with Device CE1.[edit routing-instances]user@PE1# set L3VPN-Over-GRE-PE1 protocols bgp group pe1-ce1 peer-as 200user@PE1# set L3VPN-Over-GRE-PE1 protocols bgp group pe1-ce1 neighbor 10.0.0.1 as-override
From configuration mode, confirm your configuration by entering the show interfaces, show routing-options, show protocols, and show routing-instances commands. If the output does not display the intended configuration, repeat the instructions in this example to correct the configuration.
If you are done configuring the device, enter commit from configuration mode.
Confirm that the configuration is working properly.
Verifying the Connection Between PE Devices
Verify the BGP peering status between Device PE1 and Device PE2, and the BGP routes received from Device PE2.
From operational mode, run the show bgp summary and show route receive-protocol bgp ip-address table bgp.l3vpn.0 commands.
user@PE1> show bgp summary
Groups: 2 Peers: 2 Down peers: 0 Table Tot Paths Act Paths Suppressed History Damp State Pending bgp.l3vpn.0 2 2 0 0 0 0 Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped... 127.0.0.4 100 139 136 0 0 58:23 Establ bgp.l3vpn.0: 2/2/2/0 L3VPN-Over-GRE-PE1.inet.0: 2/2/2/0 10.0.0.1 200 135 136 0 0 58:53 Establ L3VPN-Over-GRE-PE1.inet.0: 1/1/1/0
user@PE1> show route receive-protocol bgp 127.0.0.4 table bgp.l3vpn.0
bgp.l3vpn.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden) Prefix Nexthop MED Lclpref AS path 127.0.0.4:1:127.0.0.5/8 * 127.0.0.4 100 200 I 127.0.0.4:1:203.0.113.0/24 * 127.0.0.4 100 I
In the first output, the BGP session state is Establ, which means that the session is up and the PE devices are peered.
In the second output, Device PE1 has learned two BGP routes from Device PE2.
Verify the Dynamic Tunnel Routes on Device PE1
Verify the routes in the inet.3 routing table and the dynamic tunnel database information on Device PE1.
From operational mode, run the show route table inet.3 and show dynamic-tunnels database terse commands.
user@PE1> show route table inet.3
inet.3: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 127.0.0.0/8 *[Tunnel/300] 01:01:45 Tunnel 127.0.0.4/8 *[Tunnel/300] 00:10:24 Tunnel Composite
user@PE1> show dynamic-tunnels database terse
Table: inet.3 Destination-network: 127.0.0.0/8 Destination Source Next-hop Type Status 127.0.0.4/8 127.0.0.2 0xb395e70 nhid 612 gre Up
In the first output, because Device PE1 is configured with the next-hop-based dynamic GRE tunnel, a tunnel composite route is created for the inet.3 routing table route entry.
In the second output, the dynamic GRE tunnel created from Device PE1 to Device PE2 is up and has a next-hop ID assigned to it instead of a next-hop interface.
Verify the Dynamic Tunnel Routes on Device PE2
Verify the routes in the inet.3 routing table and the dynamic tunnel database information on Device PE2.
From operational mode, run the show route table inet.3 and show dynamic-tunnels database terse commands.
user@PE2> show route table inet.3
inet.3: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 127.0.0.0/8 *[Tunnel/300] 01:06:52 Tunnel 127.0.0.2/8 *[Tunnel/300] 01:04:45 > via gr-0/1/0.32769
user@PE1> show dynamic-tunnels database terse
Table: inet.3 Destination-network: 127.0.0.0/8 Destination Source Next-hop Type Status 127.0.0.2/8 127.0.0.4 gr-0/1/0.32769 gre Up
In the first output, because Device PE2 has the default interface-based dynamic GRE tunnel configuration, a new interface is created for the inet.3 routing table route entry.
In the second output, the dynamic GRE tunnel created from Device PE2 to Device PE1 is up, and has the newly created interface name assigned to it.
Verifying That the Routes Have the Expected Indirect-Next-Hop Flag
Verify that Device PE1 and Device PE2 are configured to maintain the indirect next hop to forwarding next-hop binding on the Packet Forwarding Engine forwarding table.
From operational mode, run the show krt indirect-next-hop command on Device PE1 and Device PE2.
user@PE1> show krt indirect-next-hop
Indirect Nexthop: Index: 1048574 Protocol next-hop address: 127.0.0.4 RIB Table: bgp.l3vpn.0 Label: Push 299792 Policy Version: 1 References: 2 Locks: 3 0xb2ab630 Flags: 0x0 INH Session ID: 0x0 INH Version ID: 0 Ref RIB Table: unknown Tunnel type: GRE, Reference count: 3, nhid: 612 Destination address: 127.0.0.4, Source address: 127.0.0.2 Tunnel id: 1, VPN Label: Push 299792, TTL action: prop-ttl IGP FRR Interesting proto count : 2 Chain IGP FRR Node Num : 1 IGP Resolver node(hex) : 0xb3c6d9c IGP Route handle(hex) : 0xb1ad230 IGP rt_entry protocol : Tunnel IGP Actual Route handle(hex) : 0x0 IGP Actual rt_entry protocol : Any
user@PE2> show krt indirect-next-hop
Indirect Nexthop: Index: 1048574 Protocol next-hop address: 127.0.0.2 RIB Table: bgp.l3vpn.0 Label: Push 299792 Policy Version: 2 References: 2 Locks: 3 0xb2ab630 Flags: 0x2 INH Session ID: 0x145 INH Version ID: 0 Ref RIB Table: unknown Next hop: via gr-0/1/0.32769 Label operation: Push 299792 Label TTL action: prop-ttl Load balance label: Label 299792: None; Label element ptr: 0xb395d40 Label parent element ptr: 0x0 Label element references: 1 Label element child references: 0 Label element lsp id: 0 Session Id: 0x144 IGP FRR Interesting proto count : 2 Chain IGP FRR Node Num : 1 IGP Resolver node(hex) : 0xb3d36e8 IGP Route handle(hex) : 0xb1af060 IGP rt_entry protocol : Tunnel IGP Actual Route handle(hex) : 0x0 IGP Actual rt_entry protocol : Any
In the first output, Device PE1 has a next-hop-based dynamic GRE tunnel to Device PE2.
In the second output, Device PE2 has an interface-based dynamic GRE tunnel to device PE1.
To troubleshoot the next-hop-based dynamic tunnels, see:
The next-hop-based dynamic GRE tunnel configuration is not taking effect.
To troubleshoot the next-hop-based GRE tunnel configuration, use the following traceroute commands at the [edit routing-options dynamic-tunnels] statement hierarchy:
traceoptions file file-name
traceoptions file size file-size
traceoptions flag all