Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
 

Next-Hop-Based Dynamic Tunnels

Example: Configuring Next-Hop-Based MPLS-Over-UDP Dynamic Tunnels

This example shows how to configure a dynamic MPLS-over-UDP tunnel that includes a tunnel composite next hop. The MPLS-over-UDP feature provides a scaling advantage on the number of IP tunnels supported on a device.

Starting in Junos OS Release 18.3R1, MPLS-over-UDP tunnels are supported on PTX Series routers and QFX Series switches. For every dynamic tunnel configured on a PTX router or a QFX switch, a tunnel composite next hop, an indirect next hop, and a forwarding next hop is created to resolve the tunnel destination route. You can also use policy control to resolve the dynamic tunnel over select prefixes by including the forwarding-rib configuration statement at the [edit routing-options dynamic-tunnels] hierarchy level.

Requirements

This example uses the following hardware and software components:

  • Five MX Series routers with MPCs and MICs.

  • Junos OS Release 16.2 or later running on the provider edge (PE) routers.

Before you begin:

  1. Configure the device interfaces, including the loopback interface.

  2. Configure the router ID and autonmous system number for the device.

  3. Establish an internal BGP (IBGP) session with the remote PE device.

  4. Establish OSPF peering among the devices.

Overview

Starting with Junos OS Release 16.2, a dynamic UDP tunnel supports the creation of a tunnel composite next hop for every UDP tunnel configured. These next-hop-based dynamic UDP tunnels are referred to as MPLS-over-UDP tunnels. The tunnel composite next hop are enabled by default for the MPLS-over-UDP tunnels.

MPLS-over-UDP tunnels can be bidirectional or unidirectional in nature.

  • Bidirectional—When the PE devices are connected over MPLS-over-UDP tunnels in both directions, it is called a bidirectional MPLS-over-UDP tunnel.

  • Unidirectional—When two PE devices are connected over MPLS-over-UDP tunnel in one direction, and over MPLS/IGP in the other direction, it is called an unidirectional MPLS-over-UDP tunnel.

    Unidirectional MPLS-over-UDP tunnels are used in migration scenarios, or in cases where two PE devices provide connectivity to each other over two disjoint networks. Because reverse direction tunnel does not exist for unidirectional MPLS-over-UDP tunnels, you must configure a filter-based MPLS-over-UDP decapsulation on the remote PE device for forwarding the traffic.

Starting in Junos OS Release 18.2R1, on PTX series routers and QFX10000 with unidirectional MPLS-over-UDP tunnels, you must configure the remote PE device with an input filter for MPLS-over-UDP packets, and an action for decapsulating the IP and UDP headers for forwarding the packets in the reverse tunnel direction.

For example, on the remote PE device, Device PE2, the following configuration is required for unidirectional MPLS-over-UDP tunnels:

PE2

In the above sample configuration, Decap_Filter is the name of the firewall filter used for MPLS-over-UDP decapsulation. The term udp_decap is the input filter for accepting UDP packets on the core-facing interface of Device PE2, and then decapsulate the MPLS-over-UDP packets to MPLS-over-IP packets for forwarding.

You can use the existing firewall operational mode commands, such as show firewall filter to view the filter-based MPLS-over-UDP decapsulation.

For example:

Note:

For unidirectional MPLS-over-UDP tunnels:

  • Only IPv4 address is supported as the outer header. Filter-based MPLS-over-UDP decapsulation does not support IPv6 address in the outer header.

  • Only the default routing instance is supported after decapsulation.

Starting in Junos OS Release 17.1, on MX Series routers with MPCs and MICs, the scaling limit of MPLS-over-UDP tunnels is increased.

Starting in Junos Release 19.2R1, on MX Series routers with MPCs and MICs, carrier supporting carrier (CSC) architecture can be deployed with MPLS-over-UDP tunnels carrying MPLS traffic over dynamic IPv4 UDP tunnels that are established between supporting carrier's PE devices. With this enhancement, the scaling advantage that the MPLS-over-UDP tunnels provided is further increased. The CSC support with MPLS-over-UDP tunnel is not supported for IPv6 UDP tunnel.

The existing dynamic tunnel feature requires complete static configuration. Currently, the tunnel information received from peer devices in advertised routes is ignored. Starting in Junos OS Release 17.4R1, on MX Series routers, the next-hop-based dynamic MPLS-over-UDP tunnels are signaled using BGP encapsulation extended community. BGP export policy is used to specify the tunnel types, advertise the sender side tunnel information, and parse and convey the receiver side tunnel information. A tunnel is created according to the received type tunnel community.

Multiple tunnel encapsulations are supported by BGP. On receiving multiple capability, the next-hop-based dynamic tunnel is created based on the configured BGP policy and tunnel preference. The tunnel preference should be consistent across both the tunnel ends for the tunnel to be set up. By default, MPLS-over-UDP tunnel is preferred over GRE tunnels. If dynamic tunnel configuration exists, it takes precedence over received tunnel community.

When configuring a next-hop-based dynamic MPLS-over-UDP tunnel, be aware of the following considerations:

  • An IBGP session must be configured between the PE devices.

  • A switchover between the next-hop-based dynamic tunnel encapsulations (UDP and GRE) is allowed, and this can impact network performance in terms of the supported IP tunnel scaling values in each mode.

  • Having both GRE and UDP next-hop-based dynamic tunnel encapsulation types for the same tunnel destination leads to a commit failure.

  • For unidirectional MPLS-over-UDP tunnels, you must explicitly configure filter-based MPLS-over-UDP decapsulation on the remote PE device for the packets to be forwarded.

  • Graceful Routing Engine switchover (GRES) is supported with MPLS-over-UDP, and the MPLS-over-UDP tunnel type flags are unified ISSU and NSR compliant.

  • MPLS-over-UDP tunnels are supported on virtual MX (vMX) in Lite mode.

  • MPLS-over-UDP tunnels support dynamic GRE tunnel creation based upon new IPv4-mapped-IPv6 next hops.

  • MPLS-over-UDP tunnel are supported in interoperability with contrail, wherein the MPLS-over-UDP tunnels are created from the contrail vRouter to an MX gateway. To enable this, the following community is required to be advertised in the route from the MX Series router to the contrail vRouter:

    At a given point in time, only one tunnel type is supported on the contrail vRouter—next-hop-based dynamic GRE tunnels, MPLS-over-UDP tunnels, or VXLAN.

  • The following features are not supported with the next-hop-based dynamic MPLS-over-UDP tunnel configuration:

    • RSVP automatic mesh

    • Plain IPV6 GRE and UDP tunnel configuration

    • Logical systems

Topology

Figure 1 illustrates a Layer 3 VPN scenario over dynamic MPLS-over-UDP tunnels. The customer edge (CE) devices, CE1 and CE2, connect to provider edge (PE) devices, PE1 and PE2, respectively. The PE devices are connected to a provider device (Device P1), and an internal BGP (IBGP) session interconnects the two PE devices. A dynamic next-hop-based bidirectional MPL-over-UDP tunnel is configured between the PE devices.

Figure 1: Dynamic MPLS-over-UDP TunnelsDynamic MPLS-over-UDP Tunnels

The MPLS-over-UDP tunnel is handled as follows:

  1. After a MPLS-over-UDP tunnel is configured, a tunnel destination mask route with a tunnel composite next hop is created for the tunnel in the inet.3 routing table. This IP tunnel route is withdrawn only when the dynamic tunnel configuration is deleted.

    The tunnel composite next-hop attributes include the following:

    • When Layer 3 VPN composite next hop is disabled—Source and destination address, encapsulation string, and VPN label.

    • When Layer 3 VPN composite next hop and per-prefix VPN label allocation are enabled—Source address, destination address, and encapsulation string.

    • When Layer 3 VPN composite next hop is enabled and per-prefix VPN label allocation is disabled—Source address, destination address, and encapsulation string. The route in this case is added to the other virtual routing and forwarding instance table with a secondary route.

  2. The PE devices are interconnected using an IBGP session. The IBGP route next hop to a remote BGP neighbor is the protocol next hop, which is resolved using the tunnel mask route with the tunnel next hop.

  3. After the protocol next hop is resolved over the tunnel composite next hop, indirect next hops with forwarding next hops are created.

  4. The tunnel composite next hop is used to forward the next hops of the indirect next hops.

Configuration

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

CE1

CE2

PE1

P1

PE2

Procedure

Step-by-Step Procedure

The following example requires that you navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode in the CLI User Guide.

To configure Device PE1:

  1. Configure the device interfaces including the loopback interface of the device.

  2. Configure a static route for routes from Device PE1 with Device P1 as the next-hop destination.

  3. Configure the router-ID and autonomous system number for Device PE1.

  4. (PTX Series only) Configure policy control to resolve the MPLS-over-UDP dynamic tunnel route over select prefixes.

  5. (PTX Series only) Configure the inet-import policy for resolving dynamic tunnel destination routes over .

  6. Configure IBGP peering between the PE devices.

  7. Configure OSPF on all the interfaces of Device PE1, excluding the management interface.

  8. Enable next-hop-based dynamic GRE tunnel configuration on Device PE1.

    Note:

    This step is required only for illustrating the implementation difference between next-hop-based dynamic GRE tunnels and MPLS-over-UDP tunnels.

  9. Configure the MPLS-over-UDP tunnel parameters from Device PE1 to Device PE2.

  10. Configure a VRF routing instance on Device PE1 and other routing instance parameters.

  11. Enable BGP in the routing instance configuration for peering with Device CE1.

Results

From configuration mode, confirm your configuration by entering the show interfaces, show routing-options, show protocols, and show routing-instances commands. If the output does not display the intended configuration, repeat the instructions in this example to correct the configuration.

If you are done configuring the device, enter commit from configuration mode.

Verification

Confirm that the configuration is working properly.

Verifying the Connection Between PE Devices

Purpose

Verify the BGP peering status between Device PE1 and Device PE2, and the BGP routes received from Device PE2.

Action

From operational mode, run the show bgp summary and show route receive-protocol bgp ip-address table bgp.l3vpn.0 commands.

Meaning
  • In the first output, the BGP session state is Establ, which means that the session is up and the PE devices are peered.

  • In the second output, Device PE1 has learned two BGP routes from Device PE2.

Verify the Dynamic Tunnel Routes on Device PE1

Purpose

Verify the routes in the inet.3 routing table and the dynamic tunnel database information on Device PE1.

Action

From operational mode, run the show route table inet.3, show dynamic-tunnels database terse, show dynamic-tunnels database, and show dynamic-tunnels database summary commands.

Meaning
  • In the first output, because Device PE1 is configured with the MPLS-over-UDP tunnel, a tunnel composite route is created for the inet.3 routing table route entry.

  • In the remaining outputs, the MPLS-over-UDP tunnel is displayed with the tunnel encapsulation type, tunnel next hop parameters, and tunnel status.

Verify the Dynamic Tunnel Routes on Device PE2

Purpose

Verify the routes in the inet.3 routing table and the dynamic tunnel database information on Device PE2.

Action

From operational mode, run the show route table inet.3, and the show dynamic-tunnels database terse commands.

Meaning

The outputs show the MPLS-over-UDP tunnel creation and the next-hop ID assigned as the next-hop interface, similar to Device PE1.

Verifying That the Routes Have the Expected Indirect-Next-Hop Flag

Purpose

Verify that Device PE1 and Device PE2 are configured to maintain the indirect next hop to forwarding next-hop binding on the Packet Forwarding Engine forwarding table.

Action

From operational mode, run the show krt indirect-next-hop command on Device PE1 and Device PE2.

Meaning

The outputs show that a next-hop-based dynamic MPLS-over-UDP tunnel is created between the PE devices.

Troubleshooting

To troubleshoot the next-hop-based dynamic tunnels, see:

Troubleshooting Commands

Problem

The next-hop-based dynamic MPLS-over-UDP tunnel configuration is not taking effect.

Solution

To troubleshoot the next-hop-based MPLS-over-UDP tunnel configuration, use the following traceroute commands at the [edit routing-options dynamic-tunnels] statement hierarchy:

  • traceoptions file file-name

  • traceoptions file size file-size

  • traceoptions flag all

For example:

Anti-Spoofing Protection for Next-Hop-Based Dynamic Tunnels Overview

With the rise in deployment of high-scale IP tunnels in data centers, there is a need to add security measures that allow users to limit malicious traffic from compromised virtual machines (VMs). One possible attack is the injecting of traffic into an arbitrary customer VPN from a compromised server through the gateway router. In such cases, anti-spoofing checks on IP tunnels ensure that only legitimate sources are injecting traffic into data centers from their designated IP tunnels.

Next-hop-based dynamic IP tunnels create a tunnel composite next hop for every dynamic tunnel created on the device. Because next-hop-based dynamic tunnels remove the dependency on physical interfaces for every new dynamic tunnel configured, configuring next-hop-based dynamic tunnels provides a scaling advantage over the number of dynamic tunnels that can be created on a device. Starting in Junos OS Release 17.1, anti-spoofing capabilities for next-hop-based dynamic IP tunnels is provided for next-hop-based dynamic tunnels. With this enhancement, a security measure is implemented to prevent injecting of traffic into an arbitrary customer VPN from a compromised server through the gateway router.

Anti-spoofing is implemented using reverse path forwarding checks in the Packet Forwarding Engine. The checks are implemented for the traffic coming through the tunnel to the routing instance. Currently, when the gateway router receives traffic from a tunnel, only the destination lookup us done and the packet is forwarded accordingly. When anti-spoofing protection is enabled, the gateway router also does a source address lookup of the encapsulation packet IP header in the VPN, in addition to the tunnel destination lookup. This ensures that legitimate sources are injecting traffic through their designated IP tunnels. As a result, anti-spoofing protection ensures that the tunnel traffic is received from a legitimate source on the designated tunnels.

Figure 2 illustrates a sample topology with the requirements for anti-spoofing protection.

Figure 2: Anti-Spoofing Protection for Next-Hop-Based Dynamic TunnelsAnti-Spoofing Protection for Next-Hop-Based Dynamic Tunnels

In this example, the gateway router is Router G. Router G has two VPNs—Green and Blue. The two servers, Server A and Server B, can reach the Green and Blue VPNs on Router G through the next-hop-based dynamic tunnels T1 and T2, respectively. Several hosts and virtual machines (P, Q, R, S, and T) connected to the servers can reach the VPNs through the gateway router, Router G. Router G has the virtual routing and forwarding (VRF) tables for Green and Blue VPNs, each populated with the reachability information for the virtual machines in those VPNs.

For example, in VPN Green, Router G uses tunnel T1 to reach host P, tunnel T2 to reach hosts R and S, and load balancing is done between tunnels T1 and T2 to reach the multihomed host Q. In VPN Blue, Router G uses tunnel T1 to reach hosts P and R, and tunnel T2 to reach hosts Q and T.

The check passes for reverse path forwarding when:

  • A packet comes from a legitimate source on its designated tunnel.

    Host P in VPN Green sends a packet to host X using tunnel T1. Because Router G can reach host P through tunnel T1, it allows the packet to pass and forwards the packet to host X.

  • A packet comes from a multihomed source on its designated tunnels.

    Host Q in VPN Green is multihomed on servers A and B, and can reach Router G through tunnels T1 and T2. Host Q sends a packet to host Y using tunnel T1, and a packet to host X using tunnel T2. Because Router G can reach host Q through tunnels T1 and T2, it allows the packets to pass and forwards them to hosts Y and X, respectively.

Layer 3 VPNs do not have anti-spoofing protection enabled by default. To enable anti-spoofing for next-hop-based dynamic tunnels, include the ip-tunnel-rpf-check statement at the [edit routing-instances routing-instance-name routing-options forwarding-table] hierarchy level. The reverse path forwarding check is applied to the VRF routing instance only. The default mode is set to strict, where the packet that comes from a source on a nondesignated tunnel does not pass the check. The ip-tunnel-rpf-check mode can be set as loose, where the reverse path forwarding check fails when the packet comes from a nonexistent source. An optional firewall filter can be configured under the ip-tunnel-rpf-check statement to count and log the packets that failed the reverse path forwarding check.

The following sample output shows an anti-spoofing configuration:

Take the following guidelines under consideration when configuring anti-spoofing protection for next-hop-based dynamic tunnels:

  • Anti-spoofing protection can be enabled for IPv4 tunnels and IPv4 data traffic only. The anti-spoofing capabilities are not supported on IPv6 tunnels and IPv6 data traffic.

  • Anti-spoofing for next-hop-based dynamic tunnels can detect and prevent a compromised virtual machine (inner source reverse path forwarding check) but not a compromised server that is label-spoofing.

  • The next-hop-based IP tunnels can originate and terminate on an inet.0 routing table.

  • Anti-spoofing protection is effective when the VRF routing instance has label-switched interfaces (LSIs) (using the vrf-table-label), or virtual tunnel (VT) interfaces. With per-next-hop label on the VRF routing instance, anti-spoofing protection is not supported.

  • The rpf fail-filter is applicable only to the inner IP packet.

  • Enabling anti-spoofing checks does not affect the scaling limit of the next-hop-based dynamic tunnels on a device.

  • The system resource utilization with anti-spoofing protection enabled for the VRF routing instance is slightly higher than the utilization of next-hop-based dynamic tunnels without the anti-spoofing protection enabled.

  • Anti-spoofing protection requires additional source IP address checks, which has minimal impact on network performance.

  • Graceful Routing Engine switchover (GRES) and in-service software upgrade (ISSU) are supported with anti-spoofing protection.

Example: Configuring Anti-Spoofing Protection for Next-Hop-Based Dynamic Tunnels

This example shows how to configure reverse path forwarding checks for the virtual routing and forwarding (VRF) routing instance to enable anti-spoofing protection for next-hop-based dynamic tunnels. The checks ensure that legitimate sources are injecting traffic through their designated IP tunnels.

Requirements

This example uses the following hardware and software components:

  • Three MX Series Routers with MICs, each connected to a host device.

  • Junos OS Release 17.1 or later running on one or all the routers.

Before you begin:

  • Enable tunnel services configuration on the Flexible PIC Concentrator.

  • Configure the router interfaces.

  • Configure the router-ID and assign an autonomous system number for the router.

  • Establish an internal BGP (IBGP) session with the tunnel endpoints.

  • Configure RSVP on all the routers.

  • Configure OSPF or any other interior gateway protocol on all the routers.

  • Configure two dynamic next-hop-based IP tunnels between the two routers.

  • Configure a VRF routing instance for every router-to-host connection.

Overview

Starting in Junos OS Release 17.1, anti-spoofing capabilities are added to next-hop-based dynamic IP tunnels, where checks are implemented for the traffic coming through the tunnel to the routing instance using reverse path forwarding in the Packet Forwarding Engine.

Currently, when the gateway router receives traffic from a tunnel, only the destination address lookup is done before forwarding. With anti-spoofing protection, the gateway router does a source address lookup of the encapsulation packet IP header in the VPN to ensure that legitimate sources are injecting traffic through their designated IP tunnels. This is called the strict mode and is the default behavior of anti-spoofing protection. To pass traffic from nondesignated tunnels, the reverse path forwarding check is enabled in the lose mode. For traffic received from nonexistent sources, the reverse path forwarding check fails for both the strict and loose modes.

Anti-spoofing is supported on VRF routing instances. To enable anti-spoofing for dynamic tunnels, include the ip-tunnel-rpf-check statement at the [edit routing-instances routing-instance-name routing-options forwarding-table] hierarchy level.

Topology

Figure 3 illustrates a sample network topology enabled with anti-spoofing protection. Routers R0, R1 and R2 are each connected to hosts Host0, Host1, and Host2, respectively. Two generic routing encapsulation (GRE) next-hop-based dynamic tunnels, Tunnel 1 and Tunnel 2 – connect Router R0 with Routers R1 and R2, respectively. The VRF routing instance is running between each router and its connected host devices.

Figure 3: Anti-Spoofing Protection for Next-Hop-Based Dynamic TunnelsAnti-Spoofing Protection for Next-Hop-Based Dynamic Tunnels

Taking as an example, three packets (Packets A, B, and C) are received on Router 0 from Router R2 through the next-hop-based dynamic GRE tunnel (Tunnel 2). The source IP address of these packets are 172.17.0.2 (Packet A), 172.18.0.2 (Packet B), and 172.20.0.2 (Packet C).

The source IP address of Packets A and B belong to Host 2 and Host 1, respectively. Packet C is a nonexistent source tunnel. The designated tunnel in this example is Tunnel 2, and the nondesignated tunnel is Tunnel 1. Therefore, the packets are processed as follows:

  • Packet A—Because the source is coming from a designated tunnel (Tunnel 2), Packet A passes the reverse path forwarding check and is processed for forwarding through Tunnel 2.

  • Packet B—Because the source is coming from Tunnel 1, which is a nondesinated tunnel, by default, Packet B fails the reverse path forwarding check in the strict mode. If loose mode is enabled, Packet B is allowed for forwarding.

  • Packet C—Because the source is a nonexistent tunnel source, Packet C fails the reverse path forwarding check, and the packet is not forwarded.

Configuration

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

Router R0

Router R1

R2

Procedure

Step-by-Step Procedure

The following example requires that you navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode in the CLI User Guide.

To configure Router R0:

  1. Configure Router R0’s interfaces, including the loopback interface.

  2. Assign the router ID and autonomous system number for Router R0.

  3. Configure IBGP peering between the routers.

  4. Configure OSPF on all the interfaces of Router R0, excluding the management interface.

  5. Configure RSVP on all the interfaces of Router R0, excluding the management interface.

  6. Enable next-hop-based dynamic GRE tunnel configuration on Router R0.

  7. Configure the dynamic GRE tunnel parameters from Router R0 to Router R1.

  8. Configure the dynamic GRE tunnel parameters from Router R0 to Router R2.

  9. Configure a virtual routing and forwarding (VRF) routing instance on Router R0, and assign the interface connecting to Host 1 to the VRF instance.

  10. Configure an external BGP session with Host 1 for the VRF routing instance.

  11. Configure anti-spoofing protection for the VRF routing instance on Router R0. This enables reverse path forwarding check for the next-hop-based dynamic tunnels, T1 and T2, on Router 0.

Results

From configuration mode, confirm your configuration by entering the show interfaces, show routing-options, show protocols, and show routing-options commands. If the output does not display the intended configuration, repeat the instructions in this example to correct the configuration.

Verification

Confirm that the configuration is working properly.

Verifying Basic Configuration

Purpose

Verify the OSPF and BGP peering status between the Router R0 and Routers R1 and R2.

Action

From operational mode, run the show ospf neighbor and show bgp summarycommands.

Meaning

The OSPF and BGP sessions are up and running between the Routers R0, R1, and R2.

Verifying Dynamic Tunnel Configuration

Purpose

Verify the status of the next-hop-based dynamic GRE tunnels between the Router R0 and Routers R1 and R2.

Action

From operational mode, run the show route table inet.3, and the show dynamic-tunnels database terse commands.

Meaning

The two next-hop-based dynamic GRE tunnels, Tunnel 1 and Tunnel 2, are up.

Verifying Anti-Spoofing Protection Configuration

Purpose

Verify that the reverse path forwarding check has been enabled on the VRF routing instance on Router R0.

Action

From the operational mode, run the show krt table VPN1.inet.0 detail.

Meaning

The configured reverse path forwarding check is enabled on the VRF routing instance in the strict mode.

Next-Hop-Based Dynamic Tunnel Localization Overview

Next-hop-based dynamic tunnels include generic routing encapsulation (GRE) tunnels and MPLS-over-UDP tunnels. These tunnels provide a scaling advantage over the interface-based tunnels. However, unlike the interface-based tunnels, the next-hop-based dynamic tunnels are anchorless in nature, where the forwarding information of the tunnels is distributed to the Packet Forwarding Engines (PFEs) on every line card on the device. This limits the maximum number of tunnels supported on the device to the tunnel capacity of a single line card. With the support for localization, you can configure next-hop-based dynamic tunnel localization to create the forwarding information only on the PFE of a line card that is designated as the anchor PFE. The PFEs on the other line cards on the device have state forwarding information to steer the packets to the anchor PFE. This provides a scaling advantage by increasing the maximum number of tunnels supported on a device.

Benefits of Next-Hop-Based Dynamic Tunnel Localization

Provides a scaling advantage by increasing the maximum number of tunnels supported on a device.

Use Cases for Next-Hop-Based Dynamic Tunnel Localization

  • The IPsec gateway devices that host a number of MS-MPC are used to terminate IPSec tunnels and are required to support moderate load. This support is affected with the use of next-hop-based dynamic tunnels when the scaling limit of the device is reached. With the localization of next-hop-based dynamic tunnels, the maximum number of the tunnels supported is increased, allowing the device to accommodate more tunnels at the cost of an extra fabric hop.

  • For Internet or VPN gateway devices, such as a virtual public cloud data center, there is a need for the gateway devices to communicate with a large number of servers. The data center servers are reachable through next-hop-based dynamic tunnels. The anchorless property of the dynamic tunnels limits the overall scaling numbers of the device. The gateway devices host multiple MPCs, with increased traffic demands. With the localization of the next-hop-based dynamic tunnels, the tunnels can be spread across the MPCs, thereby facilitating an increase in the tunnel scaling numbers.

Traffic Handling with Localization of Next-Hop-Based Dynamic Tunnels

With support for localization, the next-hop-based dynamic tunnel state is localized to an anchor Packet Forwarding Engine, and the other Packet Forwarding Engine has the tunnel state for steering traffic to the tunnel anchor.

Figure 4 illustrates the forwarding path of next-hop-based dynamic tunnels without localization.

Figure 4: Forwarding Path of Next-Hop-Based Dynamic Tunnels Without LocalizationForwarding Path of Next-Hop-Based Dynamic Tunnels Without Localization

Figure 5 illustrates the forwarding path of next-hop-based dynamic tunnels with localization.

Figure 5: Forwarding Path of Next-Hop-Based Dynamic Tunnels With LocalizationForwarding Path of Next-Hop-Based Dynamic Tunnels With Localization

Configuring Next-Hop-Based Dynamic Tunnels Localization

Localization support can be configured for newly created next-hop-based dynamic tunnels, or for existing non-local dynamic tunnels.

Configuring Localization for New Next-Hop-Based Dynamic Tunnels

The localization of next-hop-based dynamic tunnels uses a policy-based approach to specify prefix groups. In other words, route policies are used to apply the localization properties to the next-hop-based dynamic tunnels. Dynamic tunnel attribute profiles are created and configured under routing options for association with the prefix group using the policy.

  1. Creating dynamic tunnel profiles.

    The dynamic tunnel profile specifies the tunnel type and the anchor Packet Forwarding Engine information. Multiple dynamic tunnel profiles can be created for localization of the dynamic tunnels. The values for the dynamic tunnel type can be GRE, UDP, or BGP-SIGNAL.

    Although BGP-SIGNAL is not a valid tunnel type, on assigning BGP-SIGNAL as the tunnel type, the tunnels created from the BGP-signalled attributes are localized. When using BGP-SIGNAL, the tunnel type is decided based on the type advertised by BGP in its TLV. BGP-SIGNAL tunnels are always next-hop-based tunnels. The GRE tunnels created dynamically by BGP-SIGNAL are always next-hop-based, even if the user has manually configured tunnels created by GRE to use IFLs.

    The anchor Packet Forwarding Engine value is the line card of the anchor Packet Forwarding Engine, for example, pfe-x/y/0. This information can be viewed from the show interfaces terse pfe* command output.

    Sample Configuration:

  2. Associating dynamic tunnel profile to prefix list.

    Configuring a policy with dynamic-tunnel-attributes as the action associates the dynamic tunnel to the prefix list. The policy from action allows the creation of tunnel with specified attributes for any matching condition, such as a prefix range, community, or source address of BGP routes, and so on.

    Sample configuration:

  3. Including the tunnel policy under the forwarding table export policy.

    After the policy is configured, it is included in the forwarding table export policy for the parsing of the policy.

    Using the export-policy, the tunnel attributes get associated with the route. Whenever a route from BGP is queued for resolution, the forwarding table export policy is evaluated, and the tunnel attributes are obtained from the policy module based on the applied filters. The obtained tunnel attributes are then attached to the next hop in form of a tunnel composite next hop. The corresponding anchor forwarding structures, based on the Packet Forwarding Engine name and tunnel type, are created and sent to the forwarding table before a tunnel composite next hop is sent. However, if none of the attributes map to the tunnel composite next hop, then the forwarding structure is created on every Packet Forwarding Engine, similar to the non-localized dynamic tunnels.

    Sample configuration:

Configuring Localization for Existing Next-Hop-Based Dynamic Tunnels

CAUTION:

Making on the fly changes to dynamic tunnel attributes can result in an FPC crash due to high memory utilization. Hence, we recommend deactivating the dynamic-tunnels configuration before configuring localization.

To update tunnel attributes for existing next-hop-based dynamic tunnels, the following should be performed:

  1. Deactivate dynamic-tunnels configuration under the [edit routing-options] hierarchy level.

    Sample configuration:

  2. Change tunnel attributes as required.

  3. Activate dynamic-tunnels configuration under the [edit routing-options] hierarchy level.

    Sample configuration:

To configure localization for existing non-local next-hop-based dynamic tunnels:

CAUTION:

Making on the fly changes to configure localization for existing non-local next-hop-based dynamic tunnels can result in an FPC crash due to high memory utilization. Hence, we recommend deactivating the dynamic-tunnels configuration before configuring localization.

  1. Deactivate the dynamic-tunnels configuration at the [edit routing-options] hierarchy level.

  2. Create tunnel-attributes profile and add policy for localizing the dynamic tunnels, similar to new next-hop-based dynamic tunnels.

  3. Activate the dynamic-tunnels configuration.

Troubleshooting Localized Next-Hop-Based Dynamic Tunnels

With localization of next-hop-based dynamic tunnels, the tunnel composite next hops are associated with anchor Packet Forwarding Engine IDs. The following traceroute configuration statements at the [edit routing-options] hierarchy level help in troubleshooting the localized dynamic tunnels:

  • dynamic-tunnels traceoptions flag all—Tracking creation and deletion of tunnel in DTM.

  • resolution traceoptions flag tunnel—Tracking resolver operations on BGP route.

  • forwarding-table traceoptions flag all—Tracking tunnels sent to the kernel.

  • traceoptions flag all—Tracking of route learning process.

The following commands can be used to check if a route is using a localized next-hop-based dynamic tunnel:

  1. show route prefix extensive—To obtain the indirect next hop.

    For example:

  2. show krt indirect-next-hop index indirect-next-hop detail—To check for anchor Packet Forwarding Engine field in the detailed output of the indirect next hop.

    For example:

Unsupported Features for Next-Hop-Based Dynamic Tunnels Localization

Junos OS does not support the following functionality with localization for next-hop-based dynamic tunnels:

  • Chained composite next hops at the [edit routing-options forwarding-table chained-composite-next-hop ingress l3vpn] hierarchy level.

  • Anchor Packet Forwarding Engine resiliency.

    There is no resiliency support for next-hop-based dynamic tunnels with localization. After localization of the next-hop-based dynamic tunnels, the anchor Packer Forwarding Engine becomes the single entity for processing any given tunnel on the device. Although anchor Packer Forwarding Engine resiliency is not supported, for gateway devices, redundancy at the gateway device ensures that when the Packer Forwarding Engine to which the tunnel composite next hop is delegated goes down, the traffic must be rerouted to the redundant gateway device. The routing protocol process monitors the state of the Packer Forwarding Engine, and withdraws BGP advertisement of all the routes pointing to the tunnel composite next hops anchored on that Packer Forwarding Engine.

    Only the anchored Packet Forwarding Engine has the full-fledged tunnel composite next hop and all the other Packet Forwarding Engines have only steering entries to forward traffic to the anchor Packet Forwarding Engine. These steering entries are not withdrawn, when an anchor FPC goes down.

  • Localization of next-hop-based dynamic tunnels is not supported on logical systems.

  • IPv6 is not supported with localization of next-hop-based dynamic tunnels.

  • With localization, the show dynamic-tunnels database summary command does not display accurate tunnels summary when the state of the anchor Packet Forwarding Engine line card is not up. As a workaround, use the show dynamic-tunnels database and show dynamic-tunnels database terse command output.

Overview of Next-Hop-Based Dynamic Tunneling Using IP-Over-IP Encapsulation

SUMMARY 

Benefits

IP-over-IP tunneling provides the following benefits:

  • Alternative to MPLS over UDP—Can be used as an alternative to MPLS-over-UDP tunneling to provide IP service wherein there is a dedicated device per service.

  • Ability to steer specific traffic—Enables smooth migration when MPLS and IP networks co-exist because routes can be filtered to steer specific traffic over IP tunnels as opposed to MPLS tunnels.

  • Ability to support tunnels at increasing scale—Dynamic tunnel creation using BGP control plane can facilitate tunnel creation at scale.

What is IP-over-IP Dynamic Next Hop-based Tunneling?

An IP network contains edge devices and core devices. To achieve higher scale and reliability among these devices, you need to logically isolate the core network from the external network that the edge devices interact with, by using an overlay encapsulation.

Starting in Junos OS Release 20.3R1, we support an IP-over-IP encapsulation to facilitate IP overlay construction over IP transport network. IP over IP relies on a next hop-based infrastructure to support a higher scale. The feature supports IPv4 encapsulation of IPv6 and IPv4 payload. Among the other overlay encapsulations supported, IP-over-IP encapsulation is the only kind that allows:

  • transit devices to parse the inner payload and use inner packet fields for hash computation

  • customer edge devices to route traffic into and out of the tunnel without any throughput reduction

On MX Series routers, routing protocol daemon (RPD) sends the encapsulation header with tunnel composite nexthop and the Packet Forwarding Engine (PFE) finds the tunnel destination address and forwards the packet. On PTX Series routers and QFX10000 switches, RPD sends fully resolved next hop-based tunnel to the Packet Forwarding Engine. BGP protocol is used to distribute routes and signal dynamic tunnels.

The following illustration depicts how IPv4 or IPv6 traffic are sent from R-1 to R-5 through an IP over IP tunnel established between R-2 and R-4:

Example: Configuring Next-Hop-Based IP-Over-IP Dynamic Tunnels

SUMMARY Learn how to configure next hop-based tunnels by using IP-over-IP encapsulation.

Requirements

This example uses the following hardware and software components:

  • 4 MX Series routers and 1 PTX Series router.

  • Junos OS Release 20.3R1 or later version.

Overview

Starting in Junos OS Release 20.3R1, we support an IP-over-IP encapsulation to facilitate IP overlay construction over IP transport network. This example shows the establishment of unicast IP-over-IP tunnels between devices with a protocol next hop (PNH) through static configuration and a BGP protocol configuration.

To explain the difference in behavior between MX and PTX1000/PTX10000/QFX10000 devices, we have included a PTX Series router (R2) for this example. An IBGP is also configured between R2 and R4, which are connected over IP core, to exchange routes and signal dynamic tunnels.

Topology

Figure 1 illustrates an IP-over-IP scenario with 5 devices. In this topology, R1 is connected to R2 and R5 is connected to R4. R2 is a PTX Series device. R2 and R4 are connected to an R3 device in the center.

In this example, we are attempting to exchange routes from R1 to R5 and vice versa through IP-over-IP dynamic tunnels established between the R2 and R4 devices. Routes generated from R1 are exported to R2 and routes from R5 are exported to R4, by using an IS-IS export policy. You can configure a unicast IPIP tunnel Tunnel-01 from R2 to R4 and another tunnel Tunnel-02 from R4 to R2. Route prefixes that are generated within the network masks from the configured destination-networks of the peer device are used for creating the tunnel and traffic flows in the opposite direction of the routes in the tunnel..

Configuring IP-over-IP Dynamic Tunnels with a Protocol Next Hop

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

R1

R2

R3

R4

R5

Configuring IP-IP dynamic tunnels with a Protocol Next Hop

Step-by-Step Procedure
  1. Enter configuration mode on R1 and R5 devices.

  2. Configure basic device requirements such as VLAN, family iso, device interfaces, and loopback addresses. Routes are generated on the loopback addresses.

  3. Configure router-ID and autonomous system (AS) number to assign IP addresses to the routers and to enable BGP.

  4. Configure IS-IS protocols to enable IGP. Routes are exported from R1 to R2 and R5 to R4 devices by using IS-IS protocol.

  5. Enter commit on R1 and R5 devices from the configuration mode.

  6. Enter configuration mode on R2 and R4 to configure them.

  7. Configure basic device requirements such as VLAN, family iso, MPLS, router interfaces, and loopback addresses on R2 and R4.

  8. Configure an IBGP between R2 and R4.

  9. Configure an IS-IS export policy on R2 and R4 with BGP export to export routes from R1 to R2 and R5 to R4.

  10. Configure OSPF protocol between R2 and R4 for reachability.

  11. Configure a unicast IP-IP dynamic tunnel Tunnel-01 from R2 to R4 and Tunnel-02 from R4 to R2.

  12. Configure BGP and IS-IS export policy terms on R2 and policy statement to forward routes from BGP and RIB inet.0 to a fully resolved next hop (on R2-PTX device).

  13. Enter commit from the configuration mode on R2 and R4 devices.

  14. Configure VLAN device interfaces, loopback addresses, family mpls, router-ID, and autonomous system (AS) numbers.

  15. Configure OSPF protocols on R3 to R2 and R4 devices for reachability.

  16. Enter commit from configuration mode on R3 device.

Results

Verify your configuration by checking the below configurations from devices as follows:

Here�s how you can verify configurations on R2 device:

user@R2# show interfaces

user@R2# show routing-options

user@R2# show protocols

user@R2# show policy-options

Verification

Verify Dynamic Tunnel Database
Purpose

To verify the dynamic tunnel database information, use the show dynamic-tunnels database operational mode command.

Action
Meaning

The output indicates that an IPoIP tunnel is established with a tunnel encapsulation between R2 (192.168.0.21 source) and R4 (192.168.0.41 destination) on R2 device and another IPoIP tunnel is established a tunnel encapsulation between R4 (192.168.0.41 source) and R2 (192.168.0.21 destination) on R4 device.

Verify Route Table in inet.3
Purpose

To verify routes generated on inet.3 table, use the show route table inet.3 operational mode command.

Action
Meaning

The output indicates that R2 has received route 192.168.0.41/32 from R4 through the IPIP tunnel.

Verify BGP Routes Received and Advertised through Tunnel
Purpose

To verify BGP route prefixes received and advertised on R2 with BGP peer 192.168.0.41, use the show route receive-protocol bgp peer and show route advertising-protocol bgp peer operational mode commands.

Action
Meaning

The output indicates that R2 is receiving routes from R5 by using BGP, and R2 is advertising or sending routes from R1.

Example: Configure an IPoIP Tunnel in an MPLS Environment with LDP tunnel, Resolved Through inetcolor.0 Using Static Configuration

By default, MPLS has a higher preference than IP. For example, with MPLS and LDP configured among R2, R3, and R4, wherein R2 is reachable with R4 through LDP, routes from R2 will get resolved over LDP, instead of IP-over-IP, because of the higher preference.

If you prefer to have a particular route to resolve over the IP-over-IP instead of LDP, you can do so by creating an inetcolor table, where IP-over-IP has a higher preference and setting BGP to resolve that route over inetcolor table instead of inet3 table. The following example shows you how to do so by using static configuration.

CLI Quick Configuration

R2

R3

R4

Procedure

Step-by-Step Procedure
  1. Configure MPLS and LDP protocols between R2 and R3 and R3 and R4.

  2. Configure BGP protocols to enable BGP to import IPIP tunnel color and extended next-hop.

  3. Configure routing-options to enable these destination networks to add the color attribute 100, to create dynamic-tunnels using this attribute.

  4. Configure preference to assign a preference value.

  5. Configure policy-options to assign color red and IPIP to the chosen route filter. It is through policy-options that you can set router filters to select the routes you wish to resolve over IP-over-IP, add color and ipip to it.

  6. Configure policy-options of community to assign encapsulation to the ipip community and add attributes to the community red.

  7. Configure routing-options and policy-options to set a dynamic tunnel from R2 to R4 and R4 to R2, with the tunnel end points 192.168.0.41 and 192.168.0.21.

  8. Enter commit on R2, R3, and R4 from the configuration mode.

Results

Verify configurations set for scenario 2 from the configuration mode. Here�s how you can check the configurations on R2:

user@R2# show protocols

user@R2#show routing-options dynamic-tunnels Tunnel-01

user@R2#show policy-options policy-statement ipip-tunnel-color

user@R2# show policy-options policy-statement set-dynamic-tunnel-ep

user@R2# show policy-options community ipip

user@R2# show policy-options community red

Verification

Verify Route Resolution
Purpose

To verify the route resolution of routes in both inet.3 and inetcolor.0 tables, use the show route table inet.3 and show route table inetcolor.0 operational mode commands.

Action
Meaning

The R2 output indicates that on inet.3 table, the route 192.168.0.41 is getting resolved by LDP because of higher preference than IP-over-IP. On the other hand, in the newly created inetcolor.0 table, the route 192.168.0.41 is getting resolved through IP-over-IP tunnel with <c> attached.

The R4 output indicates that on inet.3 table, the route 192.168.0.21 is getting resolved by LDP because of higher preference, but on inetcolor.0 table, the route 192.168.0.21 is getting resolved through IP-over-IP tunnel with <c> attached.

Verify Dynamic Tunnel Database
Purpose

To verify the IP-over-IP dynamic tunnel created by routes in the inetcolor.0 table, use the show dynamic-tunnels database terse operational mode command.

Action
Meaning

The R2 output indicates that the route 192.168.0.41 has created a next-hop-based dynamic tunnel.

The R4 output indicates that the route 192.168.0.21 has created a next-hop-based dynamic tunnel.

Verify Route Next-Hops
Purpose

To verify all next-hops of the route that is set to resolve through IP-over-IP, us e the show route 192.168.0.51/32 extensive expanded-nh operational mode command.

Action
Meaning

The R2 output indicates the expanded next hop of 192.168.0.51/32 route. As R2 is a PTX Series router, it sends a protocol next hop and a composite next hop with a fully resolved tunnel.

The R4 output indicates the expanded next hop of 192.168.0.11/32 route. As R4 is an MX Series router, it sends a protocol next hop and an indirect next hop.

Example: Configure an IPoIP Tunnel with LDP tunnel in an MPLS Cloud, Resolved through inetcolor.0 Using BGP Signaling

In an MPLS environment with LDP enabled, BGP routes get resolved through LDP on inet.3 table because MPLS has higher preference than IP.

If you still prefer to have your routes resolved through IP-over-IP in the MPLS environment, you can do so by creating an inetcolor.0 table that assigns higher preference for IP-over-IP and resolve chosen routes through IP-over-IP. To enable this feature by using BGP, the route resolution is performed on the remote end device of the tunnel and with the export policy configured on the remote device, routes are received and advertised through BGP signaling. This example shows how to configure this by using the BGP protocol configuration.

Note:

If you are following this example after configuring IP-over-IP tunnel with LDP using static configuration, retain the MPLS and LDP configurations on R2, R3, and R4 devices and delete the remaining static configurations statements.

If you are following this example after configuring IP-over-IP dynamic tunnels with a protocol next hop, then set the MPLS and LDP configurations on R2, R3, and R4 devices before configuring the following statements. Remember to apply the export-tunnel-route policy before applying the export-isis policy from the first example.

CLI Quick Configuration

R2

R4

Procedure

Step-by-Step Procedure
  1. Enter configuration mode on R2 and R4 devices.

  2. Configure BGP protocols on R2 and R4 to enable nexthop tunnel and export routes from the remote end device.

  3. Configure routing options on R2 to create a tunnel from R2 to R4 and R4 to create a tunnel between R4 to R2, with color value and preference set to 100.

  4. Configure policy statement to export routes on R2 and R4 with an export filter and add tunnel attributes.

  5. Configure policy options to assign to add tunnel color, tunnel type, and a remote end point on R2 and R4.

  6. Enter commit from the configuration mode.

Results

You can check your configurations by using the following show commands from the configuration mode. Here�s how you can check the configurations on R2 device:

user@R2# show routing-options

user@R2# show policy-options policy-statement export-tunnel-route

user@R2# show policy-options tunnel-attribute tunnle-attr-02

user@R2# show protocols bgp group iBGP

Verification

Verify BGP Routes

Purpose

Verify routes sent by using BGP protocol.

Action

R2

R4

Meaning

The output shows the routes from BGP.

Verify Received Routes

Purpose

Verify the routes received through BGP by using the following operational mode commands.

Action

R2

R4

Meaning

The R2 and R4 output indicates the routes received on the devices.

Verify Advertised Routes

Purpose

Verify the routes advertised through BGP by using the following operational mode commands.

Action

R2

R4

Meaning

The R2 and R4 output indicates the routes being advertised.

Release History Table
Release
Description
19.2R1
Starting in Junos Release 19.2R1, on MX Series routers with MPCs and MICs, carrier supporting carrier (CSC) architecture can be deployed with MPLS-over-UDP tunnels carrying MPLS traffic over dynamic IPv4 UDP tunnels that are established between supporting carrier's PE devices.
18.3R1
Starting in Junos OS Release 18.3R1, MPLS-over-UDP tunnels are supported on PTX Series routers and QFX Series switches.
18.2R1
Starting in Junos OS Release 18.2R1, on PTX series routers and QFX10000 with unidirectional MPLS-over-UDP tunnels, you must configure the remote PE device with an input filter for MPLS-over-UDP packets, and an action for decapsulating the IP and UDP headers for forwarding the packets in the reverse tunnel direction.
17.4R1
Starting in Junos OS Release 17.4R1, on MX Series routers, the next-hop-based dynamic MPLS-over-UDP tunnels are signaled using BGP encapsulation extended community.
17.1R1
Starting in Junos OS Release 17.1, on MX Series routers with MPCs and MICs, the scaling limit of MPLS-over-UDP tunnels is increased.