Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Example: Configuring MPLS over GRE with IPsec Fragmentation and Reassembly

 

This example is based on a need to support a standard 1,500 byte MTU to virtual private network (VPN) clients that are supported by GRE over IPsec tunnels, when the WAN provider does not offer a Jumbo MTU option. The traffic forwarded over the 1500-byte WAN link can be dropped because the protocol encapsulation overhead (Layer 2, MPLS, GRE and IPsec) results in a frame that exceeds the WAN link MTU.

MTU related drops are mostly an issue for traffic that cannot be fragmented. For example, IP traffic that is marked as do-not-fragment, or Layer 2 VPN/VPLS traffic, which by its nature, cannot be fragmented. For performance reasons, many IPsec configurations block post encryption fragmentation, resulting in packet drop.

This document provide a solution to this problem by showing you how to configure an IPsec tunnel to perform post-fragmentation on traffic that is otherwise not able to be fragmented. In this case you trade encryption performance by forcing post-fragmentation against having to reduce the MTU of your VPN clients to prevent MTU related drops.

This example shows how to configure selective packet services mode using a single routing instance (the default one) to process VPN traffic into packet mode. In packet mode security zones are bypassed. This means that the Layer 2 and Layer 3 VRF interfaces are not placed into a security zone and no policy is needed to allow them to communicate through the internet zone.

Using the steps in this example you can perform IPsec encapsulated packet fragmentation on the outgoing physical interface of the sending device and reassembly on the receiving device before IPsec decryption.

Note

The reassembly of fragmented packets uses a lot of device resources, and the performance of the device will be slower than with nonfragmented traffic. When possible you should configure a jumbo MTU on the WAN interface to avoid the need for fragmentation. This example shows you how to provide a standard 1,500 byte MTU to client devices that block fragmentation when using IPsec over a WAN connection that does not offer jumbo support.

The topic includes the following sections:

Requirements

This example uses the following hardware and software components:

  • Two SRX Series Services Gateways

  • Junos OS Release 11.4 or later

    • This example has been revalidated on Junos OS Release 20.3R1

Note

For this example to work as documented you must ensure that your SRX configuration does not have any interfaces with family ethernet-switching enabled. Using family ethernet-switching puts the SRX device into mixed mode operation. This example is based on the route mode of operation. For details on route and mixed modes of operation see Understanding Layer 2 Interfaces on Security Devices. In addition, we tested this example with the factory default settings for the edit protocols l2-learning hierarchy.

Overview and Topology

This example includes the following configurations:

  • Configure interfaces for the appropriate protocol encapsulation and maximum transmission unit (MTU) value.

  • Apply firewall filter on the ge-0/0/0.10 interface to set the packet mode. Configure the WAN facing interface ge-0/0/1.0 with a 1,524 byte MTU.

  • Set a large MTU value to GRE and IPsec logical interfaces to avoid IPsec fragmentation at logical interfaces. The GRE encapsulated traffic is tunneled inside IPsec.

  • Add the MPLS family to the GRE interface gr-0/0/0, and apply firewall filters to enable packet mode.

  • Configure an IPsec tunnel on the device with the df-bit clear option in the IPsec VPN configuration to allow fragmentation of oversized IPsec packets on the outgoing ge-0/0/1.0 interface. This setting allows the SRX device to perform fragmentation post IPsec encryption for VPN client traffic that is marked with the do not fragment (DNF) bit. VPN client traffic that is not marked as DNF is fragmented prior to IPsec encryption to improve performance.

  • Configure all noncustomer- facing interfaces such as ge-0/0/1.0, gr-0/0/0.0, lo0.0, and st0.0 in a single security zone called “Internet”. A single security zone is used in this example to keep the focus on fragmentation issues with MPLS over GRE over IPSec. Security can be enhanced by placing the device into flow-mode for MPLS, and then placing the customer-facing interfaces into a zone. Once in a zone, security policies can control communications, and evoke advanced features like IDP and application recognition. For more information see Security Zones.

  • Configure a policy to permit all (intrazone) traffic.

  • Configure OSPF for lo0.0 address distribution, LDP for label distribution/MPLS transport, and IBGP with the inet-vpn and l2vpn families to support the VPN clients.

  • Configure two routing instances, one for a Layer 3 VPN and one for a Layer 2 VPLS service.

Figure 1 shows the topology for this example.

Figure 1: MPLS Over GRE Over IPsec Tunnels Example Topology
 MPLS Over GRE Over IPsec Tunnels Example
Topology

Table 1 provides a summary of the parameters used in this topology for the PE1 device. You can adapt the parameters for the PE2 device, or use the PE2 quick configuration provided below.

Table 1: Components of the Topology

Components

Description

PE1

PE1 SRX Series Firewall:

ge-0/0/0.10:

  • IP address: 192.168.0.1/24

  • Customer-facing L3VPN interface

  • input packet-mode-inet: inet family in packet mode

  • MTU: 4k

ge-0/0/2.11:

  • Customer-facing VPLS interface

  • vlan-vpls: VPLS encapsulation

  • MTU: 1,522

ge-0/0/1.0:

  • Outgoing interface

  • IP address: 172.16.13.1/30

  • MTU: 1,514

gr-0/0/0:

  • Core interface connecting to MPLS

  • IP address: 172.16.255.1/30

  • input packet-mode: MPLS family in packet mode

  • Inet MTU: 9k

lo0:

  • Logical Interface

  • IP address: 10.255.255.1/32

st0.0:

  • Tunnel interface

  • IP address: 172.16.0.1/30

  • Inet MTU: 9,178

  • df-bit clear — This option clears the do not fragment (DF) bit in the outgoing packet header

  • L3VPN— Routing instance for Layer3 VPN application

  • VPLS— Routing instance for VPLS application

Configuration

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

The configuration for the SRX1 (PE1) device:

The configuration for the SRX2 (PE2) device:

Step-by-Step Procedure

The following example requires you to navigate various levels in the configuration hierarchy. For instructions on how to do that, see Using the CLI Editor in Configuration Mode in the CLI User Guide.

To fragment the MPLS frame and reassemble the packet:

  1. Configure the physical Interfaces.
  2. Configure the logical Interfaces.
  3. Configure the firewall filters that are used to configure interfaces to work with packet mode.
  4. Configure the IKE and IPsec policies.
  5. Configure all noncustomer-facing interfaces in a single security zone and a policy to permit all (intrazone) traffic.
  6. Configure the OSPF protocol for lo0.0 address distribution, configure IBGP with the inet-vpn and l2vpn families. Also configure MPLS and LDP signaling.
  7. Configure the router ID and a static route to the remote end of the WAN link.
  8. Configure two routing instances, one for Layer 3 VPN and the other for the VPLS application.

Results

Display the results of the configuration:

Verification

Confirm that the configuration is working properly.

Verifying That the Physical and Logical Interfaces Are Up

Purpose

Verify that the physical and logical interfaces are up on the device.

Action

From operational mode on the SRX Series Services Gateway, enter the show interfaces terse command.

user@SRX1> show interfaces terse

Meaning

The output of the show interfaces terse command shows that all physical and logical interfaces used in this configuration are operational.

Verifying IPsec Security Associations

Purpose

Verify that the IKE and IPsec security associations are up on the device.

Action

From operational mode on the SRX Series Services Gateway, enter the show security ike security-association and show security ipsec security-association commands.

user@SRX1> show security ike security-associations
user@SRX1> show security ipsec security-associations

Meaning

The output shows the expected Up state for the IKE session and that an IPsec tunnel is successfully established.

Verifying OSPF and BGP

Purpose

Verify that OSPF and BGP are operating correctly over the GRE tunnel. Recall that the GRE tunnel is in turn routed over the IPsec tunnel verified in the previous step. Proper OSPF/BGP operation in this example indirectly verifies that traffic is able to pass over the GRE (and then the IPsec) tunnel. If desired, you can ping the GRE endpoint for added verification.

Action

From operational mode on the SRX Series Services Gateway, enter the show ospf neighbor and show bgp summary commands.

user@SRX1> show ospf neighbor
user@SRX1> show bgp summary

Meaning

The output confirms the expected OSPF neighbor state of full. This OSPF neighbor is estanlished over the GRE interface. Given OSPF is operational, you expect that the local SRX has learned the route to the remote SRX’s loopback address. This route allows the loopback based IBGP peering session to establish (over the GRE tunnel). The output of the show bgp summary command confirms the BGP session is in the established state, and that it is exchanging both L3VPN and L2VPN routes.

Verifying LDP Operation

Purpose

Verify that LDP is operating correctly over the GRE tunnel. LDP functions as the MPLS signaling protocol in this example.

Action

From operational mode on the SRX Series Services Gateway, enter the show ldp neighbor and show ldp session commands.

user@SRX1> show ldp neighbor
user@SRX1> show ldp session

Meaning

The output confirms the expected LDP neighbor relationship over the GRE interface. The output of the show ldp session command confirms successful session establishment to the remote SRX device’s loopback address. This allows LDP to exchange transport labels that in turn support MPLS forwarding for the VPN clients.

Verifying The VPLS Connection

Purpose

Verify that the VPLS connection is in an up state.

Action

From operational mode on the SRX Series Services Gateway, enter the show vpls connections command.

user@SRX1> show vpls connections

Meaning

The output shows the expected Up state for the VPLS connection. With the connection operational, the VPN client devices should be able to pass traffic.

Verifying End-to-End VPLS Connectivity for Large Packets with DNF Set

Purpose

Verify that the Layer 2 VPLS client devices are able to send 1500 byte frames with the DNF bit set. Because this is a Layer 2 service, fragmentation is not possible. As a result the DNF bit operates end-to-end. Recall that with the configuration in this example, such a setting results in the ingress SRX device fragmenting the IPsec packet after the traffic has been encrypted (post-fragmentation). The post-fragmentation occurs as the traffic egresses the WAN facing ge-0/0/1 interface.

Post-fragmentation forces the remote SRX device to reassemble the packet before it can perform decryption, which can impact forwarding performance for encrypted traffic. This is the expected behavior when the df-bit clear option is used. Demonstration this behavior is the reason for this NCE. The other df-bit options, namely df-bit copy and df-bit set, result in packet discard and generation of an ICMP error message for VPN packets that exceed the WAN MTU when the DNF bit is set by the VPN client.

Action

From operational mode on VPLS Host1, ping VPLS Host2 in a manner that generates a 1500 byte IP packet with the DNF bit set. When this traffic has the MPLS, GRE, and IPsec overhead added it exceeds the outgoing WAN interface’s MTU. Given that pre-fragmentation is blocked by virtue of this being a Layer 2 service (or in the case of the L3VPN client, by setting the DNF bit), such a packet forces post-fragmentation based on the setting of the df-bit clear option

The configuration and operation of the VPN client devices are outside the scope of this example. For testing, a MX router is used to act as the VPN clients. As a result the ping command demonstrated is based on the Junos CLI.

user@vpls-host1> ping 192.168.2.102 size 1472 do-not-fragment count 2

Meaning

The output shows the pings succeed. The 1480 bytes of echo traffic results in a 1500 byte IP packet when the 20 byte IP header is added. Thus, the results confirm the VPLS client device can exchange 1,500 byte packets over a WAN link with a 1,500 byte MTU, despite the encapsulation overhead. Recall that because this is a Layer 2 service, fragmentation is not possible and the DNF bit operates end-to-end. Using the DNF bit is significant when testing the L3VPN client, however, because the PE device is able to fragment IP traffic.

Verifying IP Fragmentation on the Outgoing Interface

Purpose

Verify that VPLS client traffic that exceeds the WAN MTU is fragmented on the outgoing ge-0/0/1.0 interface. Timing is important in this step because background OSPF, LDP, and BGP traffic causes the ge-0/0/0.0 interface counters to increment. The goal is to generate 100 1,500 byte packets from the VPLS host and then quickly compare the IPsec and interface statistics to confirm that approximately twice as many packets are seen on the outgoing WAN interface when compared to the counts on the IPsec tunnel.

Action

From operational mode on the SRX Series Services Gateway, clear both the IPsec and interface statistics with the clear interfaces statistics all and clear security ipsec statistics commands. Then generate 100 rapid pings with a 1,500 byte packet size between the VPLS endpoints. When the pings complete, display packet counts for the IPsec tunnel and the ge-0/0/1 interface with the show interfaces ge-0/0/1 detail and show security ipsec statistics commands.

user@SRX1> clear interfaces statistics all
user@SRX1> clear interfaces statistics all

Generate 100 rapid pings with a packet size of 1,500 bytes between the VPLS endpoints. This is not shown for brevity. Refer to the command in the previous step. Not shown here for brevity.

user@SRX1> show interfaces ge-0/0/1 detail
user@SRX1> show security ipsec statistics

Meaning

The output of the show interfaces ge-0/0/1.0 detail command shows that over200 packets have been sent and received. In contrast, the IPsec statistics confirm a count of around 100 packets. This confirms that each packet sent by the VPLS client was fragmented on the WAN-facing ge-0/0/1.0 interface.

Verifying The L3VPN

Purpose

Verify L3VPN Operation.

Action

From operational mode on the SRX Series Services Gateway, display the route to the remote L3VPN subnet with the show route command. Then generate pings to the remote L3VPN endpoint to verify connectivity.

user@SRX1> show route 192.168.1.0/24

Test connectivity from the local SRX to the remote VPN endpoint:

user@SRX1> ping 192.168.1.101 routing-instance L3VPN count 2
Note

In this configuration a ping from the local SRX to the local L3VPN client does not succeed. This relates to the use of packet mode and the lack of security zones for the VPN interfaces. As shown above, you are able to ping from the local SRX to the remote L3VPN destinations. Though not shown, a ping generated from the local L3VPN client to the local PE VRF interface is expected to succeed.

Test end-to-end connectivity for the L3VPN. Generate jumbo pings between L3VPN client endpoints. Recall that the L3VPN client is configured with a 4k MTU in this example. Once again we use a MX router to fill in for the L3VPN client, so Junos ping syntax is used:

user@l3vpn1> ping 192.168.1.101 size 3000 do-not-fragment count 2

Meaning

The output shows the route to the remote L3VPN client is correctly learned via BGP, and that it points to the GRE interface with an MPLS label operation. The results from ping testing confirm expected connectivity for the L3VPN even when sending 3,000 + byte pings with the DNF bit set.