Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
 

Example: Determining Member Health Using an MX Series Virtual Chassis Heartbeat Connection with Member Routers in Different Subnets

A heartbeat connection is an IP-based, bidirectional packet connection in an MX Series Virtual Chassis between the Virtual Chassis primary and backup routers. The heartbeat packets exchanged over this connection provide critical information about the availability and health of each member router. Starting in Junos OS Release 14.1, you can configure a heartbeat connection in an MX Series Virtual Chassis.

This example describes how to configure a heartbeat connection in an MX Series Virtual Chassis when the member routers reside in different subnets. For information about configuring a heartbeat connection when the Virtual Chassis member routers reside in the same subnet, see Example: Determining Member Health Using an MX Series Virtual Chassis Heartbeat Connection with Member Routers in the Same Subnet.

Requirements

This example uses the following software and hardware components:

  • Junos OS Release 14.1 and later releases

  • One MX240 Universal Routing Platform

  • One MX480 Universal Routing Platform

This configuration example has been tested using the software release listed and is assumed to work on all later releases.

Best Practice:

We recommend that you use the commit synchronize command to save any configuration changes to the Virtual Chassis. For an MX Series Virtual Chassis, the force option is the default and only behavior when you issue the commit synchronize command. Issuing the commit synchronize command for an MX Series Virtual Chassis configuration has the same effect as issuing the commit synchronize force command.

Before you configure a heartbeat connection for a Virtual Chassis:

  • Configure a Virtual Chassis consisting of two MX Series routers.

    See Example: Configuring Interchassis Redundancy for MX Series 5G Universal Routing Platforms Using a Virtual Chassis

    As part of the preprovisioned Virtual Chassis configuration shown in the configuration example, you must create and apply the member0-re0, member0-re1, member1-re0, and member1-re1 configuration groups for each member Routing Engine. Each configuration group includes a unique IP address for the management Ethernet interface (fxp0) on each Routing Engine.

    Note:

    When you create the preprovisioned Virtual Chassis configuration at the [edit virtual-chassis] hierarchy level, make sure you do not configure the no-split-detection statement to disable detection of a split in the Virtual Chassis. Using the no-split-detection statement is prohibited when you configure a Virtual Chassis heartbeat connection, and doing so causes the commit operation to fail.

  • Ensure TCP connectivity between the primary Routing Engine in the Virtual Chassis primary router (VC-Pp) and the primary Routing Engine in the Virtual Chassis backup router (VC-Bp).

    The Virtual Chassis heartbeat connection opens a proprietary TCP port numbered 33087 on the VC-Pp to listen for heartbeat messages. If your network design includes firewalls or filters, make sure the network allows traffic between TCP port 33087 on the VC-Pp and the dynamically allocated TCP port on the VC-Bp.

Overview

A heartbeat connection is an IP-based, bidirectional packet connection between the primary router and backup router in an MX Series Virtual Chassis. The member routers forming the heartbeat connection exchange heartbeat packets that provide critical information about the availability and health of each member router. During a disruption or split in the Virtual Chassis configuration, the heartbeat connection prevents the member routers from changing primary role roles unnecessarily, which can cause undesirable results.

This example configures a heartbeat connection for an MX Series Virtual Chassis in which the two member routers, each with dual Routing Engines installed, reside in different subnets. Member router gladius resides in subnet 10.4.0.0/16 and is the global primary router for the Virtual Chassis (VC-P). Member router trefoil resides in subnet 10.5.0.0/16 and is the global backup router (VC-B) for the Virtual Chassis. The heartbeat connection is configured between the primary Routing Engine in gladius (represented by VC-Pp or member0-re0) and the primary Routing Engine in trefoil (represented by VC-Bp or member1-re0).

Configuring a heartbeat connection for an MX Series Virtual Chassis when the member routers reside in different subnets consists of the following tasks:

  1. Configure two master-only IP addresses for the fxp0 management interface: one for the member routers in subnet 10.4.0.0, and a different address for the member routers in subnet 10.5.0.0.

  2. Configure a network path for the heartbeat connection to ensure that both member routers can reach each other’s networks.

    This example creates static routes to both subnet 10.4.0.0 and subnet 10.5.0.0 on each member router.

  3. Configure the Virtual Chassis heartbeat address for each member Routing Engine to cross-connect to the master-only IP address for the corresponding member Routing Engine in the other subnet.

  4. (Optional) Configure a nondefault value for the Virtual Chassis heartbeat timeout interval.

To establish the heartbeat connection in a two-member MX Series Virtual Chassis, you must configure the heartbeat address to establish the connection between the primary and backup member routers. To ensure consistent access to the primary Routing Engine in the Virtual Chassis primary router (VC-Pp) regardless of which Routing Engine is currently active, you set the heartbeat address to the previously configured master-only IP address for the fxp0 management interface.

Because the Virtual Chassis member routers in this example are in different subnets, you must configure a heartbeat address for each Routing Engine to enable a cross-connection to the master-only IP address for the corresponding Routing Engine in the other subnet, as shown in Table 1:

Table 1: Heartbeat Cross-Connections for Member Routers in Different Subnets

Routing Engine

Subnet

Cross-connected Routing Engine

Heartbeat Address

member0-re0

10.4.0.0/16

member1-re0

10.5.2.210

member0-re1

10.4.0.0/16

member1-re1

10.5.2.210

member1-re0

10.5.0.0/16

member0-re0

10.4.2.210

member1-re1

10.5.0.0/16

member0-re1

10.4.2.210

Topology

This example configures a heartbeat connection for an MX Series Virtual Chassis with member routers residing in different subnets. For redundancy, each member router is configured with two Virtual Chassis ports.

Table 2 shows the hardware and software configuration settings for each MX Series router in the Virtual Chassis.

Table 2: Components of the Sample MX Series Virtual Chassis with Member Routers in Different Subnets

Router Name

Hardware

Serial Number

Member ID

Role

Virtual Chassis Ports

Subnet

gladius

MX240 router with:

  • Primary RE-S-2000 Routing Engine in slot 0 (represented in example as member0-re0)

  • Backup RE-S-2000 Routing Engine in slot 1 (represented in example as member0-re1)

JN10C7135AFC

0

routing-engine (primary)

vcp-2/2/0

vcp-2/3/0

10.4.0.0/16

trefoil

MX480 router with:

  • Primary RE-S-2000 Routing Engine in slot 0 (represented in example as member1-re0)

  • Backup RE-S-2000 Routing Engine in slot 1 (represented in example as member1-re1)

JN115D117AFB

1

routing-engine (backup)

vcp-2/0/0

vcp-5/2/0

10.5.0.0/16

Configuration

To configure a heartbeat connection in an MX Series Virtual Chassis with member routers in different subnets, perform these tasks:

CLI Quick Configuration

To quickly configure a heartbeat connection for a Virtual Chassis with with member routers in different subnets, copy the following commands and paste them into the router terminal window:

Configuring a Consistent Management IP Address for Each Routing Engine

Step-by-Step Procedure

In addition to configuring a unique IP address for the fxp0 management interface on each Routing Engine when you first set up the Virtual Chassis, you must configure additional management IP addresses, known as the master-only address, to ensure consistent access to the fxp0 management interface on the primary Routing Engine in the Virtual Chassis primary router (VC-Pp). The master-only address is active only on the management interface for the VC-Pp. During a switchover, the master-only address moves to the new Routing Engine currently functioning as the VC-Pp.

Because the Virtual Chassis primary router and backup router in this example reside in different subnets, you must configure two different master-only IP addresses: one for the Routing Engines in subnet 10.4.0.0/16 (member0-re0 and member0-re1), and one for the Routing Engines in subnet 10.5.0.0/16 (member1-re0 and member1-re1). You then configure these master-only addresses as the subnet-specific heartbeat addresses to establish the heartbeat connection. For more information about the cross-connections in this example, see Table 1.

To configure the primary-only fxp0 IP address for each Routing Engine:

  1. From the console on member 0, configure the IP address for the fxp0 management interface for the Routing Engines in subnet 10.4.0.0/16.

  2. From the console on member 0, configure the IP address for the fxp0 management interface for the Routing Engines in subnet 10.5.0.0/16.

Results

From the console on the Virtual Chassis primary router, display the results of the configuration. For brevity, portions of the configuration unrelated to this procedure are replaced by an ellipsis (...).

For member0-re0:

For member0-re1:

For member1-re0:

For member1-re1:

If you are done configuring the device, enter commit from configuration mode.

Configuring Static Routes for Both Subnets on Each Routing Engine

Step-by-Step Procedure

You must configure secure and reliable routes for subnets 10.4.0.0/16 and 10.5.0.0/16 on each Routing Engine for the exchange of TCP/IP heartbeat packets. The heartbeat packets provide critical information about the availability and health of each member router.

The routes you configure for the heartbeat connection must be independent of the Virtual Chassis port links. Specifically, you must ensure that the primary Routing Engine in the Virtual Chassis backup router (VC-Bp) can make a TCP/IP connection to the master-only IP address of the primary Routing Engine in the Virtual Chassis primary router (VC-Pp).

This examples creates static routes to both subnets on each member Routing Engine to configure the heartbeat path. However, you can choose the method that best meets your needs to configure the heartbeat path for member routers in different subnets.

Best Practice:

We recommend that you use the router management interface (fxp0) as the heartbeat path. The management interface is generally available earlier than the line card interfaces, and is typically connected to a more secure network than the other interfaces.

To create static routes for subnets 10.4.0.0/16 and 10.5.0.0/16 on each Routing Engine:

  1. Log in to the console on member 0 (Virtual Chassis primary router).

  2. Configure the static routes for member0-re0.

  3. Configure the static routes for member0-re1.

  4. Configure the static routes for member1-re0.

  5. Configure the static routes for member1-re1.

Results

Display the results of the configuration. For brevity, portions of the configuration unrelated to this procedure are replaced by an ellipsis (...).

For member0-re0:

For member0-re1:

For member1-re0:

For member1-re1:

If you are done configuring the device, enter commit from configuration mode.

Configuring the Heartbeat Address and Heartbeat Timeout

Step-by-Step Procedure

To enable cross-connection between Virtual Chassis member routers in different subnets, you configure 10.5.2.210, which is the master-only IP address for the Routing Engines in subnet 10.5.0.0/16, as the heartbeat address for the Routing Engines in subnet 10.4.0.0/16 (member0-re0 and member0-re1). Conversely, you configure 10.4.2.210, which is the master-only IP address for the Routing Engines in subnet 10.4.0.0/16, as the heartbeat address for the Routing Engines in subnet 10.5.0.0/16 (member1-re0 and member1-re1). For more information about the cross-connections in this example, see Table 1.

Optionally, you can also configure a nondefault value for the heartbeat timeout interval. The heartbeat timeout is the maximum time within which a Virtual Chassis member router must respond to a heartbeat packet sent by the other member router. If you do not explicitly configure the heartbeat timeout interval, the default value (2 seconds) applies.

To configure the heartbeat address and heartbeat timeout:

  1. Log in to the console on member 0 (Virtual Chassis primary router).

  2. Configure the heartbeat address for each Routing Engine.

  3. (Optional) Configure a nondefault value for the heartbeat timeout interval.

Results

Display the results of the configuration.

For member0-re0:

For member0-re1:

For member1-re0:

For member1-re1:

If you are done configuring the device, enter commit from configuration mode.

Verification

To confirm that the Virtual Chassis heartbeat connection is working properly, perform these tasks:

Verifying the Virtual Chassis Heartbeat Connection

Purpose

Verify that the heartbeat connection between the Virtual Chassis member routers is properly configured and operational.

Action

Display the state of one or both member routers when a heartbeat connection is configured.

Meaning

For each member router, the command output displays the IP addresses of the local and remote member routers that form the heartbeat connection. The value Alive in the State field confirms that the primary Routing Engine in the specified member router is connected and has received a heartbeat response message. The Time field specifies the date and time of the last connection state change.

Verifying Use of the Heartbeat Connection During an Adjacency Split or Disruption

Purpose

Verify use of the heartbeat connection when an adjacency disruption or split is detected in the Virtual Chassis.

Action

Display the status of the member routers in the Virtual Chassis:

Meaning

The Status field for member ID 0 displays Heartbt, which indicates that this member router has used the heartbeat packet connection to maintain primary role roles during an adjacency disruption or split in the Virtual Chassis configuration. The Status field for member ID 1 displays Prsnt, which indicates that this member router is connected to the Virtual Chassis.

If a router is not currently connected to the Virtual Chassis, the Status field displays NotPrsnt.

Verifying Virtual Chassis Member Health from Heartbeat Statistics

Purpose

Use statistics collected by the heartbeat connection to verify the availability and health of each Virtual Chassis member router. You can also use the show virtual-chassis heartbeat detail command to determine the maximum latency and minimum latency in your network.

Action

Display and review the statistics collected by the heartbeat connection.

Meaning

In this example, the number of heartbeat request messages sent (Heartbeats sent) equals the number of heartbeat response messages received (Heartbeats received), with no heartbeat messages lost (Heartbeats lost/missed). This indicates that both member routers forming the heartbeat connection are available and operational. Any difference between Heartbeats sent and Heartbeats received appears in the Heartbeats lost/missed field.

The Maximum latency and Minimum latency fields measure the maximum and minimum number of seconds that elapse on the local router between transmission of a heartbeat request message and receipt of a heartbeat response message. In this example, the value 0 in the Maximum latency and Minimum latency fields indicates that there is no measurable network delay caused by this operation. You can use the Maximum latency value to determine whether you need to increase the heartbeat-timeout to a value higher than the default (2 seconds). If the maximum latency in your network is too high to accommodate a 2-second heartbeat-timeout value, increasing the heartbeat-timeout interval enables you to account for network delay when a Virtual Chassis adjacency disruption or split occurs.

Release History Table
Release
Description
14.1
Starting in Junos OS Release 14.1, you can configure a heartbeat connection in an MX Series Virtual Chassis.