Chassis Cluster on NFX250 NextGen Devices
A chassis cluster, where two devices operate as a single device, provides high availability (HA) on NFX250 NextGen devices. Chassis clustering involves the synchronizing of configuration files and the dynamic runtime session states between the devices, which are part of the chassis cluster setup.
NFX250 NextGen Chassis Cluster Overview
You can configure NFX250 NextGen devices to operate in cluster mode by connecting and configuring a pair of devices to operate like a single node, providing redundancy at the device, interface, and service level.
When two devices are configured to operate as a chassis cluster, each device becomes a node of that cluster. The two nodes back up each other, with one node acting as the primary device and the other node acting as the secondary device, ensuring stateful failover of processes and services when the system or hardware fails. If the primary device fails, the secondary device takes over the processing of traffic.
The nodes of a cluster are connected together through two links called control link and fabric link. The devices in a chassis cluster synchronize the configuration, kernel, and PFE session states across the cluster to facilitate high availability, failover of stateful services, and load balancing.
Control link—Synchronizes the configuration between the nodes. When you submit configuration statements to the cluster, the configuration is automatically synchronized over the control interface.
To create a control link in a chassis cluster, connect the ge-0/0/0 interface on one node to the ge-0/0/0 interface on the second node.
Note:You can use only the ge-0/0/0 interface to create a control link.
Fabric link (data link)—Forwards traffic between the nodes. Traffic arriving on a node that needs to be processed on the other node is forwarded over the fabric link. Similarly, traffic processed on a node that needs to exit through an interface on the other node is forwarded over the fabric link.
You can use any interface except the ge-0/0/0 to create a fabric link.
Chassis Cluster Modes
The chassis cluster can be configured in active/passive or active/active mode.
Active/passive mode—In active/passive mode, the transit traffic passes through the primary node while the backup node is used only in the event of a failure. When a failure occurs, the backup device becomes the primary device and takes over all forwarding tasks.
Active/active mode—In active/active mode, the transit traffic passes through both nodes all the time.
Chassis Cluster Interfaces
The chassis cluster interfaces include:
Redundant Ethernet (reth) interface—A pseudo-interface that includes a physical interface from each node of a cluster. The reth interface of the active node is responsible for passing the traffic in a chassis cluster setup.
A reth interface must contain, at minimum, a pair of Fast Ethernet interfaces or a pair of Gigabit Ethernet interfaces that are referred to as child interfaces of the redundant Ethernet interface (the redundant parent). If two or more child interfaces from each node are assigned to the redundant Ethernet interface, a redundant Ethernet interface link aggregation group can be formed.
Note:You can configure a maximum of 128 reth interfaces on NFX250 NextGen devices.
Control interface—An interface that provides the control link between the two nodes in the cluster. This interface is used for routing updates and for control plane signal traffic, such as heartbeat and threshold information that trigger node failover.
Note:By default, the ge-0/0/0 interface is configured as the dedicated control interface on NFX250 NextGen devices. Therefore, you cannot apply any configuration to ge-0/0/0 in HA mode.
Fabric interface—An interface that provides the physical connection between two nodes of a cluster. A fabric interface is formed by connecting a pair of Ethernet interfaces back-to-back (one from each node). The Packet Forwarding Engines of the cluster uses this interface to transmit transit traffic and to synchronize the runtime state of the data plane software. You must specify the physical interfaces to be used for the fabric interface in the configuration.
Chassis Cluster Limitation
Redundant LAG (RLAG) of reth member interfaces of the same node is not supported. A reth interface with more than one child interface per node is called RLAG.
Example: Configuring a Chassis Cluster on NFX250 NextGen Devices
This example shows how to set up chassis clustering on NFX250 NextGen devices.
Requirements
Before you begin:
Physically connect the two devices and ensure that they are the same NFX250 NextGen model.
Ensure that both devices are running the same Junos OS version
Remove all interface mapping for the control port ge-0/0/0 on both the nodes.
Connect the dedicated control port ge-0/0/0 on node 0 to the ge-0/0/0 port on node 1.
Connect the fabric port on node 0 to the fabric port on node 1.
Overview
Figure 1 shows the topology used in this example. This example shows how to set up basic active/passive chassis clustering. One device actively maintains control of the chassis cluster. The other device passively maintains its state for cluster failover capabilities in case the active device becomes inactive.
This example does not describe in detail miscellaneous configurations such as how to configure security features. They are essentially the same as they would be for standalone configurations.
Configuration
- Configuring a Chassis Cluster
- Configure Fabric interfaces
- Configure Redundant Groups and Redundant Interfaces
Configuring a Chassis Cluster
Step-by-Step Procedure
Configure the cluster ID on both the nodes and reboot the devices. A reboot is required to enter into cluster mode after the cluster ID and node ID are set.
Note:You must enter the operational mode to issue the commands on both devices.
user@host1> set chassis cluster cluster-id 1 node 0 reboot user@host2> set chassis cluster cluster-id 1 node 1 reboot
The cluster-id is the same on both devices, but the node ID must be different because one device is node 0 and the other device is node 1. The range for the cluster-id is 0 through 255 and setting it to 0 is equivalent to disabling cluster mode.
Verify that the chassis cluster is configured successfully:
user@host1> show chassis cluster status Monitor Failure codes: CS Cold Sync monitoring FL Fabric Connection monitoring GR GRES monitoring HW Hardware monitoring IF Interface monitoring IP IP monitoring LB Loopback monitoring MB Mbuf monitoring NH Nexthop monitoring NP NPC monitoring SP SPU monitoring SM Schedule monitoring CF Config Sync monitoring RE Relinquish monitoring Cluster ID: 1 Node Priority Status Preempt Manual Monitor-failures Redundancy group: 0 , Failover count: 0 node0 1 primary no no None node1 1 secondary no no Noneroot@host1> show chassis cluster information node0: -------------------------------------------------------------------------- Redundancy Group Information: Redundancy Group 0 , Current State: primary, Weight: 255 Time From To Reason Mar 15 11:33:47 hold secondary Hold timer expired Mar 15 11:34:03 secondary primary Only node present Chassis cluster LED information: Current LED color: Green Last LED change reason: No failures node1: -------------------------------------------------------------------------- Redundancy Group Information: Redundancy Group 0 , Current State: secondary, Weight: 255 Time From To Reason Mar 15 12:14:49 hold secondary Hold timer expired Chassis cluster LED information: Current LED color: Green Last LED change reason: No failures
After the chassis cluster is set up, you can enter the configuration mode and perform all the configurations on the primary node, node0.
Configure the host names and the out-of-band management IP addresses for nodes 0 and 1:
user@host1# set groups node0 system host-name NFX250NG-1 user@host1# set groups node0 interfaces fxp0 unit 0 family inet address 172.16.100.1/24
user@host2# set groups node1 system host-name NFX250NG-2 user@host2# set groups node1 interfaces fxp0 unit 0 family inet address 172.16.100.2/24
If you are accessing the device from a different subnet other than the one configured for the out-of-band management, then set up a static route:
user@host1# set routing-options static route 198.51.100.0/24 next-hop 172.16.0.0 user@host1# set routing-options static route 203.0.113.0/24 next-hop 172.16.0.0
Configure a backup router to access the router from an external network for the out-of-band management
user@host1# set groups node0 system backup-router 172.16.0.0 user@host1# set groups node0 system backup-router destination 172.0.0.0/8 user@host1# set groups node0 system backup-router destination 203.0.0.0/8 user@host1# set groups node1 system backup-router 172.16.0.0 user@host1# set groups node1 system backup-router destination 172.0.0.0/8 user@host1# set groups node1 system backup-router destination 203.0.0.0/8
Configure Fabric interfaces
Step-by-Step Procedure
The ge-0/0/0 interface is a pre-defined control link. Therefore, you should select any other interface on the device to configure a fabric interface. For example, in the below configuration, ge-0/0/1 is used as the fabric interface.
Connect one end of the Ethernet cable to ge-0/0/1 on NFX250NG-1 device and the other end of the cable to ge-0/0/1 on NFX250NG-2 device.
Map physical LAN to virtual WAN port:
user@host1> set vmhost virtualization-options interfaces ge-8/0/1 user@host1> set vmhost virtualization-options interfaces ge-1/0/1
Configure front panel (L2) interfaces corresponding to fabric interface:
user@host1# set interfaces ge-0/0/1 mtu 9192 user@host1# set interfaces ge-0/0/1 unit 0 family ethernet-switching interface-mode access user@host1# set interfaces ge-0/0/1 unit 0 family ethernet-switching vlan members vlan100
user@host1# set interfaces sxe-0/0/0 mtu 9192 user@host1# set interfaces sxe-0/0/0 unit 0 family ethernet-switching interface-mode trunk user@host1# set interfaces sxe-0/0/0 unit 0 family ethernet-switching vlan members vlan100 user@host1# set vlans vlan100 vlan-id 100
Configure L3 interfaces as fabric member:
user@host1# set chassis cluster fabric-member ge-1/0/1 vlan-id 100 user@host1# set interfaces fab0 fabric-options member-interfaces ge-1/0/1 user@host1# set groups fab chassis cluster fabric-member ge-1/0/1 vlan-id 100 user@host1# set groups fab chassis cluster fabric-member ge-8/0/1 vlan-id 100 user@host1# set groups fab interfaces fab0 fabric-options member-interfaces ge-1/0/1 user@host1# set groups fab interfaces fab1 fabric-options member-interfaces ge-8/0/1 user@host1# set groups fab vmhost virtualization-options interfaces ge-1/0/1 user@host1# set groups fab vmhost virtualization-options interfaces ge-8/0/1
Configure data path for fabric interfaces:
user@host1# set groups fab interfaces sxe-7/0/0 unit 0 family ethernet-switching vlan members vlan100 user@host1# set groups fab interfaces sxe-0/0/0 unit 0 family ethernet-switching vlan members vlan100 user@host1# set groups fab interfaces ge-0/0/9 mtu 9000 user@host1# set groups fab interfaces ge-0/0/9 unit 0 family ethernet-switching interface-mode access user@host1# set groups fab interfaces ge-0/0/9 unit 0 family ethernet-switching vlan members vlan100 user@host1# set groups fab interfaces ge-7/0/9 mtu 9000 user@host1# set groups fab interfaces ge-7/0/9 unit 0 family ethernet-switching interface-mode access user@host1# set groups fab interfaces ge-7/0/9 unit 0 family ethernet-switching vlan members vlan100 user@host1# set groups fab vlan vlan100 vlan-id 100 user@host1# set apply-groups fab
Configure port peering for fabric and reth members. Port peering ensures that when a LAN interface controlled by the Layer 2 dataplane (FPC0) fails, the corresponding interface on the Layer 3 dataplane (FPC1) is marked down and vice versa. This helps in the failover of the corresponding redundant group to the secondary node.
user@host1# set groups node1 chassis cluster redundant-interface ge-8/0/1 mapping-interface ge-7/0/1 user@host1# set groups node0 chassis cluster redundant-interface ge-1/0/1 mapping-interface ge-0/0/1
Enable the system to perform control link recovery automatically. After it determines that the control link is healthy, the system issues an automatic reboot on the node that was disabled when the control link failed. When the disabled node reboots, it rejoins the cluster.
user@host1# set chassis cluster control-link-recovery
Configure Redundant Groups and Redundant Interfaces
Step-by-Step Procedure
-
Configure redundancy groups 1 and 2. Both
redundancy-group 1andredundancy-group 2control the data plane and include the data plane ports. Each node has interfaces in a redundancy group. As part of redundancy group configuration, you must also define the priority for control plane and data plane—which device is preferred for the control plane, and which device is preferred for the data plane. For chassis clustering, higher priority is preferred. The higher number takes precedence.In this configuration,
node 0is the active node as it is associated withredundancy-group 1. reth0 is member ofredundancy-group 1and reth1 is member ofredundancy-group 2. You must configure all changes in the cluster through node 0. If node 0 fails, then node 1 will be the active node.user@host1# set chassis cluster reth-count 4 user@host1# set chassis cluster redundancy-group 1 node 0 priority 200 user@host1# set chassis cluster redundancy-group 1 node 1 priority 100 user@host1# set chassis cluster redundancy-group 2 node 0 priority 200 user@host1# set chassis cluster redundancy-group 2 node 1 priority 100 user@host1# set chassis cluster redundancy-group 1 preempt user@host1# set chassis cluster redundancy-group 2 preempt
Map physical LAN to virtual WAN port for reth members:
user@host1# set vmhost virtualization-options interfaces ge-1/0/3 user@host1# set vmhost virtualization-options interfaces ge-1/0/4 user@host1# set vmhost virtualization-options interfaces ge-8/0/3 user@host1# set vmhost virtualization-options interfaces ge-8/0/4
Configure front panel (L2) interfaces corresponding to reth interface:
user@host1# set interfaces ge-0/0/3 unit 0 family ethernet-switching interface-mode access user@host1# set interfaces ge-0/0/3 unit 0 family ethernet-switching vlan members vlan300
user@host1# set interfaces ge-0/0/4 unit 0 family ethernet-switching interface-mode access user@host1# set interfaces ge-0/0/4 unit 0 family ethernet-switching vlan members vlan400
user@host1# set interfaces ge-7/0/3 unit 0 family ethernet-switching interface-mode access user@host1# set interfaces ge-7/0/3 unit 0 family ethernet-switching vlan members vlan300
user@host1# set interfaces ge-7/0/4 unit 0 family ethernet-switching interface-mode access user@host1# set interfaces ge-7/0/4 unit 0 family ethernet-switching vlan members vlan400
user@host1# set interfaces sxe-0/0/1 unit 0 family ethernet-switching interface-mode trunk user@host1# set interfaces sxe-0/0/1 unit 0 family ethernet-switching vlan members vlan300 user@host1# set interfaces sxe-0/0/1 unit 0 family ethernet-switching vlan members vlan400
user@host1# set interfaces sxe-7/0/1 unit 0 family ethernet-switching interface-mode trunk user@host1# set interfaces sxe-7/0/1 unit 0 family ethernet-switching vlan members vlan300 user@host1# set interfaces sxe-7/0/1 unit 0 family ethernet-switching vlan members vlan400
user@host1# set vlans vlan300 vlan-id 300 user@host1# set vlans vlan400 vlan-id 400
Configure WAN (L3) interfaces as reth member:
user@host1# set interfaces ge-1/0/3 gigether-options redundant-parent reth0 user@host1# set interfaces ge-8/0/3 gigether-options redundant-parent reth0 user@host1# set interfaces ge-1/0/4 gigether-options redundant-parent reth1 user@host1# set interfaces ge-8/0/4 gigether-options redundant-parent reth1
Configure reth interfaces:
Configure reth0:
user@host1# set interfaces reth0 vlan-tagging user@host1# set interfaces reth0 redundant-ether-options redundancy-group 1 user@host1# set interfaces reth0 unit 0 vlan-id 300 user@host1# set interfaces reth0 unit 0 family inet address 192.0.2.0/24
Configure reth1:
user@host1# set interfaces reth1 vlan-tagging user@host1# set interfaces reth1 redundant-ether-options redundancy-group 2 user@host1# set interfaces reth1 unit 0 vlan-id 400 user@host1# set interfaces reth1 unit 0 family inet address 198.51.100.0/24
Configure interface monitoring for reth interfaces members:
user@host1# set chassis cluster redundancy-group 1 interface-monitor ge-1/0/3 weight 255 user@host1# set chassis cluster redundancy-group 1 interface-monitor ge-8/0/3 weight 255 user@host1# set chassis cluster redundancy-group 2 interface-monitor ge-1/0/4 weight 255 user@host1# set chassis cluster redundancy-group 2 interface-monitor ge-8/0/4 weight 255
Configure port peering for reth interface members:
user@host1# set groups node1 chassis cluster redundant-interface ge-8/0/3 mapping-interface ge-7/0/3 user@host1# set groups node1 chassis cluster redundant-interface ge-8/0/4 mapping-interface ge-7/0/4 user@host1# set groups node0 chassis cluster redundant-interface ge-1/0/3 mapping-interface ge-0/0/3 user@host1# set groups node0 chassis cluster redundant-interface ge-1/0/4 mapping-interface ge-0/0/4
Configure security policies to allow traffic from LAN to WAN, and from WAN to LAN:
user@host1# set security policies default-policy permit-all user@host1# set security zones security-zone trust host-inbound-traffic system-services all user@host1# set security zones security-zone trust host-inbound-traffic protocols all user@host1# set security zones security-zone trust interfaces all
Verification
Verifying Chassis Cluster Status
Purpose
Verify the status of the chassis cluster and its interfaces.
Action
From operational mode, issue the following commands:
Verify the status of the cluster:
root@host1> show chassis cluster status Monitor Failure codes: CS Cold Sync monitoring FL Fabric Connection monitoring GR GRES monitoring HW Hardware monitoring IF Interface monitoring IP IP monitoring LB Loopback monitoring MB Mbuf monitoring NH Nexthop monitoring NP NPC monitoring SP SPU monitoring SM Schedule monitoring CF Config Sync monitoring RE Relinquish monitoring IS IRQ storm Cluster ID: 1 Node Priority Status Preempt Manual Monitor-failures Redundancy group: 0 , Failover count: 1 node0 1 primary no no None node1 1 secondary no no None Redundancy group: 1 , Failover count: 1 node0 200 primary yes no None node1 100 secondary yes no None Redundancy group: 2 , Failover count: 1 node0 200 primary yes no None node1 100 secondary yes no NoneVerify the status of the redundancy groups:
root@host1> show chassis cluster information node0: -------------------------------------------------------------------------- Redundancy Group Information: Redundancy Group 0 , Current State: primary, Weight: 255 Time From To Reason Jun 8 11:24:14 hold secondary Hold timer expired Jun 8 11:24:30 secondary primary Only node present Redundancy Group 1 , Current State: primary, Weight: 255 Time From To Reason Jun 8 11:24:14 hold secondary Hold timer expired Jun 8 11:24:30 secondary primary Only node present Redundancy Group 2 , Current State: primary, Weight: 255 Time From To Reason Jun 8 11:24:14 hold secondary Hold timer expired Jun 8 11:24:30 secondary primary Only node present Chassis cluster LED information: Current LED color: Green Last LED change reason: No failures node1: -------------------------------------------------------------------------- Redundancy Group Information: Redundancy Group 0 , Current State: secondary, Weight: 255 Time From To Reason Jun 8 11:25:24 hold secondary Hold timer expired Redundancy Group 1 , Current State: secondary, Weight: 255 Time From To Reason Jun 8 11:25:24 hold secondary Hold timer expired Redundancy Group 2 , Current State: secondary, Weight: 255 Time From To Reason Jun 8 11:25:23 hold secondary Hold timer expired Chassis cluster LED information: Current LED color: Green Last LED change reason: No failuresVerify the status of the interfaces:
root@host1> show chassis cluster interfaces Control link status: Up Control interfaces: Index Interface Monitored-Status Internal-SA Security 0 em1 Up Disabled Disabled Fabric link status: Up Fabric interfaces: Name Child-interface Status Security (Physical/Monitored) fab0 ge-1/0/1 Up / Up Disabled fab0 fab1 ge-8/0/1 Up / Up Disabled fab1 Redundant-ethernet Information: Name Status Redundancy-group reth0 Up 1 reth1 Up 2 reth2 Down Not configured reth3 Down Not configured Redundant-pseudo-interface Information: Name Status Redundancy-group lo0 Up 0 Interface Monitoring: Interface Weight Status Redundancy-group (Physical/Monitored) ge-8/0/3 255 Up / Up 1 ge-1/0/3 255 Up / Up 1 ge-8/0/4 255 Up / Up 2 ge-1/0/4 255 Up / Up 2Verify the status of the port-peering interfaces:
root@host1> show chassis cluster port-peering node0: -------------------------------------------------------------------------- Port peering interfaces: Backend L3 Mapped Peer L2 Interface Status Interface Status ge-1/0/3 Up ge-0/0/3 Up ge-1/0/4 Up ge-0/0/4 Up ge-1/0/1 Up ge-0/0/1 Up node1: -------------------------------------------------------------------------- Port peering interfaces: Backend L3 Mapped Peer L2 Interface Status Interface Status ge-8/0/3 Up ge-7/0/3 Up ge-8/0/4 Up ge-7/0/4 Up ge-8/0/1 Up ge-7/0/1 Up