Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Navigation  Back up to About Overview 
ContentIndex
  
[+] Expand All
[-] Collapse All

No index entries found.

Example: Configuring High Availability for the Midsize Enterprise Campus

This example details the steps required on the devices in access, aggregation, core, and edge layers to configure them to meet the high availability goals described in Understanding the Design of the Midsize Enterprise Campus Solution.

Requirements

Table 7 shows the hardware and software requirements for this example. Table 8 shows the scaling and performance targets used for this example.

Table 7: Hardware and Software Requirements

Hardware

Device Name

Software

MX240

cs-edge-r01, cs-edge-r02

13.2 R2.4

SRX650

cs-edge-fw-01, cs-edge-fw02

12.1 X44-D39.4

EX9214

cs-core-sw01, cs-core-sw02

13.2 R3.7

EX4550

cs-agg-01

12.3 R3.4

EX2200

cs-2200-ab5

12.3 R3.4

EX3300

cs-3300-ab4

12.3 R3.4

EX4200

cs-4200-ab1

12.3 R3.4

EX4300

cs-4300-ab2, cs-4300-ab3

13.2 X51-D21.1

Table 8: Node Features and Performance/Scalability

Node

Features

Performance/Scalability Target Value

Edge (MX240, SRX650)

MC-LAG, OSPF, BGP, IRB

3k IPv4

Core (EX9214 )

VLANs, MC-LAG, LAG, IGMP snooping, OSPF, PIM-SM, IGMP, DHCP relay, IRB

3k IPv4 routes

128k MAC table entries

16k ARP entries

Aggregation (EX4550)

VLANs, LAG, IGMP snooping, OSPF, PIM-SM, IGMP, DHCP relay, RVI

3k IPv4 routes

5 IGMP groups

Access (EX3300, EX4300, EX4200)

VLANs, LAG, 802.1X, IGMP snooping, DHCP snooping, ARP inspection, IP source guard

55k MAC table entries

13k 8021.x users

5 IGMP groups

The configuration procedures that follow assume that all physical cabling has been completed and that the devices have been initially configured.

Overview and Topology

Figure 11 shows the topology used for this example.

Figure 11: High Availability Topology

High Availability Topology

In this topology, all access switches in location A are in a Virtual Chassis configuration. Link aggregation is configured on the uplink ports to the EX9214 switches, giving each Virtual Chassis a physical connection to each EX9214 switch. Similarly, the access switches in location B are in a Virtual Chassis configuration and have link aggregation configured on the uplink ports to the EX4550 Virtual Chassis, giving each access Virtual Chassis a physical link to each member of the EX4550 Virtual Chassis.

For node redundancy in the core and edge layers:

  • The EX9214 switches are in an active/active MC-LAG configuration. MC-LAG interfaces ae1, ae2, and ae3 connect to the access switches in location A, and MC-LAG interfaces ae11 and ae12 connect to the SRX650 gateways.
  • The SRX650 gateways are in an active/standby chassis cluster configuration, with redundant Ethernet interfaces reth0 and reth1 connecting to the EX9214 core switches and the MX240 edge routers.
  • The MX240 routers are in an active/active MC-LAG configuration, with MC-LAG interfaces ae1 and ae3 connecting to the SRX650 gateways.

Configuring the Access Switches for High Availability

This section provides step-by-step procedures for configuring the access switches in the access layer for high availability. It uses configuring the cs-4200-ab1 Virtual Chassis as an example—you can use the same basic procedures to configure the other Virtual Chassis in the access layer.

To configure the access switches for high availability:

Configure the Virtual Chassis

Step-by-Step Procedure

To configure the Virtual Chassis:

  • Define the members of the Virtual Chassis and their roles.
    [edit]
    user@cs-4200-ab1# set virtual-chassis preprovisioned user@cs-4200-ab1# set virtual-chassis member 0 role line-card user@cs-4200-ab1# set virtual-chassis member 0 serial-number BP0213230308 user@cs-4200-ab1# set virtual-chassis member 1 role routing-engine user@cs-4200-ab1# set virtual-chassis member 1 serial-number BP0213260624 user@cs-4200-ab1# set virtual-chassis member 2 role routing-engine user@cs-4200-ab1# set virtual-chassis member 2 serial-number BP0213260668 user@cs-4200-ab1# set virtual-chassis member 3 role line-card user@cs-4200-ab1# set virtual-chassis member 3 serial-number BP0213260540 user@cs-4200-ab1# set virtual-chassis member 4 role line-card user@cs-4200-ab1# set virtual-chassis member 4 serial-number BP0213260532 user@cs-4200-ab1# set virtual-chassis member 5 role line-card user@cs-4200-ab1# set virtual-chassis member 5 serial-number BP0213230346 user@cs-4200-ab1# set virtual-chassis member 6 role line-card user@cs-4200-ab1# set virtual-chassis member 6 serial-number FP0213313963 user@cs-4200-ab1# set virtual-chassis member 7 role line-card user@cs-4200-ab1# set virtual-chassis member 7 serial-number BP0213310009 user@cs-4200-ab1# set virtual-chassis member 8 role line-card user@cs-4200-ab1# set virtual-chassis member 8 serial-number BP0213260607 user@cs-4200-ab1# set virtual-chassis member 9 role line-card user@cs-4200-ab1# set virtual-chassis member 9 serial-number BP0213230403

Configure the LAG Interface Towards the Core or Aggregation Layer

Step-by-Step Procedure

The following procedure shows how to configure ae1 on cs-4200-ab1. You can use the same procedure for the LAGs on the other switches, substituting the information shown in Table 9.

Table 9: LAG Interfaces in the Access Layer

Virtual Chassis

LAG Name

Description String

Member Interfaces

cs-4200-ab1

ae1

“MCLAG towards core-sw1 and core-sw2”

xe-1/1/0, xe-2/1/0, xe-3/1/0, xe-4/1/0

cs-4300-ab2

ae2

“MCLAG towards core-sw1 and core-sw2”

xe-0/2/0, xe-1/2/0, xe-2/2/0, xe-4/2/0

cs-4300-ab3

ae3

“MCLAG towards core-sw1 and core-sw2”

xe-0/2/0, xe-3/2/0

cs-3300-ab4

ae4

“MCLAG towards cs-agg”

xe-0/1/0, xe-1/1/0

cs-2200-ab5

ae5

“MCLAG towards cs-agg”

ge-0/0/23, ge-0/1/0, ge-1/0/23, ge-1/1/0

To configure ae1 on cs-4200-ab1:

  1. Specify the number of LAG interfaces on the device.
    {master:1}[edit]
    user@cs-4200-ab1# set chassis aggregated-devices ethernet device-count 3
  2. Configure the LAG settings for ae1.
    {master:1}[edit]
    user@cs-4200-ab1# set interfaces ae1 description "MCLAG towards core-sw1 and core-sw2"
    user@cs-4200-ab1# set interfaces ae1 aggregated-ether-options lacp active
    user@cs-4200-ab1# set interfaces ae1 aggregated-ether-options lacp periodic fast
    
  3. Specify the members of the LAG.
    {master:1}[edit]
    user@cs-4200-ab1# set interfaces xe-1/1/0 ether-options 802.3ad ae1
    user@cs-4200-ab1# set interfaces xe-2/1/0 ether-options 802.3ad ae1
    user@cs-4200-ab1# set interfaces xe-3/1/0 ether-options 802.3ad ae1
    user@cs-4200-ab1# set interfaces xe-4/1/0 ether-options 802.3ad ae1
  4. Configure the LAG interface as a trunk interface with membership in all VLANs.

    The configuration statements used on an EX4300 switch differ from the statements used on the other EX Series switches. Examples of both configurations are shown.

    On EX2200, EX3300, and EX4200 switches, enter:

    {master:1}[edit]
    user@cs-4200-ab1# set interfaces ae1 unit 0 family ethernet-switching port-mode trunk
    user@cs-4200-ab1# set interfaces ae1 unit 0 family ethernet-switching vlan members all
    

    On EX4300 switches, enter:

    {master:1}[edit]
    user@cs-4300-ab3# set interfaces ae3 unit 0 family ethernet-switching interface-mode trunk
    user@cs-4300-ab3# set interfaces ae3 unit 0 family ethernet-switching vlan members all
    

Configure the High Availability Software

Step-by-Step Procedure

To enable graceful Routing Engine switchover (GRES), nonstop active routing (NSR), and nonstop bridging (NSB):

  • Enter the following configuration statements:

    On EX2200, EX3300, and EX4200 switches, enter:

    {master:1}[edit]
    user@cs-4200-ab1# set chassis redundancy graceful-switchover
    user@cs-4200-ab1# set ethernet-switching-options nonstop-bridging
    user@cs-4200-ab1# set routing-options nonstop-routing
    

    On EX4300 switches, enter:

    {master:1}[edit]
    user@cs-4300-ab3# set chassis redundancy graceful-switchover
    user@cs-4300-ab3# set protocols layer2-control nonstop-bridging
    user@cs-4300-ab31#  set routing-options nonstop-routing
    

Configuring the Aggregation Switches for High Availability

In location B, two EX4550 switches in a Virtual Chassis configuration function as the aggregation switch. For link redundancy, LAG interfaces ae4 and ae5 connect the aggregation switch to the access switches cs-3300-ab4 and cs-2200-ab5, respectively.

To configure the aggregation switches for high availability:

Configure the EX4550 Virtual Chassis

Step-by-Step Procedure

To configure the Virtual Chassis:

  • Enter the following commands:
    [edit]
    user@cs-agg-01# set virtual-chassis preprovisioned
    user@cs-agg-01# set virtual-chassis no-split-detection
    user@cs-agg-01# set virtual-chassis member 0 role routing-engine
    user@cs-agg-01# set virtual-chassis member 0 serial-number LX0213439586
    user@cs-agg-01# set virtual-chassis member 1 role routing-engine
    user@cs-agg-01# set virtual-chassis member 1 serial-number LX0213449606
    

Configure the LAG Interfaces Towards the Access Layer

Step-by-Step Procedure

To configure the LAG interfaces:

  1. Specify the number of LAG interfaces on the device.
    {master:0}[edit]
    user@cs-agg-01# set chassis aggregated-devices ethernet device-count 10
  2. Configure ae4.
    {master:0}[edit]
    user@cs-agg-01# set interfaces ae4 aggregated-ether-options lacp active
    user@cs-agg-01# set interfaces ae4 aggregated-ether-options lacp periodic fast
    user@cs-agg-01# set interfaces ae4 unit 0 family ethernet-switching port-mode trunk
    user@cs-agg-01# set interfaces ae4 unit 0 family ethernet-switching vlan members all
    user@cs-agg-01# set interfaces xe-0/0/30 ether-options 802.3ad ae4
    user@cs-agg-01# set interfaces xe-1/0/30 ether-options 802.3ad ae4
    
  3. Configure ae5.
    {master:0}[edit]
    user@cs-agg-01# set interfaces ae5 aggregated-ether-options lacp active
    user@cs-agg-01# set interfaces ae5 aggregated-ether-options lacp periodic fast
    user@cs-agg-01# set interfaces ae5 unit 0 family ethernet-switching port-mode trunk
    user@cs-agg-01# set interfaces ae5 unit 0 family ethernet-switching vlan members all
    user@cs-agg-01# set interfaces ge-0/0/23 ether-options 802.3ad ae5
    user@cs-agg-01# set interfaces ge-1/0/23 ether-options 802.3ad ae5
    user@cs-agg-01# set interfaces ge-0/0/31 ether-options 802.3ad ae5
    user@cs-agg-01# set interfaces ge-1/0/31 ether-options 802.3ad ae5
    

Configure the High Availability Software

Step-by-Step Procedure

To enable graceful Routing Engine switchover (GRES), nonstop active routing (NSR), and nonstop bridging (NSB):

  • Enter the following configuration statements:
    {master:0}[edit]
    user@cs-agg-01# set chassis redundancy graceful-switchover
    user@cs-agg-01# set ethernet-switching-options nonstop-bridging
    user@cs-agg-01# set routing-options nonstop-routing
    

Configuring the Core Switches for High Availability

The section provides the procedures for configuring the core switches in an active/active MC-LAG configuration.

To configure the core switches for high availability:

Configure the Number of Aggregated Ethernet Interfaces and Switch Service ID

Step-by-Step Procedure

This procedure configures two global settings for the switch:

  • Number of aggregated Ethernet Interfaces—You must specify the number of aggregated Ethernet interfaces that will be configured on the device.
  • Service ID—You must configure a service ID when the MC-LAG logical interfaces are part of a bridge domain, as they are in this example. The service ID is used to synchronize applications such as IGMP, ARP, and MAC learning across MC-LAG members.
  1. Specify the number of aggregated Ethernet interfaces to be created.
    {master}[edit]
    user@cs-core-sw1# set chassis aggregated-devices ethernet device-count 32
  2. Specify the switch service ID.
    {master}[edit]
    user@cs-core-sw1# set switch-options service-id 1

Configure the Inter-Chassis Control Protocol (ICCP) and ICCP Link

Step-by-Step Procedure

ICCP is a control plane protocol for MC-LAG. It uses TCP as a transport protocol and Bidirectional Forwarding Detection (BFD) for fast convergence. ICCP:

  • Synchronizes configurations and operational states between the two MC-LAG peers
  • Synchronizes MAC address and ARP entries learned from one MC-LAG node and shares them with the other peer

In the testing for this network configuration example, we achieved quicker convergence after a Routing Engine switchover by configuring a 3-second BFD timer for ICCP.

To configure ICCP and the ICCP link:

  1. Specify the members that belong to interface ae0, which is used for the ICCP link.

    On both cs-core-sw1 and cs-core-sw2, enter:

    {master}[edit]
    user@cs-core-sw1# set interfaces xe-0/3/6 ether-options 802.3ad ae0 
    user@cs-core-sw1# set interfaces xe-1/3/6 ether-options 802.3ad ae0
    
  2. Configure ae0 as a Layer 3 link.

    On cs-core-sw1, enter:

    {master}[edit]
    user@cs-core-sw1# set interfaces ae0 description "ICCP Layer 3 Link with 2 member,xe-0/3/6,xe-1/3/6"
    user@cs-core-sw1# set interfaces ae0 vlan-tagging
    user@cs-core-sw1# set interfaces ae0 aggregated-ether-options lacp active
    user@cs-core-sw1# set interfaces ae0 aggregated-ether-options lacp periodic fast
    user@cs-core-sw1# set interfaces ae0 unit 0 vlan-id 4000
    user@cs-core-sw1# set interfaces ae0 unit 0 family inet address 172.16.32.9/30
    

    On cs-core-sw2, enter:

    {master}[edit]
    user@cs-core-sw2# set interfaces ae0 description "ICCP Layer 3 Link with 2 member,xe-0/3/6,xe-1/3/6"
    user@cs-core-sw2# set interfaces ae0 vlan-tagging
    user@cs-core-sw2# set interfaces ae0 aggregated-ether-options lacp active
    user@cs-core-sw2# set interfaces ae0 aggregated-ether-options lacp periodic fast
    user@cs-core-sw2# set interfaces ae0 unit 0 vlan-id 4000
    user@cs-core-sw2# set interfaces ae0 unit 0 family inet address 172.16.32.10/30
    
  3. Configure ICCP, using the loopback addresses of cs-core-sw1 (172.16.32.5) and cs-core-sw2 (172.16.32.6) as the local IP addresses.

    On cs-core-sw1, enter:

    {master}[edit]
    user@cs-core-sw1# set protocols iccp local-ip-addr 172.16.32.5
    user@cs-core-sw1# set protocols iccp peer 172.16.32.6 redundancy-group-id-list 1
    user@cs-core-sw1# set protocols iccp peer 172.16.32.6 liveness-detection minimum-interval 1500
    user@cs-core-sw1#  set protocols iccp peer 172.16.32.6 liveness-detection multiplier 2
    

    On cs-core-sw2, enter:

    {master}[edit]
    user@cs-core-sw2# set protocols iccp local-ip-addr 172.16.32.6
    user@cs-core-sw2# set protocols iccp peer 172.16.32.5 redundancy-group-id-list 1
    user@cs-core-sw2# set protocols iccp peer 172.16.32.5 liveness-detection minimum-interval 1500
    user@cs-core-sw2# set protocols iccp peer 172.16.32.5 liveness-detection multiplier 2
    

    Together, the liveness-detection statements result in a BFD timer of 3 seconds (1.5 seconds * 2 multiplier).

Configure the Interchassis Link (ICL)

Step-by-Step Procedure

The ICL is a special Layer 2 link between peers in an active/active MC-LAG configuration. It provides redundancy when an active link to an MC-LAG node fails by permitting the nodes to forward traffic between them.

We recommend that you configure the ICL members with a hold-time down value that is higher than the configured BFD timer to prevent the ICL from being advertised as being down before the ICCP link is down. If the ICL goes down before the ICCP link, this causes a flap of the MC-LAG interface on the status-control standby node, which leads to a delay in convergence. This example uses a hold-time down value of 4 seconds (4000 ms), based on the ICCP BFD timer of 3 seconds. These values result in zero loss convergence during recovery of failed devices.

To configure the ICL:

  1. Configure ICL members with a hold-time value higher than the configured BFD timer.

    On both cs-core-sw1 and cs-core-sw2, enter:

    {master}[edit]
    user@cs-core-sw1# set interfaces xe-0/3/7 hold-time up 100
    user@cs-core-sw1# set interfaces xe-0/3/7 hold-time down 4000
    user@cs-core-sw1# set interfaces xe-0/3/7 ether-options 802.3ad ae29
    user@cs-core-sw1# set interfaces xe-1/3/7 hold-time up 100
    user@cs-core-sw1# set interfaces xe-1/3/7 hold-time down 4000
    user@cs-core-sw1# set interfaces xe-1/3/7 ether-options 802.3ad ae29

    Note: If you configure a hold-time down value, you must also configure a hold-time up value. We have chosen a minimal value for hold-time up in this configuration.

  2. Configure ae29, which is the LAG for the ICL.

    On both cs-core-sw1 and cs-core-sw2, enter:

    {master}[edit]
    user@cs-core-sw1#  set interfaces ae29 description "ICL Layer 2 link with 2 members,xe-0/3/7,1/3/7"
    user@cs-core-sw1# set interfaces ae29 vlan-tagging
    user@cs-core-sw1# set interfaces ae29 aggregated-ether-options lacp active
    user@cs-core-sw1# set interfaces ae29 aggregated-ether-options lacp periodic fast
    user@cs-core-sw1# set interfaces ae29 unit 0 family ethernet-switching interface-mode trunk
    user@cs-core-sw1# set interfaces ae29 unit 0 family ethernet-switching vlan members all
    

Configure the MC-LAG Links to the Access Layer

Step-by-Step Procedure

The core switches establish an MC-LAG link to each of the Virtual Chassis in the access layer. To create the MC-LAG link, you create an aggregated Ethernet interface, enable LACP on the interface, and configure the MC-LAG options under the mc-ae statement.

Table 10 describes the mc-ae options.

Table 10: mc-ae Statement Options

mc-ae Option

Description

mc-ae-id

Specifies which link aggregation group the aggregated Ethernet interface belongs to. In this solution, the mc-ae-id used matches the number of the aggregated Ethernet interface—that is, ae1 has a mc-ae-id of 1, ae2 has a mc-ae-id of 2, and ae3 has a mc-ae-id of 3.

redundancy-group

Used by ICCP to associate multiple chassis that perform similar redundancy functions and to establish a communication channel so that applications on peering chassis can send messages to each other. The MC-LAG interfaces on cs-core-sw1 and cs-core-sw2 are configured with the same redundancy group number, redundancy-group 1.

init-delay-time

Specifies the number of seconds by which to delay bringing the MC-LAG interface back to the up state when MC-LAG peer is rebooted. By delaying the bring up of the interface until after protocol convergence, you can prevent packet loss during the recovery of failed links and devices. In this solution, we found that a delay set to 520 seconds provided the quickest convergence after core switch failover. Configure this value for all MC-LAG interfaces on the core switches.

chassis-id

Used by LACP for calculating the port number of the MC-LAG physical member links. cs-core-sw1 uses chassis-id 0 to identify its MC-LAG interfaces. cs-core-sw2 uses chassis-id 1 to identify its MC-LAG interfaces.

mode

Indicates whether an MC-LAG is in active/standby mode or active/active mode. Chassis that are in the same group must be in the same mode. In this solution, the mode is active/active.

status-control

Specifies whether this node becomes active or goes into standby mode when an ICL failure occurs. Must be active on one node and standby on the other node.

events iccp-peer-down force-icl-down

Forces ICL down if the peer of this node goes down.

events iccp-peer-down prefer-status-control-active

Allows the LACP system ID to be retained during a reboot, which provides better convergence after a failover. Note that if you configure both nodes as prefer-status-control-active, as this configuration example shows, you must also configure ICCP peering using the peer’s loopback address to make sure the ICCP session does not go down due to physical link failure.

The following procedure shows how to configure the ae1 MC-LAG link to cs-4200-ab1. You can use the same procedure to configure the links to the other access switches, substituting the values shown in Table 11.

Table 11: Parameters for MC-LAGs to Access Switches

LAG

LAG Client

Member Interfaces

lacp system-id

lacp admin-key

mc-ae mc-ae-id

ae1

cs-4200-ab1

xe-0/0/0
xe-1/0/0

00:ae:01:00:00:01

1

1

ae2

cs-4300-ab2

xe-0/0/1
xe-1/0/1

00:ae:02:00:00:01

2

2

ae3

cs-4300-ab3

xe-0/0/2

00:ae:03:00:00:01

3

3

To configure the ae1 MC-LAG link to cs-4200-ab1:

  1. Specify the members to be included within the aggregated Ethernet interface ae1.

    On both cs-core-sw1 and cs-core-sw2, enter:

    {master}[edit]
    user@cs-core-sw1# set interfaces xe-0/0/0 ether-options 802.3ad ae1 
    user@cs-core-sw1# set interfaces xe-1/0/0 ether-options 802.3ad ae1
    
  2. Configure the LACP parameters on the aggregated Ethernet interface.

    On both cs-core-sw1 and cs-core-sw2, enter:

    {master}[edit]
    user@cs-core-sw1# set interfaces ae1 description "Layer 2 MCLAG between core & AB1,xe-0/0/0,1/0/0"
    user@cs-core-sw1# set interfaces ae1 aggregated-ether-options lacp active
    user@cs-core-sw1# set interfaces ae1 aggregated-ether-options lacp periodic fast
    user@cs-core-sw1# set interfaces ae1 aggregated-ether-options lacp system-priority 100
    user@cs-core-sw1# set interfaces ae1 aggregated-ether-options lacp system-id 00:ae:01:00:00:01
    user@cs-core-sw1# set interfaces ae1 aggregated-ether-options lacp admin-key 1
    
  3. Configure the mc-ae interface parameters.

    On cs-core-sw1, enter:

    {master}[edit]
    user@cs-core-sw1# set interfaces ae1 aggregated-ether-options mc-ae mc-ae-id 1
    user@cs-core-sw1# set interfaces ae1 aggregated-ether-options mc-ae redundancy-group 1
    user@cs-core-sw1# set interfaces ae1 aggregated-ether-options mc-ae chassis-id 0
    user@cs-core-sw1# set interfaces ae1 aggregated-ether-options mc-ae mode active-active
    user@cs-core-sw1# set interfaces ae1 aggregated-ether-options mc-ae status-control active
    user@cs-core-sw1# set  interfaces ae1 aggregated-ether-options mc-ae init-delay-time 520
    user@cs-core-sw1# set interfaces ae1 aggregated-ether-options mc-ae events iccp-peer-down force-icl-down
    user@cs-core-sw1# set interfaces ae1 aggregated-ether-options mc-ae events iccp-peer-down prefer-status-control-active
    user@cs-core-sw1# set interfaces ae1 unit 0 multi-chassis-protection 172.16.32.6 interface ae29.0
    

    On cs-core-sw2, enter:

    {master}[edit]
    user@cs-core-sw2# set interfaces ae1 aggregated-ether-options mc-ae mc-ae-id 1
    user@cs-core-sw2# set interfaces ae1 aggregated-ether-options mc-ae redundancy-group 1
    user@cs-core-sw2# set interfaces ae1 aggregated-ether-options mc-ae chassis-id 1
    user@cs-core-sw2# set interfaces ae1 aggregated-ether-options mc-ae mode active-active
    user@cs-core-sw2# set interfaces ae1 aggregated-ether-options mc-ae status-control standby
    user@cs-core-sw2# set interfaces ae1 aggregated-ether-options mc-ae init-delay-time 520
    user@cs-core-sw2# set interfaces ae1 aggregated-ether-options mc-ae events iccp-peer-down force-icl-down
    user@cs-core-sw2# set interfaces ae1 aggregated-ether-options mc-ae events iccp-peer-down prefer-status-control-active
    user@cs-core-sw2# set interfaces ae1 unit 0 multi-chassis-protection 172.16.32.5 interface ae29.0
    
  4. Configure ae1 as a trunk port, with membership in all VLANS.

    On both cs-core-sw1 and cs-core-sw2, enter:

    user@cs-core-sw1# set interfaces ae1 unit 0 family ethernet-switching interface-mode trunk
    user@cs-core-sw1# set interfaces ae1 unit 0 family ethernet-switching vlan members all

Configure the MC-LAG Links to the Edge Firewalls

Step-by-Step Procedure

The following procedure shows how to configure the ae11 MC-LAG link to cs-edge-fw01 on cs-core-sw01 and cs-core-sw02. You can use the same procedure to configure the ae12 MC-LAG link to cs-edge-fw02 on both switches, substituting the values shown in Table 12.

Table 12: Parameters for Edge Router MC-LAG Interfaces Connecting to Edge Firewalls

LAG

LAG Client

Member Interface

lacp system-id

lacp admin-key

mc-ae mc-ae-id

ae11

reth 0 on cs-edge-fw01

ge-2/0/0

00:ae:11:00:00:01

11

11

ae12

reth 0 on cs-edge-fw02

ge-2/0/1

00:ae:12:00:00:01

12

12

To configure the ae11 MC-LAG link to the core switches:

  1. Specify the interface to be included within the aggregated Ethernet interface ae11.

    On both cs-core-sw1 and cs-core-sw2, enter:

    user@cs-core-sw1# set interfaces ge-2/0/0 ether-options 802.3ad ae11
  2. Configure the LACP parameters on the aggregated Ethernet interface.

    On both cs-core-sw1 and cs-core-sw2, enter:

    user@cs-core-sw1# set interfaces ae11 description "MC-LAG to edge-fw1"
    user@cs-core-sw1# set interfaces ae11 vlan-tagging
    user@cs-core-sw1# set interfaces ae11 aggregated-ether-options lacp active
    user@cs-core-sw1# set interfaces ae11 aggregated-ether-options lacp periodic fast
    user@cs-core-sw1# set interfaces ae11 aggregated-ether-options lacp system-priority 100
    user@cs-core-sw1# set interfaces ae11 aggregated-ether-options lacp system-id 00:ae:11:00:00:01
    user@cs-core-sw1# set interfaces ae11 aggregated-ether-options lacp admin-key 11
    
  3. Configure the mc-ae interface parameters.

    On cs-core-sw1, enter:

    user@cs-core-sw1# set interfaces ae11 aggregated-ether-options mc-ae mc-ae-id 11
    user@cs-core-sw1# set interfaces ae11 aggregated-ether-options mc-ae redundancy-group 1
    user@cs-core-sw1# set interfaces ae11 aggregated-ether-options mc-ae chassis-id 0
    user@cs-core-sw1# set interfaces ae11 aggregated-ether-options mc-ae mode active-active
    user@cs-core-sw1# set interfaces ae11 aggregated-ether-options mc-ae status-control active
    user@cs-core-sw1# set interfaces ae11 aggregated-ether-options mc-ae init-delay-time 520
    user@cs-core-sw1# set interfaces ae11 aggregated-ether-options mc-ae events iccp-peer-down force-icl-down
    user@cs-core-sw1# set interfaces ae11 aggregated-ether-options mc-ae events iccp-peer-down prefer-status-control-active
    

    On cs-core-sw2, enter:

    user@cs-core-sw2# set interfaces ae11 aggregated-ether-options mc-ae mc-ae-id 11
    user@cs-core-sw2# set interfaces ae11 aggregated-ether-options mc-ae redundancy-group 1
    user@cs-core-sw2# set interfaces ae11 aggregated-ether-options mc-ae chassis-id 1
    user@cs-core-sw2# set interfaces ae11 aggregated-ether-options mc-ae mode active-active
    user@cs-core-sw2# set interfaces ae11 aggregated-ether-options mc-ae status-control standby
    user@cs-core-sw2# set interfaces ae11 aggregated-ether-options mc-ae init-delay-time 520
    user@cs-core-sw2# set interfaces ae11 aggregated-ether-options mc-ae events iccp-peer-down force-icl-down
    user@cs-core-sw2# set interfaces ae11 aggregated-ether-options mc-ae events iccp-peer-down prefer-status-control-active
    
  4. Configure ae11.0 as a trunk interface and as a member of the Firewall-trust VLAN.

    On cs-core-sw1, enter:

    user@cs-core-sw1# set interfaces ae11 vlan-tagging
    user@cs-core-sw1# set interfaces ae11 unit 0 family ethernet-switching interface-mode trunk
    user@cs-core-sw1# set interfaces ae11 unit 0 family ethernet-switching vlan members Firewall-trust
    user@cs-core-sw1# set interfaces ae11 unit 0 multi-chassis-protection 172.16.32.6 interface ae29.0
    

    On cs-core-sw2, enter:

    user@cs-core-sw2# set interfaces ae11 vlan-tagging
    user@cs-core-sw2# set interfaces ae11 unit 0 family ethernet-switching interface-mode trunk
    user@cs-core-sw2# set interfaces ae11 unit 0 family ethernet-switching vlan members Firewall-trust
    user@cs-core-sw2# set interfaces ae11 unit 0 multi-chassis-protection 172.16.32.5 interface ae29.0
    

Configure the Bridge Domain on the MC-LAG Interfaces to the Edge Firewalls

Step-by-Step Procedure

The active node in the SRX chassis cluster uses gratuitous ARP to advertise to connecting devices that it is the next-hop gateway. This requires that the interfaces between the connecting devices and the SRX chassis cluster be in the same bridge domain.

Table 13 summarizes the configuration of this bridge domain.

Table 13: VLAN 600 Configuration

VLAN Name

VLAN ID

IRB Name

IP Address Information

Mask

cs-core-sw01 Address

cs-core-sw01 Address

Virtual IP Address

Firewall-Trust

600

irb.600

/29

172.16.33.3

172.16.33.2

172.16.33.1

To configure the required bridge domain:

  1. Create the bridge domain.

    On cs-core-sw1, enter:

    user@cs-core-sw1# set vlans Firewall-trust vlan-id 600
    user@cs-core-sw1# set vlans Firewall-trust l3-interface irb.600
    user@cs-core-sw1# set vlans Firewall-trust switch-options interface ae29.0 static-mac 28:8a:1c:e5:3b:f0
    user@cs-core-sw1# set vlans Firewall-trust domain-type bridge

    On cs-core-sw2, enter:

    user@cs-core-sw2# set vlans Firewall-trust vlan-id 600
    user@cs-core-sw2# set vlans Firewall-trust l3-interface irb.600
    user@cs-core-sw2# set vlans Firewall-trust switch-options interface ae29.0 static-mac 28:8a:1c:e3:f7:f0
    user@cs-core-sw2# set vlans Firewall-trust domain-type bridge

    Note: The static-mac option on VLAN 600 (Firewall-trust) prevents traffic arriving from the SRX chassis cluster from flooding the VLAN.

    The SRX chassis cluster sends traffic to both core switches using the IRB 600 MAC address for routing the packet. The IRB 600 MAC addresses on cs-core-sw1 and cs-core-sw2 are different. Because the reth1 interface on the chassis cluster is a single LAG, the reth0 LAG address hashing results in a packet destined to the cs-core-sw1 MAC address being sent to cs-core-sw2. In an MC-LAG configuration, MAC address learning does not occur on the ICL link, and, as a result, cs-core-sw2 floods the packet on VLAN 600. To avoid flooding on VLAN 600, specify the MAC address for cs-core-sw1 in the static-mac option on cs-core-sw2 and vice versa. When a packet destined to cs-core-sw1 arrives at cs-core-sw2, cs-core-sw2 sends the packet to cs-core-sw1 using the static MAC address.

  2. Configure an IRB interface on the VLAN and enable VRRP on the IRB interface.

    On cs-core-sw1, enter:

    user@cs-core-sw1# set interfaces irb unit 600 family inet address 172.16.33.3/29 arp 172.16.33.2 l2-interface ae29.0
    user@cs-core-sw1# set interfaces irb unit 600 family inet address 172.16.33.3/29 arp 172.16.33.2 mac 28:8a:1c:e5:3b:f0
    user@cs-core-sw1# set interfaces irb unit 600 family inet address 172.16.33.3/29 vrrp-group 1 virtual-address 172.16.33.1
    user@cs-core-sw1# set interfaces irb unit 600 family inet address 172.16.33.3/29 vrrp-group 1 priority 125
    user@cs-core-sw1# set interfaces irb unit 600 family inet address 172.16.33.3/29 vrrp-group 1 preempt
    user@cs-core-sw1# set interfaces irb unit 600 family inet address 172.16.33.3/29 vrrp-group 1 accept-data
    user@cs-core-sw1# set interfaces irb unit 600 family inet address 172.16.33.3/29 vrrp-group 1 authentication-type md5
    user@cs-core-sw1# set interfaces irb unit 600 family inet address 172.16.33.3/29 vrrp-group 1 authentication-key "$9$9FCMt0IylMNdsEcds24DjCtu"

    On cs-core-sw2, enter:

    user@cs-core-sw2# set interfaces irb unit 600 family inet address 172.16.33.2/29 arp 172.16.33.3 l2-interface ae29.0
    user@cs-core-sw2# set interfaces irb unit 600 family inet address 172.16.33.2/29 arp 172.16.33.3 mac 28:8a:1c:e3:f7:f0
    user@cs-core-sw2# set interfaces irb unit 600 family inet address 172.16.33.2/29 vrrp-group 1 virtual-address 172.16.33.1
    user@cs-core-sw2# set interfaces irb unit 600 family inet address 172.16.33.2/29 vrrp-group 1 priority 125
    user@cs-core-sw2# set interfaces irb unit 600 family inet address 172.16.33.2/29 vrrp-group 1 preempt
    user@cs-core-sw2# set interfaces irb unit 600 family inet address 172.16.33.2/29 vrrp-group 1 accept-data
    user@cs-core-sw2# set interfaces irb unit 600 family inet address 172.16.33.2/29 vrrp-group 1 authentication-type md5
    user@cs-core-sw2# set interfaces irb unit 600 family inet address 172.16.33.2/29 vrrp-group 1 authentication-key "$9$p11sOIcKMXbs4yls4aZkquO1"

Configure Hold-Up Timers on Other Interfaces

Step-by-Step Procedure

In addition to the MC-LAG interfaces, the core switches have other Layer 2 and Layer 3 interfaces, such as the Layer 3 interface connecting to the aggregation switch in location B. To avoid having these interfaces come up before the MC-LAG synchronization completes after a failover, you can configure a hold-up timer on the interfaces. The interfaces will not come up until the timer expires.

In our testing, we found that a hold-up timer of 467 seconds gave the best convergence results.

To configure the hold-up timer on an interface (in this case, the interface connecting to aggregation switch):

  • On both cs-core-sw1 and cs-core-sw2, enter the following configuration statements:
    user@cs-core-sw1# set  interfaces xe-0/1/0 hold-time up 467000
    user@cs-core-sw1# set  interfaces xe-0/1/0 hold-time down 10
    

Configure VRRP on IRB Interfaces

Step-by-Step Procedure

VRRP is used in conjunction with MC-LAG on the core switches. VRRP permits redundant routers to appear as a single virtual router to the other devices. In a VRRP implementation, each VRRP peer shares a common virtual IP address and virtual MAC address in addition to its unique physical IP address and MAC address. Thus, each IRB configured on the core switches must have a virtual IP address.

To configure VRRP on an IRB—in this case, the IRB that is the Layer 3 interface for the eng1_data_wired VLAN:

  1. Configure the eng1_data_wired VLAN and the IRB as the routing interface for the VLAN.

    On both cs-core-sw1 and cs-core-sw2, enter:

    user@cs-core-sw1# set vlans eng1_data_wired vlan-id 60
    user@cs-core-sw1# set vlans eng1_data_wired l3-interface irb.60
    user@cs-core-sw1# set vlans eng1_data_wired domain-type bridge
    
  2. Configure the IRB and enable VRRP on it.

    On cs-core-sw1, enter:

    user@cs-core-sw1# set interfaces irb unit 60 family inet address 10.32.0.3/20 arp 10.32.0.2 l2-interface ae29.0
    user@cs-core-sw1# set interfaces irb unit 60 family inet address 10.32.0.3/20 arp 10.32.0.2 mac 28:8a:1c:e5:3b:f0
    user@cs-core-sw1# set interfaces irb unit 60 family inet address 10.32.0.3/20 vrrp-group 1 virtual-address 10.32.0.1
    user@cs-core-sw1# set interfaces irb unit 60 family inet address 10.32.0.3/20 vrrp-group 1 priority 125
    user@cs-core-sw1# set interfaces irb unit 60 family inet address 10.32.0.3/20 vrrp-group 1 preempt
    user@cs-core-sw1# set interfaces irb unit 60 family inet address 10.32.0.3/20 vrrp-group 1 accept-data
    user@cs-core-sw1# set interfaces irb unit 60 family inet address 10.32.0.3/20 vrrp-group 1 authentication-type md5
    user@cs-core-sw1# set interfaces irb unit 60 family inet address 10.32.0.3/20 vrrp-group 1 authentication-key "$9$lN3v87wYojHm-VHmfT/9evW"
    

    On cs-core-sw2, enter:

    user@cs-core-sw2# set interfaces irb unit 60 family inet address 10.32.0.2/20 arp 10.32.0.3 l2-interface ae29.0
    user@cs-core-sw2# set interfaces irb unit 60 family inet address 10.32.0.2/20 arp 10.32.0.3 mac 28:8a:1c:e3:f7:f0
    user@cs-core-sw2# set interfaces irb unit 60 family inet address 10.32.0.2/20 vrrp-group 1 virtual-address 10.32.0.1
    user@cs-core-sw2# set interfaces irb unit 60 family inet address 10.32.0.2/20 vrrp-group 1 priority 125
    user@cs-core-sw2# set interfaces irb unit 60 family inet address 10.32.0.2/20 vrrp-group 1 preempt
    user@cs-core-sw2# set interfaces irb unit 60 family inet address 10.32.0.2/20 vrrp-group 1 accept-data
    user@cs-core-sw2# set interfaces irb unit 60 family inet address 10.32.0.2/20 vrrp-group 1 authentication-type md5
    user@cs-core-sw2# set interfaces irb unit 60 family inet address 10.32.0.2/20 vrrp-group 1 authentication-key "$9$bIY4ZHqfn/tUj/tuOcSwYg"
    

Configure the High Availability Software

Step-by-Step Procedure

To enable graceful Routing Engine switchover (GRES), nonstop active routing (NSR), and nonstop bridging (NSB):

  • On both cs-core-sw1 and cs-core-sw2, enter the following configuration statements:
    user@cs-core-sw1# set chassis redundancy graceful-switchover
    user@cs-core-sw1# set protocols layer2-control nonstop-bridging
    user@cs-core-sw1# set routing-options nonstop-routing
    

Configuring the Edge Firewalls for High Availability

The section provides the procedures for configuring the edge firewalls in a chassis cluster configuration and for configuring the redundant Ethernet interfaces.

To configure the edge firewalls for high availability:

Enable Chassis Cluster Mode

Step-by-Step Procedure

The command for enabling chassis cluster mode is an operational command, not a configuration statement, and must be executed on each member. The command causes the cluster member to reboot.

When you enable chassis cluster mode, you specify a cluster ID for the cluster. Because this network configuration example has only a single cluster, it uses cluster ID 1 for the cluster, with cs-edge-fw01 configured as node 0 and cs-edge-fw02 configured as node 1.

After you enable chassis clustering, the cluster members share a single, common configuration. All subsequent configuration steps can be done from the primary cluster member (node 0).

To enable chassis clustering on each member:

  1. On cs-edge-fw01, enter the following operational command:
    user@cs-edge-fw01> set chassis cluster cluster-id 1 node 0 reboot
  2. On cs-edge-fw02, enter the following operational command:
    user@cs-edge-fw02> set chassis cluster cluster-id 1 node 1 reboot

    After the chassis members finish rebooting, the slot numbering on node 1 is changed so that numbering begins with slot 9 instead of slot 0. In addition, the interfaces shown in Table 14 are automatically mapped to the fxp0 and fxp1 interfaces.

    Table 14: Mapping of Interfaces After Chassis Clustering Is Enabled

    Interface on Node 0

    Interface on Node 1

    Mapped to

    Purpose

    ge-0/0/0

    ge-9/0/0

    fxp0

    Out-of-band management

    ge-0/0/1

    ge-9/0/1

    fxp1

    Chassis cluster control link

Configure the Chassis Cluster Data Fabric

Step-by-Step Procedure

After the chassis cluster has formed, you must configure the fabric ports for the cluster. These ports are used to pass real-time objects (RTOs) in active/passive mode. RTOs are messages that the cluster members use to synchronize information with each other.

To configure the data fabric, you must configure two fabric interfaces (one on each chassis) as shown:

  1. Configure the fabric link for cs-edge-fw01.
    user@user@cs-edge-fw01# set interfaces fab0 fabric-options member-interfaces ge-0/0/2
  2. Configure the fabric link for cs-edge-fw02.
    user@user@cs-edge-fw01# set interfaces fab1 fabric-options member-interfaces ge-9/0/2

Configure Chassis Clustering Groups

Step-by-Step Procedure

Although the chassis cluster configuration is held within a single common configuration, some elements of the configuration need to be applied to a specific member. Examples include the host name and the out-of-band management interface.

To apply the configuration to a specific member, you use the node-specific configuration method called groups.

To configure chassis clustering groups:

  1. Configure node-specific information for cs-edge-fw01 (node 0):
    user@cs-edge-fw01-node0# set groups node0 system host-name cs-edge-fw01-node0
    user@cs-edge-fw01-node0# set groups node0 interfaces fxp0 unit 0 family inet address 10.92.76.63/23
    
  2. Configure node-specific information for cs-edge-fw02 (node 1):
    user@cs-edge-fw01-node0# set groups node1 system host-name cs-edge-fw02-node1
    user@cs-edge-fw01-node0# set groups node1 interfaces fxp0 unit 0 family inet address 10.92.76.64/23
  3. Configure apply groups.
    user@cs-edge-fw01-node0# set apply-groups "${node}"

    This command uses the node variable to define how the groups are applied to the nodes (each node will recognize its number and accept the configuration accordingly).

Configure Chassis Cluster Redundancy Groups

Step-by-Step Procedure

The next step in configuring chassis clustering is to configure redundancy groups. Redundancy group 0 is always for the control plane, while redundancy group 1+ is always for the data plane ports. Because active/backup mode allows only one chassis member to be active at a time, you define only redundancy groups 0 and 1.

You also need to define which device has priority for the control plane, as well as which device has priority for the data plane. Although the control plane can be active on a different chassis than the data plane in active/passive clustering, many administrators prefer having both the control plane and data plane active on the same chassis member. This example gives node 0 priority for both the control plane and data plane.

To configure chassis cluster redundancy groups:

  • Enter the following commands:
    user@cs-edge-fw01-node0# set chassis cluster redundancy-group 1 node 0 priority 100
    user@cs-edge-fw01-node0# set chassis cluster redundancy-group 1 node 1 priority 1
    user@cs-edge-fw01-node0# set chassis cluster redundancy-group 0 node 0 priority 100
    user@cs-edge-fw01-node0# set chassis cluster redundancy-group 0 node 1 priority 1

Configure the Redundant Ethernet Interfaces

Step-by-Step Procedure

The redundant Ethernet interfaces connect the SRX chassis cluster to the core switches and edge routers. They allow the backup chassis member to take over the connections seamlessly in the event of a data plane failover. To configure the redundant Ethernet interfaces, you define which interfaces belong to the redundant Ethernet interface, define which redundancy group the redundant Ethernet interface belongs to (in an active/passive cluster, the interface always belongs to redundancy group 1), and define the redundant Ethernet interface information, such as the IP address of the interface.

To configure redundant Ethernet interfaces on the chassis cluster:

  1. Specify the number of redundant Ethernet interfaces to be configured.

    This is similar to how you configure the number of aggregated Ethernet interfaces on a switch.

    user@cs-edge-fw01-node0# set chassis cluster reth-count 2
    
  2. Configure redundant Ethernet interface reth0 toward the core switches.
    user@cs-edge-fw01-node0# set interfaces reth0 description "Trust Zone towards Core"
    user@cs-edge-fw01-node0# set interfaces reth0 vlan-tagging
    user@cs-edge-fw01-node0# set interfaces reth0 redundant-ether-options redundancy-group 1
    user@cs-edge-fw01-node0# set interfaces reth0 redundant-ether-options minimum-links 1
    user@cs-edge-fw01-node0# set interfaces reth0 redundant-ether-options lacp active
    user@cs-edge-fw01-node0# set interfaces reth0 redundant-ether-options lacp periodic fast
    user@cs-edge-fw01-node0# set interfaces reth0 unit 0 vlan-id 600
    user@cs-edge-fw01-node0# set interfaces reth0 unit 0 family inet address 172.16.33.4/29
  3. Configure the member links for reth0.
    user@cs-edge-fw01-node0# set interfaces ge-2/0/16 gigether-options redundant-parent reth0
    user@cs-edge-fw01-node0# set interfaces ge-2/0/17 gigether-options redundant-parent reth0
    user@cs-edge-fw01-node0# set interfaces ge-11/0/16 gigether-options redundant-parent reth0 user@cs-edge-fw01-node0# set interfaces ge-11/0/17 gigether-options redundant-parent reth0
  4. Configure redundant Ethernet interface reth1 toward the edge routers.
    user@cs-edge-fw01-node0# set interfaces reth1 description "Untrust Zone towards Edge-routers"
    user@cs-edge-fw01-node0# set interfaces reth1 vlan-tagging
    user@cs-edge-fw01-node0# set interfaces reth1 redundant-ether-options redundancy-group 1
    user@cs-edge-fw01-node0# set interfaces reth1 redundant-ether-options minimum-links 1
    user@cs-edge-fw01-node0# set interfaces reth1 redundant-ether-options lacp active
    user@cs-edge-fw01-node0# set interfaces reth1 redundant-ether-options lacp periodic fast
    user@cs-edge-fw01-node0# set interfaces reth1 unit 0 vlan-id 601
    user@cs-edge-fw01-node0# set interfaces reth1 unit 0 family inet address 172.16.33.12/29
  5. Configure the member links for reth1.
    user@cs-edge-fw01-node0# set interfaces ge-2/0/18 gigether-options redundant-parent reth1
    user@cs-edge-fw01-node0# set interfaces ge-2/0/19 gigether-options redundant-parent reth1
    user@cs-edge-fw01-node0# set interfaces ge-11/0/18 gigether-options redundant-parent reth1
    user@cs-edge-fw01-node0# set interfaces ge-11/0/19 gigether-options redundant-parent reth1

Configure the Bridge Domains

Step-by-Step Procedure

As previously described, the active node uses gratuitous Address Resolution Protocol (ARP) to advertise to the connecting devices that it is the next-hop gateway. This requires that the redundant Ethernet interface members and their connecting interfaces on the other devices belong to the same bridge domain.

To configure the bridge domains for reth0 and reth1:

  • Enter the following commands:
    user@cs-edge-fw01-node0# set bridge-domains reth-bd vlan-id-list 600
    user@cs-edge-fw01-node0# set bridge-domains reth-bd vlan-id-list 601

Configuring the Edge Routers for High Availability

The section provides the procedures for configuring the edge routers in an MC-LAG configuration and for configuring the high availability software.

To configure the edge routers for high availability:

Configure the Number of Aggregated Ethernet Interfaces and the Service ID

Step-by-Step Procedure

This procedure configures two global settings for the router:

  • Number of aggregated Ethernet interfaces—You must specify the number of aggregated Ethernet interfaces that will be configured on the device.
  • Service ID—You must configure a service ID when the MC-LAG logical interfaces are part of a bridge domain, as they are in this example. The service ID is used to synchronize applications such as IGMP, ARP, and MAC learning across MC-LAG members.

On both cs-edge-r01 and cs-edge-r02:

  1. Specify the number of aggregated Ethernet interfaces to be created.
    {master}[edit]
    user@cs-edge-r01# set chassis aggregated-devices ethernet device-count 5
  2. Specify the switch service ID.
    {master}[edit]
    user@cs-edge-r01# set switch-options service-id 1

Configure the Inter-Chassis Control Protocol (ICCP) and ICCP Link

Step-by-Step Procedure

To configure ICCP and the ICCP link:

  1. Specify the member interface that belongs to interface ae0, which will be used for the ICCP link.

    On both cs-edge-r01 and cs-edge-r02, enter:

    {master}[edit]
    user@cs-edge-r01# set interfaces ge-1/2/4 gigether-options 802.3ad ae0
    
  2. Configure ae0 as a Layer 3 link for ICCP.

    On cs-edge-r01, enter:

    {master}[edit]
    user@cs-edge-r01# set interfaces ae0 flexible-vlan-tagging
    user@cs-edge-r01# set interfaces ae0 encapsulation flexible-ethernet-services
    user@cs-edge-r01# set interfaces ae0 aggregated-ether-options link-speed 1g
    user@cs-edge-r01# set interfaces ae0 aggregated-ether-options lacp active
    user@cs-edge-r01# set interfaces ae0 aggregated-ether-options lacp periodic slow
    user@cs-edge-r01# set interfaces ae0 unit 0 description "ICCP Link between edge-r1 & edge-r2"
    user@cs-edge-r01# set interfaces ae0 unit 0 vlan-id 4000
    user@cs-edge-r01# set interfaces ae0 unit 0 family inet address 172.16.32.41/30
    

    On cs-edge-r02, enter:

    {master}[edit]
    user@cs-edge-r02# set interfaces ae0 flexible-vlan-tagging
    user@cs-edge-r02# set interfaces ae0 encapsulation flexible-ethernet-services
    user@cs-edge-r02# set interfaces ae0 aggregated-ether-options link-speed 1g
    user@cs-edge-r02# set interfaces ae0 aggregated-ether-options lacp active
    user@cs-edge-r02# set interfaces ae0 aggregated-ether-options lacp periodic slow
    user@cs-edge-r02# set interfaces ae0 unit 0 description "ICCP link between edge-r2 to edge-r1"
    user@cs-edge-r02# set interfaces ae0 unit 0 vlan-id 4000
    user@cs-edge-r02# set interfaces ae0 unit 0 family inet address 172.16.32.42/30
    
  3. Configure ICCP, using the loopback addresses of cs-edge-r01 (172.16.32.33) and cs-edge-r02 (172.16.32.34) as the local IP addresses.

    On cs-edge-r01, enter:

    {master}[edit]
    user@cs-edge-r01# set protocols iccp local-ip-addr 172.16.32.33
    user@cs-edge-r01# set protocols iccp peer 172.16.32.34 redundancy-group-id-list 1
    user@cs-edge-r01# set protocols iccp peer 172.16.32.34 liveness-detection minimum-interval 500
    user@cs-edge-r01# set protocols iccp peer 172.16.32.34 liveness-detection multiplier 3
    

    On cs-edge-r02, enter:

    {master}[edit]
    user@cs-edge-r02# set protocols iccp local-ip-addr 172.16.32.34
    user@cs-edge-r02# set protocols iccp peer 172.16.32.33 redundancy-group-id-list 1
    user@cs-edge-r02# set protocols iccp peer 172.16.32.33 liveness-detection minimum-interval 500
    user@cs-edge-r02# set protocols iccp peer 172.16.32.33 liveness-detection multiplier 3
    

    Note: The BFD timer is configured to be 1.5 sec, which provides faster convergence in this network configuration.

Configure the Interchassis Link (ICL) on the Edge Routers

Step-by-Step Procedure

To configure the ICL link on the edge routers:

  1. Configure the ICL member link.

    On both cs-edge-r01 and cs-edge-r02, enter:

    {master}[edit]
    user@cs-edge-r01# set interfaces ge-1/2/5 hold-time up 100
    user@cs-edge-r01# set interfaces ge-1/2/5 hold-time down 3000
    user@cs-edge-r01# set interfaces ge-1/2/5 gigether-options 802.3ad ae4
    

    For faster convergence, the hold-down timer is configured to be greater than the ICCP BFD timer, which is set to 1.5 seconds.

  2. Configure ae4, which will be used for the ICL link.

    On both cs-edge-r01 and cs-edge-r02, enter:

    {master}[edit]
    user@cs-edge-r01# set interfaces ae4 description ICL
    user@cs-edge-r01# set interfaces ae4 flexible-vlan-tagging
    user@cs-edge-r01# set interfaces ae4 encapsulation flexible-ethernet-services
    user@cs-edge-r01# set interfaces ae4 aggregated-ether-options link-speed 1g
    user@cs-edge-r01# set interfaces ae4 aggregated-ether-options lacp active
    user@cs-edge-r01# set interfaces ae4 aggregated-ether-options lacp periodic slow
    user@cs-edge-r01# set interfaces ae4 unit 0 description "ICL Link to edge-r2-vlan-601"
    user@cs-edge-r01# set interfaces ae4 unit 0 encapsulation vlan-bridge
    user@cs-edge-r01# set interfaces ae4 unit 0 vlan-id 601
    

Configure the MC-LAG Links from the Routers to the Firewalls

Step-by-Step Procedure

The edge routers establish MC-LAG links to each of the SRX Series gateways in the chassis cluster. To create the MC-LAG link, you create an aggregated Ethernet interface, enable LACP on the interface, and configure the MC-LAG options under the mc-ae option. Table 10 describes the MC-LAG options.

The following procedure shows how to configure the ae1 MC-LAG link to edge-fw-1. You can use the same procedure to configure the ae3 link to edge-fw-2, substituting the values shown in Table 15.

Table 15: Parameters for MC-LAG Interfaces from Routers to Firewalls

LAG

LAG Client

Description String

Member Interface

lacp system-id

lacp admin-key

mc-ae mc-ae-id

ae1

reth 1 on cs-edge-fw01

“To-Firewall-reth1”

ge-1/0/0

00:ae:01:00:00:01

1

1

ae3

reth 1 on cs-edge-fw02

“To-Firewall-Standby”

ge-1/0/1

00:ae:03:00:00:01

3

3

To configure the MC-LAG interfaces to the firewalls:

  1. Specify the members to be included within the aggregated Ethernet interface ae1.

    On both cs-edge-r01 and cs-edge-r02, enter:

    {master}[edit]
    user@cs-edge-r01#  set interfaces ge-1/0/0 gigether-options 802.3ad ae1
  2. Configure flexible VLAN tagging and the LACP parameters on the aggregated Ethernet interface.

    On cs-edge-r01 and cs-edge-r02, enter:

    {master}[edit]
    user@cs-edge-r01# set interfaces ae1 description To-Firewall-reth1
    user@cs-edge-r01# set interfaces ae1 flexible-vlan-tagging
    user@cs-edge-r01# set interfaces ae1 encapsulation flexible-ethernet-services
    user@cs-edge-r01# set interfaces ae1 aggregated-ether-options link-speed 1g
    user@cs-edge-r01# set interfaces ae1 aggregated-ether-options lacp active
    user@cs-edge-r01# set interfaces ae1 aggregated-ether-options lacp periodic fast
    user@cs-edge-r01# set interfaces ae1 aggregated-ether-options lacp system-priority 100
    user@cs-edge-r01# set interfaces ae1 aggregated-ether-options lacp system-id 00:ae:01:00:00:01
    user@cs-edge-r01# set interfaces ae1 aggregated-ether-options lacp admin-key 1
    
    
  3. Configure the mc-ae interface parameters.

    On cs-edge-r01, enter:

    {master}[edit]
    user@cs-edge-r01# set interfaces ae1 aggregated-ether-options mc-ae mc-ae-id 1
    user@cs-edge-r01# set interfaces ae1 aggregated-ether-options mc-ae redundancy-group 1
    user@cs-edge-r01# set interfaces ae1 aggregated-ether-options mc-ae chassis-id 0
    user@cs-edge-r01# set interfaces ae1 aggregated-ether-options mc-ae mode active-active
    user@cs-edge-r01# set interfaces ae1 aggregated-ether-options mc-ae status-control active
    user@cs-edge-r01# set interfaces ae1 aggregated-ether-options mc-ae events iccp-peer-down force-icl-down
    user@cs-edge-r01# set interfaces ae1 aggregated-ether-options mc-ae events iccp-peer-down prefer-status-control-active
    

    On cs-edge-r02, enter:

    {master}[edit]
    user@cs-edge-r02# set interfaces ae1 aggregated-ether-options mc-ae mc-ae-id 1
    user@cs-edge-r02# set interfaces ae1 aggregated-ether-options mc-ae redundancy-group 1
    user@cs-edge-r02# set interfaces ae1 aggregated-ether-options mc-ae chassis-id 1
    user@cs-edge-r02# set interfaces ae1 aggregated-ether-options mc-ae mode active-active
    user@cs-edge-r02# set interfaces ae1 aggregated-ether-options mc-ae status-control standby
    user@cs-edge-r02# set interfaces ae1 aggregated-ether-options mc-ae events iccp-peer-down force-icl-down
    user@cs-edge-r02# set interfaces ae1 aggregated-ether-options mc-ae events iccp-peer-down prefer-status-control-active
    
  4. Configure a logical interface on ae1, with membership in VLAN 601.

    On cs-edge-r01, enter:

    {master}[edit]
    user@cs-edge-r01# set interfaces ae1 unit 0 encapsulation vlan-bridge
    user@cs-edge-r01# set interfaces ae1 unit 0 vlan-id 601
    user@cs-edge-r01# set interfaces ae1 unit 0 multi-chassis-protection 172.16.32.34 interface ae4.0
    

    On cs-edge-r02, enter:

    {master}[edit]
    user@cs-edge-r02# set interfaces ae1 unit 0 encapsulation vlan-bridge
    user@cs-edge-r02# set interfaces ae1 unit 0 vlan-id 601
    user@cs-edge-r02# set interfaces ae1 unit 0 multi-chassis-protection 172.16.32.33 interface ae4.0
    

Configure the Bridge Domain on the MC-LAG Interfaces to the Firewalls

Step-by-Step Procedure

The active node in the SRX chassis cluster uses gratuitous ARP to advertise to connecting devices that it is the next-hop gateway. This requires that the interfaces between the connecting devices and the SRX chassis cluster be in the same bridge domain.

Table 16 summarizes the configuration of this bridge domain.

Table 16: Bridge Domain 601 Configuration

Name

ID

IRB Name

IP Address Information

Mask

cs-edge-r01 Address

cs-edge-r02 Address

Virtual IP Address

bd1

601

irb.601

/29

172.16.33.10

172.16.33.11

172.16.33.9

To configure the required bridge domain:

  1. Create the bridge domain.

    On cs-edge-r01, enter:

    {master}[edit]
    user@cs-edge-r01# set bridge-domains bd1 domain-type bridge
    user@cs-edge-r01# set bridge-domains bd1 vlan-id 601
    user@cs-edge-r01# set bridge-domains bd1 interface ae1.0
    user@cs-edge-r01# set bridge-domains bd1 interface ae3.0
    user@cs-edge-r01# set bridge-domains bd1 interface ae4.0
    user@cs-edge-r01# set bridge-domains bd1 routing-interface irb.601
    user@cs-edge-r01# set bridge-domains bd1 bridge-options interface ae4.0 static-mac 3c:8a:b0:cf:1f:f0
    

    On cs-edge-r02, enter:

    {master}[edit]
    user@cs-edge-r02# set bridge-domains bd1 domain-type bridge
    user@cs-edge-r02# set bridge-domains bd1 vlan-id 601
    user@cs-edge-r02# set bridge-domains bd1 interface ae1.0
    user@cs-edge-r02# set bridge-domains bd1 interface ae3.0
    user@cs-edge-r02# set bridge-domains bd1 interface ae4.0
    user@cs-edge-r02# set bridge-domains bd1 routing-interface irb.601
    user@cs-edge-r02# set bridge-domains bd1 bridge-options interface ae4.0 static-mac 3c:8a:b0:ce:0f:f0
    

    Note: The static-mac option on bridge domain 601 (bd1) prevents traffic arriving from the SRX chassis cluster from flooding the VLAN.

    The SRX chassis cluster sends traffic to both edge routers using the IRB 601 MAC address for routing the packet. The IRB 601 MAC addresses on cs-edge-r01 and cs-edge-r02 are different. Because the reth1 interface on the chassis cluster is a single LAG, the reth1 LAG address hashing results in a packet destined to the cs-edge-r01 MAC address being sent to cs-edge-r02. In an MC-LAG configuration, MAC address learning does not occur on the ICL link, and, as a result, cs-edge-r02 floods the packet on bridge domain 601. To avoid flooding on bridge domain 601, specify the MAC address for cs-edge-r01 in the static-mac option on cs-edge-r02 and vice versa. When a packet destined to cs-edge-r01 arrives at cs-edge-r02, cs-edge-r02 sends the packet to cs-edge-r01 using the static MAC address.

  2. Configure an IRB interface on the bridge domain and enable VRRP on the IRB interface.

    On cs-edge-r01, enter:

    {master}[edit]
    user@cs-edge-r01# set interfaces irb unit 601 family inet address 172.16.33.10/29 arp 172.16.33.11 l2-interface ae0.1
    user@cs-edge-r01# set interfaces irb unit 601 family inet address 172.16.33.10/29 arp 172.16.33.11 mac 3c:8a:b0:cf:1f:f0
    user@cs-edge-r01# set interfaces irb unit 601 family inet address 172.16.33.10/29 arp 172.16.33.11 publish
    user@cs-edge-r01# set interfaces irb unit 601 family inet address 172.16.33.10/29 vrrp-group 1 virtual-address 172.16.33.9
    user@cs-edge-r01# set interfaces irb unit 601 family inet address 172.16.33.10/29 vrrp-group 1 priority 250
    user@cs-edge-r01# set interfaces irb unit 601 family inet address 172.16.33.10/29 vrrp-group 1 preempt
    user@cs-edge-r01# set interfaces irb unit 601 family inet address 172.16.33.10/29 vrrp-group 1 accept-data
    user@cs-edge-r01# set interfaces irb unit 601 family inet address 172.16.33.10/29 vrrp-group 1 authentication-type md5
    user@cs-edge-r01# set interfaces irb unit 601 family inet address 172.16.33.10/29 vrrp-group 1 authentication-key "$9$nDoy9tOhSeX7V1R7VwYZG69A"
    

    On cs-edge-r02, enter:

    {master}[edit]
    user@cs-edge-r02# set interfaces irb unit 601 family inet address 172.16.33.11/29 arp 172.16.33.10 l2-interface ae0.1
    user@cs-edge-r02# set interfaces irb unit 601 family inet address 172.16.33.11/29 arp 172.16.33.10 mac 3c:8a:b0:ce:0f:f0
    user@cs-edge-r02# set interfaces irb unit 601 family inet address 172.16.33.11/29 arp 172.16.33.10 publish
    user@cs-edge-r02# set interfaces irb unit 601 family inet address 172.16.33.11/29 vrrp-group 1 virtual-address 172.16.33.9
    user@cs-edge-r02# set interfaces irb unit 601 family inet address 172.16.33.11/29 vrrp-group 1 priority 125
    user@cs-edge-r02# set interfaces irb unit 601 family inet address 172.16.33.11/29 vrrp-group 1 preempt
    user@cs-edge-r02# set interfaces irb unit 601 family inet address 172.16.33.11/29 vrrp-group 1 accept-data
    user@cs-edge-r02# set interfaces irb unit 601 family inet address 172.16.33.11/29 vrrp-group 1 authentication-type md5
    user@cs-edge-r02# set interfaces irb unit 601 family inet address 172.16.33.11/29 vrrp-group 1 authentication-key "$9$H.fz9A0hSe36SevW-dk.P"
    

Configure the High Availability Software

Step-by-Step Procedure

To enable graceful Routing Engine switchover (GRES), nonstop active routing (NSR), and nonstop bridging (NSB):

  • On both cs-edge-r01 and cs-edge-r02, enter the following configuration statements:
    {master}[edit]
    user@cs-edge-r01# set chassis redundancy graceful-switchover
    user@cs-edge-r01# set protocols layer2-control nonstop-bridging
    user@cs-edge-r01# set routing-options nonstop-routing
    

Verification

Confirm that the configuration is working properly.

Verifying the High Availability Configuration of the Access Switches

Purpose

Verify the Virtual Chassis, LAG, and high availability software configuration on the access switches.

Action

Perform the following steps for each Virtual Chassis in the access layer:

  1. Verify the Virtual Chassis status.
    user@cs-4200-ab1> show virtual-chassis status
    Preprovisioned Virtual Chassis
    Virtual Chassis ID: 0315.fd43.9a83
    Virtual Chassis Mode: Enabled
                                               Mstr           Mixed Neighbor List
    Member ID  Status   Serial No    Model     prio  Role      Mode ID  Interface
    0 (FPC 0)  Prsnt    BP0213230308 ex4200-48t   0  Linecard     N  2  vcp-0
                                                                     9  vcp-1
    1 (FPC 1)  Prsnt    BP0213260624 ex4200-48t 129  Master*      N  3  vcp-0
                                                                     2  vcp-1
    2 (FPC 2)  Prsnt    BP0213260668 ex4200-48t 129  Backup       N  1  vcp-0
                                                                     0  vcp-1
    3 (FPC 3)  Prsnt    BP0213260540 ex4200-48t   0  Linecard     N  4  vcp-0
                                                                     1  vcp-1
    4 (FPC 4)  Prsnt    BP0213260532 ex4200-48t   0  Linecard     N  5  vcp-0
                                                                     3  vcp-1
    5 (FPC 5)  Prsnt    BP0213230346 ex4200-48t   0  Linecard     N  6  vcp-0
                                                                     4  vcp-1
    6 (FPC 6)  Prsnt    FP0213313963 ex4200-48px   0 Linecard     N  7  vcp-0
                                                                     5  vcp-1
    7 (FPC 7)  Prsnt    BP0213310009 ex4200-48t   0  Linecard     N  8  vcp-0
                                                                     6  vcp-1
    8 (FPC 8)  Prsnt    BP0213260607 ex4200-48t   0  Linecard     N  9  vcp-0
                                                                     7  vcp-1
    9 (FPC 9)  Prsnt    BP0213230403 ex4200-48t   0  Linecard     N  0  vcp-0
                                                                     8  vcp-1
    
  2. Verify the LACP status of the uplink aggregated Ethernet interface.

    user@cs-4200-ab1> show lacp interfaces
    Aggregated interface: ae1
        LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
          xe-4/1/0       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
          xe-4/1/0     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
          xe-3/1/0       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
          xe-3/1/0     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
          xe-1/1/0       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
          xe-1/1/0     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
          xe-2/1/0       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
          xe-2/1/0     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
        LACP protocol:        Receive State  Transmit State          Mux State
          xe-4/1/0                  Current   Fast periodic Collecting distributing
          xe-3/1/0                  Current   Fast periodic Collecting distributing
          xe-1/1/0                  Current   Fast periodic Collecting distributing
          xe-2/1/0                  Current   Fast periodic Collecting distributing
    
  3. Verify that GRES is enabled by entering the following command on the backup Virtual Chassis member:
    user@cs-4200-ab1> show system switchover
    fpc2:
    --------------------------------------------------------------------------
    Graceful switchover: On
    Configuration database: Ready
    Kernel database: Ready
    Peer state: Steady State
    

Verifying the High Availability Configuration of the Aggregation Switches

Purpose

Verify the Virtual Chassis, LAG, and high availability software configuration on the EX4550 switches in location B.

Action

  1. Verify the Virtual Chassis status.
    user@cs-agg-01> show virtual-chassis status
    Preprovisioned Virtual Chassis
    Virtual Chassis ID: 0cf5.0cd4.e2f3
    Virtual Chassis Mode: Enabled
                                               Mstr           Mixed Neighbor List
    Member ID  Status   Serial No    Model     prio  Role      Mode ID  Interface
    0 (FPC 0)  Prsnt    LX0213439586 ex4550-32f 129  Master*      N  1  vcp-255/0/14
                                                                     1  vcp-255/0/15
    1 (FPC 1)  Prsnt    LX0213449606 ex4550-32f 129  Backup       N  0  vcp-255/0/14
                                                                     0  vcp-255/0/15
  2. Verify the LACP status of the aggregated Ethernet interfaces to the access switches.
    user@cs-agg-01> show lacp interfaces
    Aggregated interface: ae4
        LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
          xe-0/0/30      Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
          xe-0/0/30    Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
          xe-1/0/30      Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
          xe-1/0/30    Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
        LACP protocol:        Receive State  Transmit State          Mux State
          xe-0/0/30                 Current   Fast periodic Collecting distributing
          xe-1/0/30                 Current   Fast periodic Collecting distributing
    
    Aggregated interface: ae5
        LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
          ge-1/0/23      Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
          ge-1/0/23    Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
          ge-0/0/23      Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
          ge-0/0/23    Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
          ge-0/0/31      Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
          ge-0/0/31    Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
          ge-1/0/31      Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
          ge-1/0/31    Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
        LACP protocol:        Receive State  Transmit State          Mux State
          ge-1/0/23                 Current   Fast periodic Collecting distributing
          ge-0/0/23                 Current   Fast periodic Collecting distributing
          ge-0/0/31                 Current   Fast periodic Collecting distributing
          ge-1/0/31                 Current   Fast periodic Collecting distributing
    
  3. Verify that GRES is enabled by entering the following command on the backup Virtual Chassis member:
    {backup:1}
    user@cs-agg-01> show system switchover
    fpc1:
    --------------------------------------------------------------------------
    Graceful switchover: On
    Configuration database: Ready
    Kernel database: Ready
    Peer state: Steady State
    
  4. Verify that nonstop active routing is enabled.
    user@cs-agg-01> show task replication
    Stateful Replication: Enabled
            RE mode: Master
    
        Protocol                Synchronization Status
        OSPF                    Complete
        PIM                     Complete
    

    Note: If you have not configured routing yet, you might not see the protocols and their synchronization status listed.

Verifying the High Availability Configuration of the Core Switches

Purpose

Verify the MC-LAG configuration and high availability software configuration on the core switches.

Action

Perform the following steps on both cs-core-sw01 and cs-core-sw02:

  1. Verify that ICCP is configured.
    user@cs-core-sw01> show iccp
    Redundancy Group Information for peer 172.16.32.6
      TCP Connection       : Established
      Liveliness Detection : Up
      Redundancy Group ID          Status
        1                           Up
    
    Client Application: MCSNOOPD
      Redundancy Group IDs Joined: 1
    
    Client Application: lacpd
      Redundancy Group IDs Joined: 1
    
    Client Application: l2ald_iccpd_client
      Redundancy Group IDs Joined: 1
    
  2. Verify that the ICL link has been configured with membership in all the VLANs.
    user@cs-core-sw01> show configuration interfaces ae29
    description "ICL Layer 2 link with 2 members,xe-0/3/7,1/3/7";
    vlan-tagging;
    aggregated-ether-options {
        lacp {
            active;
            periodic fast;
        }
    }
    unit 0 {
        family ethernet-switching {
            interface-mode trunk;
            vlan {
                members all;
            }
        }
    }
    
  3. Verify the status of the ICL link.
    user@cs-core-sw01> show interfaces ae29 extensive
    Physical interface: ae29, Enabled, Physical link is Up
      Interface index: 157, SNMP ifIndex: 738, Generation: 160
      Description: ICL Layer 2 link with 2 members,xe-0/3/7,1/3/7
      Link-level type: Ethernet, MTU: 1518, Speed: 20Gbps, BPDU Error: None, MAC-REWRITE Error: None,
      Loopback: Disabled, Source filtering: Disabled, Flow control: Disabled, Minimum links needed: 1,
      Minimum bandwidth needed: 0
      Device flags   : Present Running
      Interface flags: SNMP-Traps Internal: 0x4000
      Current address: 2a:8a:1c:e3:f1:46, Hardware address: 2a:8a:1c:e3:f1:46
      Last flapped   : 2014-06-02 03:34:58 PDT (14:50:58 ago)
      Statistics last cleared: 2014-06-02 18:23:16 PDT (00:02:40 ago)
      Traffic statistics:
       Input  bytes  :            102872394              5144888 bps
       Output bytes  :            103878646              5145608 bps
       Input  packets:               830281                 5206 pps
       Output packets:               845410                 5238 pps
       IPv6 transit statistics:
        Input  bytes  :                   0
        Output bytes  :                   0
        Input  packets:                   0
        Output packets:                   0
      Dropped traffic statistics due to STP State:
       Input  bytes  :                    0
       Output bytes  :                    0
       Input  packets:                    0
       Output packets:                    0
      Input errors:
        Errors: 0, Drops: 0, Framing errors: 0, Runts: 0, Giants: 0, Policed discards: 0,
        Resource errors: 0
      Output errors:
        Carrier transitions: 0, Errors: 0, Drops: 0, MTU errors: 0, Resource errors: 0
      Ingress queues: 8 supported, 6 in use
      Queue counters:       Queued packets  Transmitted packets      Dropped packets
        0 Best-Effort                    0                    0                    0
        1 Mission-Crit                   0                    0                    0
        2 assured-forw                   0                    0                    0
        3 Video                          0                    0                    0
        5 Voice                          0                    0                    0
        7 Network-Cont                   0                    0                    0
      Egress queues: 8 supported, 6 in use
      Queue counters:       Queued packets  Transmitted packets      Dropped packets
        0 Best-Effort               825903               825903                    0
        1 Mission-Crit                   0                    0                    0
        2 assured-forw                   0                    0                    0
        3 Video                      19696                19696                    0
        5 Voice                          0                    0                    0
        7 Network-Cont                 131                  131                    0
      Queue number:         Mapped forwarding classes
        0                   Best-Effort
        1                   Mission-Critical
        2                   assured-forwarding
        3                   Video
        5                   Voice
        7                   Network-Control
    
      Logical interface ae29.0 (Index 338) (SNMP ifIndex 744) (Generation 147)
        Flags: SNMP-Traps 0x20024000 Encapsulation: Ethernet-Bridge
        Statistics        Packets        pps         Bytes          bps
        Bundle:
            Input :        829950       5204     102829392      5142904
            Output:        845398       5216     104067298      5131000
        Link:
          xe-1/3/7.0
            Input :        413483       2609      51119271      2580464
            Output:        420951       2616      51342053      2544200
          xe-0/3/7.0
            Input :        416467       2595      51710121      2562440
            Output:        424447       2600      52725245      2586800
        Marker Statistics:   Marker Rx     Resp Tx   Unknown Rx   Illegal Rx
          xe-1/3/7.0                 0           0            0            0
          xe-0/3/7.0                 0           0            0            0
        Protocol eth-switch, MTU: 1518, Generation: 169, Route table: 5
          Flags: Trunk-Mode
    
      Logical interface ae29.32767 (Index 337) (SNMP ifIndex 806) (Generation 146)
        Flags: SNMP-Traps 0x4004000 VLAN-Tag [ 0x0000.0 ]  Encapsulation: ENET2
        Statistics        Packets        pps         Bytes          bps
        Bundle:
            Input :           320          2         39680         1984
            Output:             0          0             0            0
        Link:
          xe-1/3/7.32767
            Input :           159          1         19716          992
            Output:             6          0          1812            0
          xe-0/3/7.32767
            Input :           161          1         19964          992
            Output:             6          0          1812            0
        LACP info:        Role     System             System      Port    Port  Port
                                 priority          identifier  priority  number   key
          xe-1/3/7.32767    Actor      127 28:8a:1c:e3:f7:c0       127      23    30
          xe-1/3/7.32767  Partner      127 28:8a:1c:e5:3b:c0       127      23    30
          xe-0/3/7.32767    Actor      127 28:8a:1c:e3:f7:c0       127      11    30
          xe-0/3/7.32767  Partner      127 28:8a:1c:e5:3b:c0       127      11    30
        LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx
          xe-1/3/7.32767           150         150            0            0
          xe-0/3/7.32767           150         150            0            0
        Marker Statistics:   Marker Rx     Resp Tx   Unknown Rx   Illegal Rx
          xe-1/3/7.32767             0           0            0            0
          xe-0/3/7.32767             0           0            0            0
        Protocol multiservice, MTU: Unlimited, Generation: 168, Route table: 0
          Flags: None
    
  4. Verify that all the MC-LAG interfaces are up.
    user@cs-core-sw01> show interfaces mc-ae
    Member Link                  : ae1
     Current State Machine's State: mcae active state
     Local Status                 : active
     Local State                  : up
     Peer Status                  : active
     Peer State                   : up
         Logical Interface        : ae1.0
         Topology Type            : bridge
         Local State              : up
         Peer State               : up
         Peer Ip/MCP/State        : 172.16.32.6 ae29.0 up
    
     Member Link                  : ae2
     Current State Machine's State: mcae active state
     Local Status                 : active
     Local State                  : up
     Peer Status                  : active
     Peer State                   : up
         Logical Interface        : ae2.0
         Topology Type            : bridge
         Local State              : up
         Peer State               : up
         Peer Ip/MCP/State        : 172.16.32.6 ae29.0 up
    
     Member Link                  : ae3
     Current State Machine's State: mcae active state
     Local Status                 : active
     Local State                  : up
     Peer Status                  : active
     Peer State                   : up
         Logical Interface        : ae3.0
         Topology Type            : bridge
         Local State              : up
         Peer State               : up
         Peer Ip/MCP/State        : 172.16.32.6 ae29.0 up
    
     Member Link                  : ae7
     Current State Machine's State: mcae active state
     Local Status                 : active
     Local State                  : up
     Peer Status                  : active
     Peer State                   : up
         Logical Interface        : ae7.0
         Topology Type            : bridge
         Local State              : up
         Peer State               : up
         Peer Ip/MCP/State        : 172.16.32.6 ae29.0 up
    
     Member Link                  : ae11
     Current State Machine's State: mcae active state
     Local Status                 : active
     Local State                  : up
     Peer Status                  : active
     Peer State                   : up
         Logical Interface        : ae11.0
         Topology Type            : bridge
         Local State              : up
         Peer State               : up
         Peer Ip/MCP/State        : 172.16.32.6 ae29.0 up
    
     Member Link                  : ae12
     Current State Machine's State: mcae active state
     Local Status                 : active
     Local State                  : up
     Peer Status                  : active
     Peer State                   : up
         Logical Interface        : ae12.0
         Topology Type            : bridge
         Local State              : up
         Peer State               : up
         Peer Ip/MCP/State        : 172.16.32.6 ae29.0 up
    
     Member Link                  : ae30
     Current State Machine's State: mcae active state
     Local Status                 : active
     Local State                  : up
     Peer Status                  : active
     Peer State                   : up
         Logical Interface        : ae30.0
         Topology Type            : bridge
         Local State              : up
         Peer State               : up
         Peer Ip/MCP/State        : 172.16.32.6 ae29.0 up
    
     Member Link                  : ae31
     Current State Machine's State: mcae active state
     Local Status                 : active
     Local State                  : up
     Peer Status                  : active
     Peer State                   : up
         Logical Interface        : ae31.0
         Topology Type            : bridge
         Local State              : up
         Peer State               : up
         Peer Ip/MCP/State        : 172.16.32.6 ae29.0 up
        
    
  5. Verify that ICL (ae29) and the MC-LAG interfaces are in the same broadcast domains.

    In following example, the broadcast domain eng1_data_wired is used.

    user@cs-core-sw01> show vlans eng1_data_wired
    Routing instance        VLAN name             Tag          Interfaces
    default-switch          eng1_data_wired       60
                                                               ae1.0* 
                                                               
                                                               ae29.0* 
    
  6. Verify the status of VRRP.

    1. On cs-core-sw01, enter:
      user@cs-core-sw01> show vrrp summary
      Interface     State       Group   VR state       VR Mode    Type   Address
      irb.10        up              1   master          Active    lcl    10.16.0.3
                                                                  vip    10.16.0.1
      irb.11        up              1   master          Active    lcl    10.16.16.3
                                                                  vip    10.16.16.1
      irb.12        up              1   master          Active    lcl    10.16.32.3
                                                                  vip    10.16.32.1
      irb.13        up              1   master          Active    lcl    10.16.48.3
                                                                  vip    10.16.48.1
      irb.20        up              1   master          Active    lcl    10.17.0.3
                                                                  vip    10.17.0.1
      irb.21        up              1   master          Active    lcl    10.17.4.3
                                                                  vip    10.17.4.1
      irb.22        up              1   master          Active    lcl    10.17.8.3
                                                                  vip    10.17.8.1
      irb.23        up              1   master          Active    lcl    10.17.12.3
                                                                  vip    10.17.12.1
      irb.30        up              1   master          Active    lcl    10.17.64.3
                                                                  vip    10.17.64.1
      irb.31        up              1   master          Active    lcl    10.17.68.3
                                                                  vip    10.17.68.1
      irb.32        up              1   master          Active    lcl    10.17.72.3
                                                                  vip    10.17.72.1
      irb.33        up              1   master          Active    lcl    10.17.76.3
                                                                  vip    10.17.76.1
      irb.40        up              1   master          Active    lcl    10.17.128.3
                                                                  vip    10.17.128.1
      irb.41        up              1   master          Active    lcl    10.17.132.3
                                                                  vip    10.17.132.1
      irb.42        up              1   master          Active    lcl    10.17.136.3
                                                                  vip    10.17.136.1
      irb.43        up              1   master          Active    lcl    10.17.140.3
                                                                  vip    10.17.140.1
      irb.50        up              1   master          Active    lcl    10.18.0.3
                                                                  vip    10.18.0.1
      irb.51        up              1   master          Active    lcl    10.18.16.3
                                                                  vip    10.18.16.1
      irb.52        up              1   master          Active    lcl    10.18.32.3
                                                                  vip    10.18.32.1
      irb.53        up              1   master          Active    lcl    10.18.48.3
                                                                  vip    10.18.48.1
      irb.60        up              1   master          Active    lcl    10.32.0.3
                                                                  vip    10.32.0.1
      irb.61        up              1   master          Active    lcl    10.32.16.3
                                                                  vip    10.32.16.1
      irb.62        up              1   master          Active    lcl    10.32.32.3
                                                                  vip    10.32.32.1
      irb.63        up              1   master          Active    lcl    10.32.48.3
                                                                  vip    10.32.48.1
      irb.70        up              1   master          Active    lcl    10.33.0.3
                                                                  vip    10.33.0.1
      irb.71        up              1   master          Active    lcl    10.33.16.3
                                                                  vip    10.33.16.1
      irb.72        up              1   master          Active    lcl    10.33.32.3
                                                                  vip    10.33.32.1
      irb.73        up              1   master          Active    lcl    10.33.48.3
                                                                  vip    10.33.48.1
      irb.80        up              1   master          Active    lcl    10.34.0.3
                                                                  vip    10.34.0.1
      irb.81        up              1   master          Active    lcl    10.34.16.3
                                                                  vip    10.34.16.1
      irb.82        up              1   master          Active    lcl    10.34.32.3
                                                                  vip    10.34.32.1
      irb.83        up              1   master          Active    lcl    10.34.48.3
                                                                  vip    10.34.48.1
      irb.90        up              1   master          Active    lcl    10.35.0.3
                                                                  vip    10.35.0.1
      irb.91        up              1   master          Active    lcl    10.35.16.3
                                                                  vip    10.35.16.1
      irb.92        up              1   master          Active    lcl    10.35.32.3
                                                                  vip    10.35.32.1
      irb.93        up              1   master          Active    lcl    10.35.48.3
                                                                  vip    10.35.48.1
      irb.100       up              1   master          Active    lcl    10.36.0.3
                                                                  vip    10.36.0.1
      irb.101       up              1   master          Active    lcl    10.36.16.3
                                                                  vip    10.36.16.1
      irb.102       up              1   master          Active    lcl    10.36.32.3
                                                                  vip    10.36.32.1
      irb.103       up              1   master          Active    lcl    10.36.48.3
                                                                  vip    10.36.48.1
      irb.201       up              1   master          Active    lcl    172.16.128.3
                                                                  vip    172.16.128.1
      irb.399       up              1   master          Active    lcl    172.16.12.3
                                                                  vip    172.16.12.1
      irb.400       up              1   master          Active    lcl    172.16.35.67
                                                                  vip    172.16.35.65
      irb.500       up              1   master          Active    lcl    172.16.11.3
                                                                  vip    172.16.11.1
      irb.600       up              1   master          Active    lcl    172.16.33.3
                                                                  vip    172.16.33.1
      irb.786       up              1   master          Active    lcl    78.1.1.3
                                                                  vip    78.1.1.1
      irb.1000      up              1   master          Active    lcl    3.3.0.3
                                                                  vip    3.3.0.1
      irb.1001      up              1   master          Active    lcl    3.4.0.3
                                                                  vip    3.4.0.1 
      
    2. On cs-core-sw02, enter:
      user@cs-core-sw02> show vrrp summary
      Interface     State       Group   VR state       VR Mode    Type   Address
      irb.10        up              1   backup          Active    lcl    10.16.0.2
                                                                  vip    10.16.0.1
      irb.11        up              1   backup          Active    lcl    10.16.16.2
                                                                  vip    10.16.16.1
      irb.12        up              1   backup          Active    lcl    10.16.32.2
                                                                  vip    10.16.32.1
      irb.13        up              1   backup          Active    lcl    10.16.48.2
                                                                  vip    10.16.48.1
      irb.20        up              1   backup          Active    lcl    10.17.0.2
                                                                  vip    10.17.0.1
      irb.21        up              1   backup          Active    lcl    10.17.4.2
                                                                  vip    10.17.4.1
      irb.22        up              1   backup          Active    lcl    10.17.8.2
                                                                  vip    10.17.8.1
      irb.23        up              1   backup          Active    lcl    10.17.12.2
                                                                  vip    10.17.12.1
      irb.30        up              1   backup          Active    lcl    10.17.64.2
                                                                  vip    10.17.64.1
      irb.31        up              1   backup          Active    lcl    10.17.68.2
                                                                  vip    10.17.68.1
      irb.32        up              1   backup          Active    lcl    10.17.72.2
                                                                  vip    10.17.72.1
      irb.33        up              1   backup          Active    lcl    10.17.76.2
                                                                  vip    10.17.76.1
      irb.40        up              1   backup          Active    lcl    10.17.128.2
                                                                  vip    10.17.128.1
      irb.41        up              1   backup          Active    lcl    10.17.132.2
                                                                  vip    10.17.132.1
      irb.42        up              1   backup          Active    lcl    10.17.136.2
                                                                  vip    10.17.136.1
      irb.43        up              1   backup          Active    lcl    10.17.140.2
                                                                  vip    10.17.140.1
      irb.50        up              1   backup          Active    lcl    10.18.0.2
                                                                  vip    10.18.0.1
      irb.51        up              1   backup          Active    lcl    10.18.16.2
                                                                  vip    10.18.16.1
      irb.52        up              1   backup          Active    lcl    10.18.32.2
                                                                  vip    10.18.32.1
      irb.53        up              1   backup          Active    lcl    10.18.48.2
                                                                  vip    10.18.48.1
      irb.60        up              1   backup          Active    lcl    10.32.0.2
                                                                  vip    10.32.0.1
      irb.61        up              1   backup          Active    lcl    10.32.16.2
                                                                  vip    10.32.16.1
      irb.62        up              1   backup          Active    lcl    10.32.32.2
                                                                  vip    10.32.32.1
      irb.63        up              1   backup          Active    lcl    10.32.48.2
                                                                  vip    10.32.48.1
      irb.70        up              1   backup          Active    lcl    10.33.0.2
                                                                  vip    10.33.0.1
      irb.71        up              1   backup          Active    lcl    10.33.16.2
                                                                  vip    10.33.16.1
      irb.72        up              1   backup          Active    lcl    10.33.32.2
                                                                  vip    10.33.32.1
      irb.73        up              1   backup          Active    lcl    10.33.48.2
                                                                  vip    10.33.48.1
      irb.80        up              1   backup          Active    lcl    10.34.0.2
                                                                  vip    10.34.0.1
      irb.81        up              1   backup          Active    lcl    10.34.16.2
                                                                  vip    10.34.16.1
      irb.82        up              1   backup          Active    lcl    10.34.32.2
                                                                  vip    10.34.32.1
      irb.83        up              1   backup          Active    lcl    10.34.48.2
                                                                  vip    10.34.48.1
      irb.90        up              1   backup          Active    lcl    10.35.0.2
                                                                  vip    10.35.0.1
      irb.91        up              1   backup          Active    lcl    10.35.16.2
                                                                  vip    10.35.16.1
      irb.92        up              1   backup          Active    lcl    10.35.32.2
                                                                  vip    10.35.32.1
      irb.93        up              1   backup          Active    lcl    10.35.48.2
                                                                  vip    10.35.48.1
      irb.100       up              1   backup          Active    lcl    10.36.0.2
                                                                  vip    10.36.0.1
      irb.101       up              1   backup          Active    lcl    10.36.16.2
                                                                  vip    10.36.16.1
      irb.102       up              1   backup          Active    lcl    10.36.32.2
                                                                  vip    10.36.32.1
      irb.103       up              1   backup          Active    lcl    10.36.48.2
                                                                  vip    10.36.48.1
      irb.201       up              1   backup          Active    lcl    172.16.128.2
                                                                  vip    172.16.128.1
      irb.399       up              1   backup          Active    lcl    172.16.12.2
                                                                  vip    172.16.12.1
      irb.400       up              1   backup          Active    lcl    172.16.35.66
                                                                  vip    172.16.35.65
      irb.500       up              1   backup          Active    lcl    172.16.11.2
                                                                  vip    172.16.11.1
      irb.600       up              1   backup          Active    lcl    172.16.33.2
                                                                  vip    172.16.33.1
      irb.786       up              1   backup          Active    lcl    78.1.1.2
                                                                  vip    78.1.1.1
      irb.1000      up              1   backup          Active    lcl    3.3.0.2
                                                                  vip    3.3.0.1
      irb.1001      up              1   backup          Active    lcl    3.4.0.2
                                                                  vip    3.4.0.1
      irb.1001      up              1   master          Active    lcl    3.4.0.3
                                                                  vip    3.4.0.1 
      
  7. Verify that GRES is enabled.

    1. On the backup Routing Engine of cs-core-sw01, enter:
      user@cs-core-sw01-1> show system switchover
      Graceful switchover: On
      Configuration database: Ready
      Kernel database: Synchronizing
      Peer state: Steady State
      
    2. On the backup Routing Engine of cs-core-sw02, enter:
      user@cs-core-sw02-1> show system switchover
      Graceful switchover: On
      Configuration database: Ready
      Kernel database: Ready
      Peer state: Steady State
      
  8. Verify that nonstop active routing is enabled.
    user@cs-core-sw01> show task replication
    Stateful Replication: Enabled
            RE mode: Master
    
        Protocol                Synchronization Status
        OSPF                    Complete
        PIM                     Complete
    

    Note: If you have not configured routing yet, you might not see the protocols and their synchronization status listed.

Verifying the High Availability Configuration of the Edge Firewalls

Purpose

Verify the chassis cluster configuration and the status of the control, fabric, and redundant Ethernet interfaces.

Action

  1. Verify the chassis cluster configuration and status.
    user@cs-edge-fw01-node0> show chassis cluster status
    Cluster ID: 1
    Node                  Priority          Status    Preempt  Manual failover
    
    Redundancy group: 0 , Failover count: 1
        node0                   100         primary        no       no
        node1                   1           secondary      no       no
    
    Redundancy group: 1 , Failover count: 1
        node0                   100         primary        no       no
        node1                   1           secondary      no       no
    
  2. Verify the status of the control, fabric, and redundant Ethernet interfaces.
    user@cs-edge-fw01-node0> show chassis cluster interfaces
    Control link status: Up
    
    Control interfaces:
        Index   Interface        Status
        0       fxp1             Up
    
    Fabric link status: Up
    
    Fabric interfaces:
        Name    Child-interface    Status
                                   (Physical/Monitored)
        fab0    ge-0/0/2           Up   / Up
        fab0
        fab1    ge-9/0/2           Up   / Up
        fab1
    
    Redundant-ethernet Information:
        Name         Status      Redundancy-group
        reth0        Up          1
        reth1        Up          1
        reth2        Down        Not configured
        reth3        Down        Not configured
    
    Redundant-pseudo-interface Information:
        Name         Status      Redundancy-group
        lo0          Up          0
    

Verifying the High Availability Configuration of the Edge Routers

Purpose

Verify the status of the MC-LAG interfaces and that the router is forwarding traffic to the SRX chassis cluster correctly.

Action

Perform the following steps on both cs-edge-r01 and cs-edge-r02.

  1. Verify the status of the MC-LAG interfaces.
    user@cs-edge-r01> show interfaces mc-ae
     Member Link                  : ae1
     Current State Machine's State: mcae active state
     Local Status                 : active
     Local State                  : up
     Peer Status                  : active
     Peer State                   : up
         Logical Interface        : ae1.0
         Topology Type            : bridge
         Local State              : up
         Peer State               : up
         Peer Ip/MCP/State        : 172.16.32.34 ae0.1 up
    
     Member Link                  : ae3
     Current State Machine's State: mcae active state
     Local Status                 : active
     Local State                  : up
     Peer Status                  : active
     Peer State                   : up
         Logical Interface        : ae3.0
         Topology Type            : bridge
         Local State              : up
         Peer State               : up
         Peer Ip/MCP/State        : 172.16.32.34 ae0.1 up
    
  2. Verify that the router is forwarding traffic to the active firewall node, based on the gratuitous ARP message sent by the active node.
    1. Display route information for 172.16.4.0/24.
      user@cs-edge-r01> show route 172.16.4.0/24
      inet.0: 3165 destinations, 3166 routes (3165 active, 0 holddown, 0 hidden)
      + = Active Route, - = Last Active, * = Both
      
      172.16.4.0/24      *[OSPF/10] 00:04:30, metric 1601
                          > to 172.16.33.12 via irb.601
      
      
    2. Check the forwarding table to see if the next hop and interface are chosen correctly.
      user@cs-edge-r01> show route forwarding-table destination 172.16.4.0/24
      Routing table: default.inet
      Internet:
      Destination        Type RtRef Next hop           Type Index    NhRef Netif
      172.16.4.0/24      user     0 172.16.33.12       ucst      607  3148 ae1.0
      
      Routing table: __master.anon__.inet
      Internet:
      Destination        Type RtRef Next hop           Type Index    NhRef Netif
      default            perm     0                    rjct      519     1
      
  3. Verify the LACP state of the LAG interfaces.

    Both LAGs should be up, even though only the LAG connecting to the active firewall node forwards traffic.

    1. Show the LACP state for interface ae1.
      user@cs-edge-r01> show lacp interfaces ae1
      Aggregated interface: ae1
          LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
            ge-1/0/0       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
            ge-1/0/0     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
          LACP protocol:        Receive State  Transmit State          Mux State
            ge-1/0/0                  Current   Fast periodic Collecting distributing
      
    2. Show the LACP state for interface ae3.
      user@cs-edge-r01> show lacp interfaces ae3
      Aggregated interface: ae3
          LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
            ge-1/0/1       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
            ge-1/0/1     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
          LACP protocol:        Receive State  Transmit State          Mux State
            ge-1/0/1                  Current   Fast periodic Collecting distributing
      
  4. Verify that nonstop active routing is enabled.
    user@cs-edge-r01> show task replication
    Stateful Replication: Enabled
            RE mode: Master
    
        Protocol                Synchronization Status
        OSPF                    Complete
        BGP                     Complete
    

    Note: If you have not configured routing yet, you might not see the protocols and their synchronization status listed.

Related Documentation

Modified: 2016-11-08