Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Navigation
Guide That Contains This Content
[+] Expand All
[-] Collapse All

    Example: Configuring EX4200 Switches for the QFX3000-M QFabric System Control Plane

    This example shows you how to connect QFabric system components and configure the EX4200 switches used for the QFX3000-M QFabric system control plane network. Proper wiring of Director devices, Interconnect devices, and Node devices to the EX4200 switches, combined with a standard configuration, enables you to bring up the internal QFabric system management network and prepare your QFabric system for full operation.

    Note: The EX4200 switch configuration is the same for both the copper-based and fiber-based QFX3000-M QFabric system control plane networks. Hence, a separate example for configuring EX4200 switches for the fiber-based control plane network is not provided.

    However, because you cannot mix and match fiber and copper in the same control plane network, you must select only one type of control plane for each QFX3000-M QFabric system you install. The primary focus of this example is a copper-based control plane network. Before you use this example to configure a fiber-based control plane network, ensure that you have installed and wired the QFabric system hardware and EX4200 switches as required for a fiber-based control plane network (see QFX3000-M QFabric System Installation Overview).

    Requirements

    This example uses the following hardware and software components:

    • One QFX3000-M QFabric system containing:
      • Two QFX3100 Director devices with 1000BASE-T network modules installed
      • Two QFX3600-I Interconnect devices
      • Eight QFX3500 Node devices with a 1000BASE-T management board installed
    • Two EX4200-24T switches with SFP+ uplink module installed
    • Junos OS Release 13.2X52-D10 for the QFabric system components
    • Junos OS Release 12.3R6.6 for the EX Series switches

    Before you begin:

    Overview

    The QFX3000-M QFabric system control plane network connects the Director group, Interconnect devices, and Node devices in a QFabric system across a pair of redundant EX4200 switches. By separating the management control plane from the data plane, the QFabric system can scale efficiently. The copper-based control plane network uses Gigabit Ethernet cabling and connections between components, and two 1-Gigabit Ethernet connections configured in a link aggregation group (LAG) between the redundant EX4200 switches.

    Specific ports have been reserved on the EX4200 switches to connect to each of the QFabric system device types. Such design simplifies installation and facilitates timely deployment of a QFabric system. It also permits the use of a standard EX4200 switch configuration included as part of this example. The standard configuration can scale from the 8 Node devices shown in this example to a maximum of 16 Node devices.

    Topology

    Figure 1 shows the general port ranges where QFabric system devices must be connected to the EX4200 switches. For each EX4200 switch, connect ports 0 through 15 to Node devices, ports 16 through 19 to Interconnect devices, ports 20 through 23 to Director devices, and uplink ports 0 and 1 to the other control plane switch as an inter-switch LAG. Table 1 shows the details of the QFabric system component-to-EX4200 switch port mappings.

    Figure 1: QFX3000-M QFabric System Control Plane—EX4200 Switch Port Ranges

    QFX3000-M QFabric System Control Plane—EX4200
Switch Port Ranges

    Caution:

    • The control plane network within a QFabric system is a critical component of the system that should not be shared with other network traffic. In order to scale efficiently, the control plane network must be reserved for the QFabric system and its components. As a result, the ports of the QFabric system control plane must never be used for any purpose other than to transport QFabric system control plane traffic, and we neither recommend nor support the connection of other devices to the QFabric system control plane network.
    • Do not install Junos Space and AI-Scripts (AIS) on the control plane network EX4200 switches in a QFX3000-M QFabric system.

    Table 1 shows the specific mappings of QFabric system control plane network ports from the QFabric system components to the EX4200 switches.

    Note: The uplink ports 2 and 3 on the EX4200 switches are reserved for future use.

    Table 1: QFX3000-M QFabric System Copper-Based Control Plane—QFabric Component-to-EX4200 Switch Port Mappings

    EX4200 Switch 1 (EX0)

    EX4200 Switch 2 (EX1)

    QFabric System Component

    Node0, management port C0 to port 0
    (ge-0/0/0)

    Node0, management port C1 to port 0
    (ge-0/0/0)

    Node device 0

    Node1, management port C0 to port 1
    (ge-0/0/1)

    Node1, management port C1 to port 1
    (ge-0/0/1)

    Node device 1

    Node2, management port C0 to port 2
    (ge-0/0/2)

    Node2, management port C1 to port 2
    (ge-0/0/2)

    Node device 2

    Node3, management port C0 to port 3
    (ge-0/0/3)

    Node3, management port C1 to port 3
    (ge-0/0/3)

    Node device 3

    Node4, management port C0 to port 4
    (ge-0/0/4)

    Node4, management port C1 to port 4
    (ge-0/0/4)

    Node device 4

    Node5, management port C0 to port 5
    (ge-0/0/5)

    Node5, management port C1 to port 5
    (ge-0/0/5)

    Node device 5

    Node6, management port C0 to port 6
    (ge-0/0/6)

    Node6, management port C1 to port 6
    (ge-0/0/6)

    Node device 6

    Node7, management port C0 to port 7
    (ge-0/0/7)

    Node7, management port C1 to port 7
    (ge-0/0/7)

    Node device 7

    Node8, management port C0 to port 8
    (ge-0/0/8)

    Node8, management port C1 to port 8
    (ge-0/0/8)

    Node device 8

    Node9, management port C0 to port 9
    (ge-0/0/9)

    Node9, management port C1 to port 9
    (ge-0/0/9)

    Node device 9

    Node10, management port C0 to port 10
    (ge-0/0/10)

    Node10, management port C1 to port 10
    (ge-0/0/10)

    Node device 10

    Node11, management port C0 to port 11
    (ge-0/0/11)

    Node11, management port C1 to port 11
    (ge-0/0/11)

    Node device 11

    Node12, management port C0 to port 12
    (ge-0/0/12)

    Node12, management port C1 to port 12
    (ge-0/0/12)

    Node device 12

    Node13, management port C0 to port 13
    (ge-0/0/13)

    Node13, management port C1 to port 13
    (ge-0/0/13)

    Node device 13

    Node14, management port C0 to port 14
    (ge-0/0/14)

    Node14, management port C1 to port 14
    (ge-0/0/14)

    Node device 14

    Node15, management port C0 to port 15
    (ge-0/0/15)

    Node15, management port C1 to port 15
    (ge-0/0/15)

    Node device 15

    IC0, management port C0 to port 16
    (ge-0/0/16)

    IC0, management port C1 to port 16
    (ge-0/0/16)

    Interconnect device 0

    IC1, management port C0 to port 17
    (ge-0/0/17)

    IC1, management port C1 to port 17
    (ge-0/0/17)

    Interconnect device 1

    IC2, management port C0 to port 18
    (ge-0/0/18)

    IC2, management port C1 to port 18
    (ge-0/0/18)

    Interconnect device 2

    IC3, management port C0 to port 19
    (ge-0/0/19)

    IC3, management port C1 to port 19
    (ge-0/0/19)

    Interconnect device 3

    DG0 module 0, port 0 to port 20
    (ge-0/0/20)

    DG0 module 1, port 0 to port 20
    (ge-0/0/20)

    Director device 0

    DG0 module 0, port 1 to port 21
    (ge-0/0/21)

    DG0 module 1, port 1 to port 21
    (ge-0/0/21)

    Director device 0

    DG1 module 0, port 0 to port 22
    (ge-0/0/22)

    DG1 module 1, port 0 to port 22
    (ge-0/0/22)

    Director device 1

    DG1 module 0, port 1 to port 23
    (ge-0/0/23)

    DG1 module 1, port 1 to port 23
    (ge-0/0/23)

    Director device 1

    EX0, uplink port 0 to EX1, uplink port 0
    (ge-0/1/0)

    EX1, uplink port 0 to EX0, uplink port 0
    (ge-0/1/0)

    Inter-EX4200 switch LAG

    EX0, uplink port 1 to EX1, uplink port 1
    (ge-0/1/1)

    EX1, uplink port 1 to EX0, uplink port 1
    (ge-0/1/1)

    Inter-EX4200 switch LAG

    Reserved

    Uplink port 2 (ge-0/1/2)

    Reserved

    Uplink port 2 (ge-0/1/2)

    Future use

    Reserved

    Uplink port 3 (ge-0/1/3)

    Reserved

    Uplink port 3 (ge-0/1/3)

    Future use

    Next, connect the Director devices to the EX4200 switches. In general, you want to accomplish the following:

    • Connect two ports from one network module in a Director device to the first EX4200 switch, and two ports from the second network module to the second EX4200 switch.
    • Connect the Director devices to each other and create a Director group. You can use either straight-through RJ-45 patch cables or crossover cables, because the Director devices contain autosensing modules. Connect one port from each network module on the first Director device to one port in each network module on the second Director device.

    Figure 2 shows the specific ports on the Director group that you must connect to the EX4200 switches and interconnect between the Director devices.

    Figure 2: QFX3000-M QFabric System Control Plane—Director Group to EX4200 Switch Connections

    QFX3000-M QFabric System Control Plane—Director
Group to EX4200 Switch Connections

    In this specific example, connect ports 0 and 1 from module 0 on Director device DG0 to ports 20 and 21 on EX4200 switch EX0 (ge-0/0/20 and ge-0/0/21), and connect ports 0 and 1 from module 1 to ports 20 and 21 on the second EX4200 switch EX1 (ge-0/0/20 and ge-0/0/21).

    For Director device DG1, connect ports 0 and 1 from module 0 to ports 22 and 23 on EX4200 switch EX0 (ge-0/0/22 and ge-0/0/23), and connect ports 0 and 1 from module 1 to ports 22 and 23 on the second EX4200 switch EX1 (ge-0/0/22 and ge-0/0/23).

    To form the Director group, connect port 3 on module 0 on Director device DG0 to port 3 on module 0 on Director device DG1. Similarly, connect port 3 on module 1 on Director device DG0 to port 3 on module 1 on Director device DG1. Table 2 shows the port mappings for the Director group in this example.

    Table 2: Director Group Port Mappings

    Director Device

    EX4200 Switch EX0

    EX4200 Switch EX1

    DG0

    • DG0 module 0, port 0 to port 20 on EX0
      (ge-0/0/20)
    • DG0 module 0, port 1 to port 21 on EX0
      (ge-0/0/21)
    • DG0 module 0, port 3 to module 0, port 3 on DG1
    • DG0 module 1, port 0 to port 20 on EX1
      (ge-0/0/20)
    • DG0 module 1, port 1 to port 21 on EX1
      (ge-0/0/21)
    • DG0 module 1, port 3 to module 1, port 3 on DG1

    DG1

    • DG1 module 0, port 0 to port 22 on EX0
      (ge-0/0/22)
    • DG1 module 0, port 1 to port 23 on EX0
      (ge-0/0/23)
    • DG1 module 0, port 3 to module 0, port 3 on DG0
    • DG1 module 1, port 0 to port 22 on EX1
      (ge-0/0/22)
    • DG1 module 1, port 1 to port 23 on EX1
      (ge-0/0/23)
    • DG1 module 1, port 3 to module 1, port 3 on DG0

    In the software, the ports of each network module on a Director device are reversed, numbered from right to left, and incremented sequentially across modules. If you issue interface operational commands directly on the Director device, note the following port mappings as shown in Table 3:

    Table 3: Hardware to Software Port Mappings for Director Device Network Modules

    Network Module

    Port 0

    Port 1

    Port 2

    Port 3

    Module 0

    eth5

    eth4

    eth3

    eth2

    Module 1

    eth9

    eth8

    eth7

    eth6

    Figure 3 shows the specific ports on the QFX3600-I Interconnect devices that you must connect to the EX4200 switches. In general, connect the first management port in an Interconnect device to the first EX4200 switch, and the second management port to the second EX4200 switch.

    Figure 3: QFX3000-M QFabric System Control Plane—Interconnect Device to EX4200 Switch Connections

    QFX3000-M QFabric System Control Plane—Interconnect
Device to EX4200 Switch Connections

    In this specific example, for both Interconnect devices IC0 and IC1, connect management port C0 to EX4200 switches EX0 and EX1 and management port C1 to EX4200 switches EX0 and EX1. Connect the management port C0 cables to port 16 on EX4200 switches EX0 and EX1 (ge-0/0/16), and connect the management port C1 cables to port 17 on EX4200 switches EX0 and EX1 (ge-0/0/17). Table 4 shows the port mappings for the Node devices in this example.

    Table 4: Interconnect Device Port Mappings

    Interconnect Device

    EX4200 Switch EX0

    EX4200 Switch EX1

    IC0

    IC0, management port C0 to port 16
    (ge-0/0/16)

    IC0, management port C1 to port 16
    (ge-0/0/16)

    IC1

    IC1, management port C0 to port 17
    (ge-0/0/17)

    IC1, management port C1 to port 17
    (ge-0/0/17)

    Figure 4, Figure 5, and Figure 6 show the specific ports on the Node devices that you must connect to the EX4200 switches when using a copper-based control plane. In general, connect the first management port from a Node device to the first EX4200 switch, and the second management port to the second EX4200 switch.

    Figure 4: QFX3000-M QFabric System Control Plane—QFX3500 Node Device to EX4200 Switch Connections

    QFX3000-M QFabric System Control Plane—QFX3500
Node Device to EX4200 Switch Connections

    Figure 5: QFX3000-M QFabric System Control Plane—QFX3600 Node Device to EX4200 Switch Connections

    QFX3000-M QFabric System Control Plane—QFX3600
Node Device to EX4200 Switch Connections

    Figure 6: QFX3000-M QFabric System Control Plane—QFX5100 Node Device to EX4200 Switch Connections

    QFX3000-M QFabric System Control Plane—QFX5100
Node Device to EX4200 Switch Connections

    When implementing a fiber-based control plane, refer to Figure 7, Figure 8, and Figure 9 for the proper control plane connections.

    Figure 7: QFX3000-M QFabric System Fiber-Based Control Plane—QFX3500 Node Device to EX4200 Switch Connections

    QFX3000-M QFabric System Fiber-Based Control
Plane—QFX3500 Node Device to EX4200 Switch Connections

    Figure 8: QFX3000-M QFabric System Fiber-Based Control Plane—QFX3600 Node Device to EX4200 Switch Connections

    QFX3000-M QFabric System Fiber-Based Control
Plane—QFX3600 Node Device to EX4200 Switch Connections

    Figure 9: QFX3000-M QFabric System Fiber-Based Control Plane—QFX5100 Node Device to EX4200 Switch Connections

    QFX3000-M QFabric System Fiber-Based Control
Plane—QFX5100 Node Device to EX4200 Switch Connections

    In this specific example, for Node device Node0, connect management port C0 (also known as me5) to EX4200 switch EX0 port 0 (ge-0/0/0), and connect management port C1 (also known as me6) to the second EX4200 switch EX1 port 0 (ge-0/0/0).

    For the remaining seven Node devices, connect management port C0 to the ge-0/0/X port on EX4200 switch EX0 that matches the Node device number. Similarly, connect management port C1 to the port on the second EX4200 switch EX1 that matches the Node device number. For example, you would connect Node device Node5 to port 5 (ge-0/0/5). Table 5 shows the full set of port mappings for the Node devices in this example.

    Table 5: Node Device to EX4200 Switch Port Mappings

    Node Device

    EX4200 Switch EX0

    EX4200 Switch EX1

    Node0

    Node0, management port C0 to port 0
    (ge-0/0/0)

    Node0, management port C1 to port 0
    (ge-0/0/0)

    Node1

    Node1, management port C0 to port 1
    (ge-0/0/1)

    Node1, management port C1 to port 1
    (ge-0/0/1)

    Node2

    Node2, management port C0 to port 2
    (ge-0/0/2)

    Node2, management port C1 to port 2
    (ge-0/0/2)

    Node3

    Node3, management port C0 to port 3
    (ge-0/0/3)

    Node3, management port C1 to port 3
    (ge-0/0/3)

    Node4

    Node4, management port C0 to port 4
    (ge-0/0/4)

    Node4, management port C1 to port 4
    (ge-0/0/4)

    Node5

    Node5, management port C0 to port 5
    (ge-0/0/5)

    Node5, management port C1 to port 5
    (ge-0/0/5)

    Node6

    Node6, management port C0 to port 6
    (ge-0/0/6)

    Node6, management port C1 to port 6
    (ge-0/0/6)

    Node7

    Node7, management port C0 to port 7
    (ge-0/0/7)

    Node7, management port C1 to port 7
    (ge-0/0/7)

    Figure 10 shows the specific uplink ports on the first EX4200 switch that you must connect to the second EX4200 switch. These connections create a link aggregation group (LAG) that provides redundancy and resiliency for the EX4200 switch portion of the control plane. In general, connect each 1-Gigabit Ethernet uplink port from the first EX4200 switch to the corresponding 1-Gigabit Ethernet uplink port on the second EX4200 switch.

    Figure 10: QFX3000-M QFabric System Control Plane—Inter-EX4200 Switch LAG Connections

    QFX3000-M QFabric System Control Plane—Inter-EX4200
Switch LAG Connections

    In this specific example, for EX4200 switch EX0, connect uplink port 0 (ge-0/1/0) to EX4200 switch EX1 uplink port 0 (ge-0/1/0). Then connect uplink port 1 (ge-0/1/1) on EX4200 switch EX0 to uplink port 1 (ge-0/1/1) on EX4200 switch EX1.

    Table 6 shows the port mappings for the EX4200 switch LAG connections in this example.

    Table 6: EX4200 Switch LAG Port Mappings

    EX0 and EX1

    EX0

    EX1

    Uplink port 0

    ge-0/1/0 to ge-0/1/0

    ge-0/1/0 to ge-0/1/0

    Uplink port 1

    ge-0/1/1 to ge-0/1/1

    ge-0/1/1 to ge-0/1/1

    Configuration

    CLI Quick Configuration

    To configure the QFX3000-M QFabric system control plane EX4200 switches quickly, copy the following commands, paste them in a text file, remove any line breaks, change any details necessary to match your network, and then copy and paste the commands into the EX4200 switch CLI at the [edit] hierarchy level.

    set groups qfabric chassis aggregated-devices ethernet device-count 3set groups qfabric chassis alarm management-ethernet link-down ignoreset groups qfabric chassis lcd-menu fpc 0 menu-item maintenance-menu disableset groups qfabric protocols rstp interface ae2.0 mode point-to-pointset groups qfabric protocols rstp interface all edgeset groups qfabric protocols rstp interface all no-root-portset groups qfabric protocols rstp bpdu-block-on-edgeset groups qfabric protocols lldp interface allset groups qfabric ethernet-switching-options storm-control interface all bandwidth 10000set groups qfabric vlans qfabric vlan-id 100set groups qfabric vlans qfabric dot1q-tunnelingset groups qfabric-int interfaces <*> mtu 9216set groups qfabric-int interfaces <*> unit 0 family ethernet-switching port-mode accessset groups qfabric-int interfaces <*> unit 0 family ethernet-switching vlan members qfabricset groups qfabric-ae interfaces <*> aggregated-ether-options link-speed 1gset groups qfabric-ae interfaces <*> aggregated-ether-options lacp activeset apply-groups qfabricset chassis fpc 0 pic 1 sfpplus pic-mode 1gset interfaces interface-range Node_Device_Interfaces member "ge-0/0/[0-15]"set interfaces interface-range Node_Device_Interfaces description "QFabric Node Device"set interfaces interface-range Node_Device_Interfaces mtu 9216set interfaces interface-range Node_Device_Interfaces unit 0 family ethernet-switching port-mode accessset interfaces interface-range Node_Device_Interfaces unit 0 family ethernet-switching vlan members qfabricset interfaces interface-range Interconnect_Device_Interfaces member "ge-0/0/[16-17]"set interfaces interface-range Interconnect_Device_Interfaces description "QFabric Interconnect Device"set interfaces interface-range Interconnect_Device_Interfaces mtu 9216set interfaces interface-range Interconnect_Device_Interfaces unit 0 family ethernet-switching port-mode accessset interfaces interface-range Interconnect_Device_Interfaces unit 0 family ethernet-switching vlan members qfabricset interfaces interface-range Director_Device_DG0_LAG_Interfaces member "ge-0/0/[20-21]"set interfaces interface-range Director_Device_DG0_LAG_Interfaces description "QFabric Director Device - DG0"set interfaces interface-range Director_Device_DG0_LAG_Interfaces ether-options speed 1gset interfaces interface-range Director_Device_DG0_LAG_Interfaces ether-options 802.3ad ae0set interfaces interface-range Director_Device_DG1_LAG_Interfaces member "ge-0/0/[22-23]"set interfaces interface-range Director_Device_DG1_LAG_Interfaces description "QFabric Director Device - DG1"set interfaces interface-range Director_Device_DG1_LAG_Interfaces ether-options speed 1gset interfaces interface-range Director_Device_DG1_LAG_Interfaces ether-options 802.3ad ae1set interfaces interface-range Control_Plane_Inter_LAG_Interfaces member "ge-0/1/[0-1]"set interfaces interface-range Control_Plane_Inter_LAG_Interfaces description "QFabric Control Plane (Inter - Switch LAG)"set interfaces interface-range Control_Plane_Inter_LAG_Interfaces ether-options 802.3ad ae2set interfaces ae0 apply-groups qfabric-intset interfaces ae0 apply-groups qfabric-aeset interfaces ae0 description "QFabric Director Device - DG0"set interfaces ae1 apply-groups qfabric-intset interfaces ae1 apply-groups qfabric-aeset interfaces ae1 description "QFabric Director Device - DG1"set interfaces ae2 description "QFabric Control Plane (Inter-Switch LAG)"set interfaces ae2 mtu 9216set interfaces ae2 aggregated-ether-options link-speed 1gset interfaces ae2 aggregated-ether-options lacp activeset interfaces ae2 unit 0 family ethernet-switching vlan members qfabricset class-of-service classifiers ieee-802.1 onep_qfabric_classifier forwarding-class class_3 loss-priority low code-points 110set class-of-service classifiers ieee-802.1 onep_qfabric_classifier forwarding-class class_3 loss-priority low code-points 111set class-of-service classifiers ieee-802.1 onep_qfabric_classifier forwarding-class class_2 loss-priority low code-points 100set class-of-service classifiers ieee-802.1 onep_qfabric_classifier forwarding-class class_2 loss-priority high code-points 101set class-of-service classifiers ieee-802.1 onep_qfabric_classifier forwarding-class class_0 loss-priority low code-points 010set class-of-service classifiers ieee-802.1 onep_qfabric_classifier forwarding-class class_0 loss-priority high code-points 001set class-of-service classifiers inet-precedence IP_qfabric_classifier forwarding-class class_3 loss-priority low code-points 110set class-of-service classifiers inet-precedence IP_qfabric_classifier forwarding-class class_3 loss-priority low code-points 111set class-of-service classifiers inet-precedence IP_qfabric_classifier forwarding-class class_2 loss-priority low code-points 100set class-of-service classifiers inet-precedence IP_qfabric_classifier forwarding-class class_2 loss-priority high code-points 101set class-of-service classifiers inet-precedence IP_qfabric_classifier forwarding-class class_0 loss-priority low code-points 010set class-of-service classifiers inet-precedence IP_qfabric_classifier forwarding-class class_0 loss-priority high code-points 001set class-of-service forwarding-classes class class_3 queue-num 7set class-of-service forwarding-classes class class_2 queue-num 2set class-of-service forwarding-classes class class_0 queue-num 0set class-of-service interfaces ge-*/0/* scheduler-map cpe_network_smapset class-of-service interfaces ge-*/0/* unit 0 classifiers ieee-802.1 onep_qfabric_classifierset class-of-service interfaces ge-*/0/* unit 0 classifiers inet-precedence IP_qfabric_classifierset class-of-service interfaces ae* scheduler-map cpe_network_smapset class-of-service interfaces ae* unit 0 classifiers ieee-802.1 onep_qfabric_classifierset class-of-service interfaces ae* unit 0 classifiers inet-precedence IP_qfabric_classifierset class-of-service scheduler-maps cpe_network_smap forwarding-class class_3 scheduler scheduler_3set class-of-service scheduler-maps cpe_network_smap forwarding-class class_2 scheduler scheduler_2set class-of-service scheduler-maps cpe_network_smap forwarding-class class_0 scheduler scheduler_0set class-of-service schedulers scheduler_3 buffer-size percent 30set class-of-service schedulers scheduler_3 priority strict-highset class-of-service schedulers scheduler_2 transmit-rate percent 75set class-of-service schedulers scheduler_2 buffer-size percent 30set class-of-service schedulers scheduler_2 priority lowset class-of-service schedulers scheduler_0 transmit-rate percent 25set class-of-service schedulers scheduler_0 buffer-size percent 40set class-of-service schedulers scheduler_0 priority lowset system host-name qfabric-control-planeset system services sshset system services telnetset system services web-management httpset system syslog user * any emergencyset system syslog file messages any noticeset system syslog file messages authorization infoset system syslog file messages archive world-readableset system syslog file messages explicit-priorityset system syslog file interactive-commands interactive-commands anyset system syslog file secure authorization infoset system syslog file default-log-messages any anyset system syslog file default-log-messages structured-dataset system syslog file console any errorset system syslog time-format millisecondset interfaces me0 unit 0 family inet address 192.168.157.26/24set routing-options static route 0.0.0.0/0 next-hop 192.168.157.1

    Step-by-Step Procedure

    The following example requires that you navigate various levels in the configuration hierarchy. For instructions on how to do that, see Using the CLI Editor in Configuration Mode in the CLI User Guide.

    To configure a EX4200 switch for the QFX3000-M QFabric system control plane network:

    1. Create a configuration group called qfabric to define global QFabric system control plane properties. Set up the number of aggregated Ethernet devices, configure alarm and LCD management, activate loop prevention and storm control, specify a global VLAN (VLAN 100) and 802.1q tunneling, define options for aggregated Ethernet interfaces, and apply the qfabric group settings to the configuration.
      [edit]user@switch# set groups qfabric chassis aggregated-devices ethernet device-count 3user@switch# set groups qfabric chassis alarm management-ethernet link-down ignoreuser@switch# set groups qfabric chassis lcd-menu fpc 0 menu-item maintenance-menu disableuser@switch# set groups qfabric protocols rstp interface ae2.0 mode point-to-pointuser@switch# set groups qfabric protocols rstp interface all edgeuser@switch# set groups qfabric protocols rstp interface all no-root-portuser@switch# set groups qfabric protocols rstp bpdu-block-on-edgeuser@switch# set groups qfabric protocols lldp interface alluser@switch# set groups qfabric ethernet-switching-options storm-control interface all bandwidth 10000user@switch# set groups qfabric vlans qfabric vlan-id 100user@switch# set groups qfabric vlans qfabric dot1q-tunnelinguser@switch# set groups qfabric-int interfaces <*> mtu 9216user@switch# set groups qfabric-int interfaces <*> unit 0 family ethernet-switching port-mode accessuser@switch# set groups qfabric-int interfaces <*> unit 0 family ethernet-switching vlan members qfabricuser@switch# set groups qfabric-ae interfaces <*> aggregated-ether-options link-speed 1guser@switch# set groups qfabric-ae interfaces <*> aggregated-ether-options lacp activeuser@switch# set apply-groups qfabric
    2. Configure interfaces for the QFabric system control plane network. Enable the EX4200 switch SFP+ uplink module for 1-Gigabit Ethernet operation. Set the interface ranges where Node devices (0 through 15), Interconnect devices (16 and 17), and Director devices (20 through 23) connect to the control plane network through the EX4200 switches. Configure the inter-EX4200 switch LAG connections for the ae2 interface and apply the qfabric-int and qfabric-ae configuration groups to the aggregated Ethernet interfaces (ae0 and ae1) for the Director devices.
      [edit]user@switch# set chassis fpc 0 pic 1 sfpplus pic-mode 1guser@switch# set interfaces interface-range Node_Device_Interfaces member "ge-0/0/[0-15]"user@switch# set interfaces interface-range Node_Device_Interfaces description "QFabric Node Device"user@switch# set interfaces interface-range Node_Device_Interfaces mtu 9216user@switch# set interfaces interface-range Node_Device_Interfaces unit 0 family ethernet-switching port-mode accessuser@switch# set interfaces interface-range Node_Device_Interfaces unit 0 family ethernet-switching vlan members qfabricuser@switch# set interfaces interface-range Interconnect_Device_Interfaces member "ge-0/0/[16-17]"user@switch# set interfaces interface-range Interconnect_Device_Interfaces description "QFabric Interconnect Device"user@switch# set interfaces interface-range Interconnect_Device_Interfaces mtu 9216user@switch# set interfaces interface-range Interconnect_Device_Interfaces unit 0 family ethernet-switching port-mode accessuser@switch# set interfaces interface-range Interconnect_Device_Interfaces unit 0 family ethernet-switching vlan members qfabricuser@switch# set interfaces interface-range Director_Device_DG0_LAG_Interfaces member "ge-0/0/[20-21]"user@switch# set interfaces interface-range Director_Device_DG0_LAG_Interfaces description "QFabric Director Device - DG0"user@switch# set interfaces interface-range Director_Device_DG0_LAG_Interfaces ether-options speed 1guser@switch# set interfaces interface-range Director_Device_DG0_LAG_Interfaces ether-options 802.3ad ae0user@switch# set interfaces interface-range Director_Device_DG1_LAG_Interfaces member "ge-0/0/[22-23]"user@switch# set interfaces interface-range Director_Device_DG1_LAG_Interfaces description "QFabric Director Device - DG1"user@switch# set interfaces interface-range Director_Device_DG1_LAG_Interfaces ether-options speed 1guser@switch# set interfaces interface-range Director_Device_DG1_LAG_Interfaces ether-options 802.3ad ae1user@switch# set interfaces interface-range Control_Plane_Inter_LAG_Interfaces member "ge-0/1/[0-1]"user@switch# set interfaces interface-range Control_Plane_Inter_LAG_Interfaces description "QFabric Control Plane (Inter - Switch LAG)"user@switch# set interfaces interface-range Control_Plane_Inter_LAG_Interfaces ether-options 802.3ad ae2user@switch# set interfaces ae0 apply-groups qfabric-intuser@switch# set interfaces ae0 apply-groups qfabric-aeuser@switch# set interfaces ae0 description "QFabric Director Device - DG0"user@switch# set interfaces ae1 apply-groups qfabric-intuser@switch# set interfaces ae1 apply-groups qfabric-aeuser@switch# set interfaces ae1 description "QFabric Director Device - DG1"user@switch# set interfaces ae2 description "QFabric Control Plane (Inter-Switch LAG)"user@switch# set interfaces ae2 mtu 9216user@switch# set interfaces ae2 aggregated-ether-options link-speed 1guser@switch# set interfaces ae2 aggregated-ether-options lacp activeuser@switch# set interfaces ae2 unit 0 family ethernet-switching vlan members qfabric
    3. Enable class of service (CoS) for the QFabric system control plane network. Establish forwarding classes, priorities, scheduler maps, classifiers, and queues for three types of traffic: control traffic, interdevice traffic, and best-effort traffic.
      [edit]user@switch# set class-of-service classifiers ieee-802.1 onep_qfabric_classifier forwarding-class class_3 loss-priority low code-points 110user@switch# set class-of-service classifiers ieee-802.1 onep_qfabric_classifier forwarding-class class_3 loss-priority low code-points 111user@switch# set class-of-service classifiers ieee-802.1 onep_qfabric_classifier forwarding-class class_2 loss-priority low code-points 100user@switch# set class-of-service classifiers ieee-802.1 onep_qfabric_classifier forwarding-class class_2 loss-priority high code-points 101user@switch# set class-of-service classifiers ieee-802.1 onep_qfabric_classifier forwarding-class class_0 loss-priority low code-points 010user@switch# set class-of-service classifiers ieee-802.1 onep_qfabric_classifier forwarding-class class_0 loss-priority high code-points 001user@switch# set class-of-service classifiers inet-precedence IP_qfabric_classifier forwarding-class class_3 loss-priority low code-points 110user@switch# set class-of-service classifiers inet-precedence IP_qfabric_classifier forwarding-class class_3 loss-priority low code-points 111user@switch# set class-of-service classifiers inet-precedence IP_qfabric_classifier forwarding-class class_2 loss-priority low code-points 100user@switch# set class-of-service classifiers inet-precedence IP_qfabric_classifier forwarding-class class_2 loss-priority high code-points 101user@switch# set class-of-service classifiers inet-precedence IP_qfabric_classifier forwarding-class class_0 loss-priority low code-points 010user@switch# set class-of-service classifiers inet-precedence IP_qfabric_classifier forwarding-class class_0 loss-priority high code-points 001user@switch# set class-of-service forwarding-classes class class_3 queue-num 7user@switch# set class-of-service forwarding-classes class class_2 queue-num 2user@switch# set class-of-service forwarding-classes class class_0 queue-num 0user@switch# set class-of-service interfaces ge-*/0/* scheduler-map cpe_network_smapuser@switch# set class-of-service interfaces ge-*/0/* unit 0 classifiers ieee-802.1 onep_qfabric_classifieruser@switch# set class-of-service interfaces ge-*/0/* unit 0 classifiers inet-precedence IP_qfabric_classifieruser@switch# set class-of-service interfaces ae* scheduler-map cpe_network_smapuser@switch# set class-of-service interfaces ae* unit 0 classifiers ieee-802.1 onep_qfabric_classifieruser@switch# set class-of-service interfaces ae* unit 0 classifiers inet-precedence IP_qfabric_classifieruser@switch# set class-of-service scheduler-maps cpe_network_smap forwarding-class class_3 scheduler scheduler_3user@switch# set class-of-service scheduler-maps cpe_network_smap forwarding-class class_2 scheduler scheduler_2user@switch# set class-of-service scheduler-maps cpe_network_smap forwarding-class class_0 scheduler scheduler_0user@switch# set class-of-service schedulers scheduler_3 buffer-size percent 30user@switch# set class-of-service schedulers scheduler_3 priority strict-highuser@switch# set class-of-service schedulers scheduler_2 transmit-rate percent 75user@switch# set class-of-service schedulers scheduler_2 buffer-size percent 30user@switch# set class-of-service schedulers scheduler_2 priority lowuser@switch# set class-of-service schedulers scheduler_0 transmit-rate percent 25user@switch# set class-of-service schedulers scheduler_0 buffer-size percent 40user@switch# set class-of-service schedulers scheduler_0 priority low
    4. Configure settings to enable the EX4200 switches to interoperate with your management network. Set a hostname, system services (such as Telnet), system log thresholds, management interface parameters, default routes, and any additional preferences you might have.
      [edit]user@switch# set system host-name qfabric-control-planeuser@switch# set system services sshuser@switch# set system services telnetuser@switch# set system services web-management httpuser@switch# set system syslog user * any emergencyuser@switch# set system syslog file messages any noticeuser@switch# set system syslog file messages authorization infouser@switch# set system syslog file messages archive world-readableuser@switch# set system syslog file messages explicit-priorityuser@switch# set system syslog file interactive-commands interactive-commands anyuser@switch# set system syslog file secure authorization infouser@switch# set system syslog file default-log-messages any anyuser@switch# set system syslog file default-log-messages structured-datauser@switch# set system syslog file console any erroruser@switch# set system syslog time-format milliseconduser@switch# set interfaces me0 unit 0 family inet address 192.168.157.26/24user@switch# set routing-options static route 0.0.0.0/0 next-hop 192.168.157.1

    Results

    To view the configuration, issue the show command in configuration mode or the show configuration command in operational mode. If the output does not display the intended configuration, repeat the configuration instructions in this example to correct it.

    The following configuration is the standard configuration that applies universally to both EX4200 switches in your QFabric system control plane network.

    [edit]
    groups {qfabric {chassis {aggregated-devices {ethernet {device-count 3;}}alarm {management-ethernet {link-down ignore;}}lcd-menu {fpc 0 { maintenance-menu disable;}}}protocols {rstp {interface ae2.0 {mode point-to-point;}interface all {edge;no-root-port;}bpdu-block-on-edge;}lldp {interface all;}}ethernet-switching-options {storm-control {interface all {bandwidth 10000;}}}vlans {qfabric {vlan-id 100;dot1q-tunneling;}}}qfabric-int {interfaces {<*> {mtu 9216;unit 0 {family ethernet-switching {port-mode access;vlan {members qfabric;}}}}}}qfabric-ae {interfaces {<*> {aggregated-ether-options {link-speed 1g;lacp {active;}}}}}}
    apply-groups [qfabric];
    chassis {fpc 0 {pic 1 {sfpplus {pic-mode 10g;}}}}
    interfaces {interface-range Node_Device_Interfaces {member "ge-0/0/[0-15]";description "QFabric Node Device";mtu 9216;unit 0 {family ethernet-switching {port-mode access;vlan {members qfabric;}}}}interface-range Interconnect_Device_Interfaces {member "ge-0/0/[16-17]";description "QFabric Interconnect Device";mtu 9216;unit 0 {family ethernet-switching {port-mode access;vlan {members qfabric;}}}}interface-range Director_Device_DG0_LAG_Interfaces {member "ge-0/0/[20-21]";description "QFabric Director Device - DG0";ether-options {speed {1g;}802.3ad ae0;}}interface-range Director_Device_DG1_LAG_Interfaces {member "ge-0/0/[22-23]";description "QFabric Director Device - DG1";ether-options {speed {1g;}802.3ad ae1;}}interface-range Control_Plane_Inter_LAG_Interfaces {member "ge-0/1/[0-1]";description "QFabric Control Plane (Inter-Switch LAG)";ether-options {802.3ad ae2;}}ae0 {apply-groups [ qfabric-int qfabric-ae ];description "QFabric Director Device - DG0";}ae1 {apply-groups [ qfabric-int qfabric-ae ];description "QFabric Director Device - DG1";}ae2 {description "QFabric Control Plane (Inter-Switch LAG)";mtu 9216;aggregated-ether-options {link-speed 1g;lacp {active;}}unit 0 {family ethernet-switching {vlan {members qfabric;}}}}}
    class-of-service {classifiers {ieee-802.1 onep_qfabric_classifier {forwarding-class class_3 {loss-priority low code-points [ 110 111 ];}forwarding-class class_2 {loss-priority low code-points 100;loss-priority high code-points 101;}forwarding-class class_0 {loss-priority low code-points 010;loss-priority high code-points 001;}}inet-precedence IP_qfabric_classifier {forwarding-class class_3 {loss-priority low code-points [ 110 111 ];}forwarding-class class_2 {loss-priority low code-points 100;loss-priority high code-points 101;}forwarding-class class_0 {loss-priority low code-points 010;loss-priority high code-points 001;}}}forwarding-classes {class class_3 queue-num 7;class class_2 queue-num 2;class class_0 queue-num 0;}interfaces {ge-*/0/* {scheduler-map cpe_network_smap;unit 0 {classifiers {ieee-802.1 onep_qfabric_classifier;inet-precedence IP_qfabric_classifier;}}}ae* {scheduler-map cpe_network_smap;unit 0 {classifiers {ieee-802.1 onep_qfabric_classifier;inet-precedence IP_qfabric_classifier;}}}}scheduler-maps {cpe_network_smap {forwarding-class class_3 scheduler scheduler_3;forwarding-class class_2 scheduler scheduler_2;forwarding-class class_0 scheduler scheduler_0;}}schedulers {scheduler_3 {buffer-size percent 30;priority strict-high;}scheduler_2 {transmit-rate percent 75;buffer-size percent 30;priority low;}scheduler_0 {transmit-rate percent 25;buffer-size percent 40;priority low;}}}

    The following portion of the configuration applies to the specific requirements of your management network. Modify this section to meet the needs of your network.

    [edit]
    system {host-name qfabric-control-plane;services {ssh;telnet;web-management {http;}}syslog {user * {any emergency;}file messages {any notice;authorization info;archive world-readable;explicit-priority;}file interactive-commands {interactive-commands any;}file secure {authorization info;}file default-log-messages {any any;structured-data;}file console {any error;}time-format millisecond;}}
    interfaces {me0 {unit 0 {family inet {address 192.168.157.26/24;}}}}
    routing-options {static {route 0.0.0.0/0 next-hop 192.168.157.1;}}

    To verify the syntax of your configuration prior to committing it, enter commit check from configuration mode. If you are done configuring the device, enter commit from configuration mode.

    Verification

    Confirm that the EX4200 switch configuration is working properly.

    Verifying the QFX3000-M QFabric System Control Plane—EX4200 Switch EX0

    Purpose

    Verify that the control plane is properly connected on your first EX4200 switch.

    Action

    Connect to the Junos OS CLI of EX4200 switch EX0, either from your management network or from the console port of the switch. In operational mode, enter the show interfaces terse command.

    Sample Output

    user@ex0> show interfaces terse
    Interface               Admin Link   Proto    Local                 Remote
    ge-0/0/0                up    up  
    ge-0/0/0.0              up    up     eth-switch
    ge-0/0/1                up    up  
    ge-0/0/1.0              up    up     eth-switch
    ge-0/0/2                up    up  
    ge-0/0/2.0              up    up     eth-switch
    ge-0/0/3                up    up  
    ge-0/0/3.0              up    up     eth-switch
    ge-0/0/4                up    up  
    ge-0/0/4.0              up    up     eth-switch
    ge-0/0/5                up    up  
    ge-0/0/5.0              up    up     eth-switch
    ge-0/0/6                up    up  
    ge-0/0/6.0              up    up     eth-switch
    ge-0/0/7                up    up  
    ge-0/0/7.0              up    up     eth-switch
    ge-0/0/8                up    down  
    ge-0/0/8.0              up    down   eth-switch
    ge-0/0/9                up    down  
    ge-0/0/9.0              up    down   eth-switch
    ge-0/0/10               up    down  
    ge-0/0/10.0             up    down   eth-switch
    ge-0/0/11               up    down  
    ge-0/0/11.0             up    down   eth-switch
    ge-0/0/12               up    down  
    ge-0/0/12.0             up    down   eth-switch
    ge-0/0/13               up    down  
    ge-0/0/13.0             up    down   eth-switch
    ge-0/0/14               up    down  
    ge-0/0/14.0             up    down   eth-switch
    ge-0/0/15               up    down  
    ge-0/0/15.0             up    down   eth-switch
    ge-0/0/16               up    up  
    ge-0/0/16.0             up    up     eth-switch
    ge-0/0/17               up    up  
    ge-0/0/17.0             up    up     eth-switch
    ge-0/0/18               up    down  
    ge-0/0/18.0             up    down   eth-switch
    ge-0/0/19               up    down  
    ge-0/0/19.0             up    down   eth-switch
    ge-0/0/20               up    up  
    ge-0/0/20.0             up    up     aenet    --> ae0.0
    ge-0/0/21               up    up  
    ge-0/0/21.0             up    up     aenet    --> ae0.0
    ge-0/0/22               up    up  
    ge-0/0/22.0             up    up     aenet    --> ae1.0
    ge-0/0/23               up    up  
    ge-0/0/23.0             up    up     aenet    --> ae1.0
    ge-0/1/0                up    up  
    ge-0/1/0.0              up    up     aenet    --> ae2.0
    ge-0/1/1                up    up  
    ge-0/1/1.0              up    up     aenet    --> ae2.0
    vcp-0                   up    down
    vcp-0.32768             up    down
    vcp-1                   up    down
    vcp-1.32768             up    down
    ae0                     up    up  
    ae0.0                   up    up     eth-switch
    ae1                     up    up  
    ae1.0                   up    up     eth-switch
    ae2                     up    up  
    ae2.0                   up    up     eth-switch
    bme0                    up    up  
    bme0.32768              up    up     inet     128.0.0.1/2     
                                                  128.0.0.16/2    
                                                  128.0.0.32/2    
                                         tnp      0x10            
    dsc                     up    up  
    gre                     up    up  
    ipip                    up    up  
    lo0                     up    up  
    lo0.0                   up    up     inet     127.0.0.1           --> 0/0
    lsi                     up    up        
    me0                     up    up  
    me0.0                   up    up     inet     192.168.157.26/24
    mtun                    up    up  
    pimd                    up    up  
    pime                    up    up  
    tap                     up    up  
    vlan                    up    up  
    vme                     up    down

    Meaning

    In the output of the show interfaces terse command, if all interfaces that connect to the QFabric system devices are listed as up (such as ge-0/0/16 and ge-0/0/17 for the Interconnect devices; ge-0/0/20 through ge-0/0/23 for the Director devices; ge-0/0/0 through ge-0/0/7 for the Node devices; and ge-0/1/0 and ge-0/1/1 for the inter-EX4200 switch connections), the control plane is properly connected.

    Verifying the QFX3000-M QFabric System Control Plane—EX4200 Switch EX1

    Purpose

    Verify that the control plane is properly connected on your second EX4200 switch.

    Action

    Connect to the Junos OS CLI of EX4200 switch EX1, either from your management network or from the console port of the switch. In operational mode, enter the show interfaces terse command.

    Sample Output

    user@ex1> show interfaces terse
    Interface               Admin Link   Proto    Local                 Remote
    ge-0/0/0                up    up  
    ge-0/0/0.0              up    up     eth-switch
    ge-0/0/1                up    up  
    ge-0/0/1.0              up    up     eth-switch
    ge-0/0/2                up    up  
    ge-0/0/2.0              up    up     eth-switch
    ge-0/0/3                up    up  
    ge-0/0/3.0              up    up     eth-switch
    ge-0/0/4                up    up  
    ge-0/0/4.0              up    up     eth-switch
    ge-0/0/5                up    up  
    ge-0/0/5.0              up    up     eth-switch
    ge-0/0/6                up    up  
    ge-0/0/6.0              up    up     eth-switch
    ge-0/0/7                up    up  
    ge-0/0/7.0              up    up     eth-switch
    ge-0/0/8                up    down  
    ge-0/0/8.0              up    down   eth-switch
    ge-0/0/9                up    down  
    ge-0/0/9.0              up    down   eth-switch
    ge-0/0/10               up    down  
    ge-0/0/10.0             up    down   eth-switch
    ge-0/0/11               up    down  
    ge-0/0/11.0             up    down   eth-switch
    ge-0/0/12               up    down  
    ge-0/0/12.0             up    down   eth-switch
    ge-0/0/13               up    down  
    ge-0/0/13.0             up    down   eth-switch
    ge-0/0/14               up    down  
    ge-0/0/14.0             up    down   eth-switch
    ge-0/0/15               up    down  
    ge-0/0/15.0             up    down   eth-switch
    ge-0/0/16               up    up  
    ge-0/0/16.0             up    up     eth-switch
    ge-0/0/17               up    up  
    ge-0/0/17.0             up    up     eth-switch
    ge-0/0/18               up    down  
    ge-0/0/18.0             up    down   eth-switch
    ge-0/0/19               up    down  
    ge-0/0/19.0             up    down   eth-switch
    ge-0/0/20               up    up  
    ge-0/0/20.0             up    up     aenet    --> ae0.0
    ge-0/0/21               up    up  
    ge-0/0/21.0             up    up     aenet    --> ae0.0
    ge-0/0/22               up    up  
    ge-0/0/22.0             up    up     aenet    --> ae1.0
    ge-0/0/23               up    up  
    ge-0/0/23.0             up    up     aenet    --> ae1.0
    ge-0/1/0                up    up  
    ge-0/1/0.0              up    up     aenet    --> ae2.0
    ge-0/1/1                up    up  
    ge-0/1/1.0              up    up     aenet    --> ae2.0
    vcp-0                   up    down
    vcp-0.32768             up    down
    vcp-1                   up    down
    vcp-1.32768             up    down
    ae0                     up    down
    ae0.0                   up    down   eth-switch
    ae1                     up    down
    ae1.0                   up    down   eth-switch
    ae2                     up    up  
    ae2.0                   up    up     eth-switch
    bme0                    up    up  
    bme0.32768              up    up     inet     128.0.0.1/2     
                                                  128.0.0.16/2    
                                                  128.0.0.32/2    
                                         tnp      0x10            
    dsc                     up    up  
    gre                     up    up  
    ipip                    up    up  
    lo0                     up    up  
    lo0.0                   up    up     inet     127.0.0.1           --> 0/0
    lsi                     up    up        
    me0                     up    up  
    me0.0                   up    up     inet     192.168.157.26/24
    mtun                    up    up  
    pimd                    up    up  
    pime                    up    up  
    tap                     up    up  
    vlan                    up    up  
    vme                     up    down

    Meaning

    In the output of the show interfaces terse command, if all interfaces that connect to the QFabric system devices are listed as up (such as ge-0/0/16 and ge-0/0/17 for the Interconnect devices; ge-0/0/20 through ge-0/0/23 for the Director devices; ge-0/0/0 through ge-0/0/7 for the Node devices; and ge-0/1/0 and ge-0/1/1 for the inter-EX4200 switch connections), the control plane is properly connected.

    Modified: 2015-06-30