使用 Q-in-Q 的静态 VXLAN 隧道
对于小型 MC-LAG 网络,您可以使用静态 VXLAN 来降低网络中控制平面的复杂性。在静态 VXLAN 上配置 VTEP 非常简单。使用此示例可在数据中心之间配置带有 Q-in-Q 标记(VLAN 转换)的静态 VXLAN 隧道。在此示例中,我们重点介绍以下功能:
-
静态 VXLAN — 静态 VXLAN 通过创建第 2 层路径(隧道)连接不同数据中心的服务器。有关静态 VXLAN 的详细信息,请参阅静态 VXLAN。
-
Q-in-Q 隧道 — Q-in-Q 隧道将不同的客户 VLAN (C-VLAN) 流量隔离并捆绑到单个服务提供商 VLAN 中。
有关 Q-in-Q 隧道的更多信息,请参阅 配置 Q-in-Q 隧道和 VLAN Q-in-Q 隧道和 VLAN 转换。
-
MC-LAG — MC LAG 提供冗余和负载平衡。我们在两个对等设备之间配置 ICL 和 ICCP 连接以创建 MC-LAG。有关 MC-LAG 的详细信息,请参阅 了解多机箱链路聚合组
图 1 显示了脊叶式数据中心 (POD) 的一部分。在 POD 中,TOR 设备(TOR1 和 TOR2)从下面的服务器收集 VLAN,并管理 VLAN 转换(Q-in-Q 隧道)。聚合器从不同的 TOR 设备收集 VLAN,并充当 POD 的网关。我们使用静态 VXLAN 隧道作为两个 POD 之间的网关。我们在对等 TOR 设备和对等聚合器之间配置 MC-LAG。在我们的参考测试环境中,我们测试了具有 64 个 Pod 的配置。在此示例中,我们将介绍如何在单个 Pod 中配置聚合器和 TOR 设备。

此示例配置在现有 IP 交换矩阵之上。请参阅 IP 交换矩阵底层网络设计和实施。
配置聚合器
以下部分介绍如何配置聚合器。
- 将聚合器配置为支持聚合以太网和 MC-LAG。
-
设置聚合以太网接口的最大数量。
-
设置 LAG 的服务标识符 (SID)。
-
配置环路地址。
-
配置管理端口。我们将管理接口用作“始终运行”端口,以支持 ICCP 对等方之间的激活通信。
AGG1 和 AGG2
set chassis aggregated-devices ethernet device-count 64 set switch-options service-id 1
AGG1型
set interfaces lo0 unit 0 family inet address 192.168.1.4/32 primary set interfaces em0 unit 0 family inet address 10.48.49.69/221
AGG2型
set interfaces lo0 unit 0 family inet address 192.168.1.5/32 primary set nterfaces em0 unit 0 family inet address 10.48.49.117/22
-
- 分配聚合以太网接口。
-
ae0 和 ae1 构成聚合器之间的 ICL 和 ICCP 链路。
-
AE3 将聚合器连接到主干设备。
-
ae4 将聚合器连接到 TOR 设备。
AGG1 和 AGG2
set interfaces xe-0/0/49:0 ether-options 802.3ad ae0 set interfaces xe-0/0/49:1 ether-options 802.3ad ae1 set interfaces xe-0/0/48:0 ether-options 802.3ad ae3 set interfaces xe-0/0/50:0 ether-options 802.3ad ae4 set interfaces xe-0/0/50:1 ether-options 802.3ad ae4
-
- 在聚合以太网接口上启用 LACP。启用 LACP,以快速的周期间隔每秒发送一个数据包。
AGG1 和 AGG2
set interfaces ae4 aggregated-ether-options minimum-links 1 set interfaces ae4 aggregated-ether-options lacp active set interfaces ae4 aggregated-ether-options lacp periodic fast set interfaces ae4 aggregated-ether-options lacp system-id 00:00:00:00:04:01 set interfaces ae4 aggregated-ether-options lacp admin-key 4
- 配置从聚合器到 TOR 设备的 MC-LAG 接口,并将其设置为 A/A 模式。为每个对等方设置唯一的机箱 ID。
AGG1 和 AGG2
set interfaces ae4 aggregated-ether-options mc-ae mc-ae-id 4 set interfaces ae4 aggregated-ether-options mc-ae redundancy-group 1 set interfaces ae4 aggregated-ether-options mc-ae mode active-active set interfaces ae4 aggregated-ether-options mc-ae status-control active set interfaces ae4 aggregated-ether-options mc-ae init-delay-time 300
AGG1型
set interfaces ae4 aggregated-ether-options mc-ae chassis-id 0
AGG2型
set interfaces ae4 aggregated-ether-options mc-ae chassis-id 1
- 在 ICL 中配置 ICCP 对等方(AGG1 和 AGG2)。在配置
backup-liveness-detection
交换激活消息时,我们将使用管理链路的 IP 地址。AGG1型
set interfaces ae0 description "ICCP link Connected to MCLAG peer" set interfaces ae0 unit 0 family inet address 172.16.10.1/30 set multi-chassis multi-chassis-protection 172.16.10.2 interface ae1 set protocols iccp local-ip-addr 172.16.10.1 set protocols iccp peer 172.16.10.2 session-establishment-hold-time 600 set protocols iccp peer 172.16.10.2 redundancy-group-id-list 1 set protocols iccp peer 172.16.10.2 backup-liveness-detection backup-peer-ip 10.48.49.69 set protocols iccp peer 172.16.10.2 liveness-detection minimum-interval 1000
AGG2型
set interfaces ae0 description "ICCP link Connected to MCLAG peer" set interfaces ae0 unit 0 family inet address 172.16.10.2/30 set multi-chassis multi-chassis-protection 172.16.10.1 interface ae1 set protocols iccp local-ip-addr 172.16.10.2 set protocols iccp peer 172.16.10.1 session-establishment-hold-time 600 set protocols iccp peer 172.16.10.1 redundancy-group-id-list 1 set protocols iccp peer 172.16.10.1 backup-liveness-detection backup-peer-ip 10.48.49.117 set protocols iccp peer 172.16.10.1 liveness-detection minimum-interval 1000
- 将接口配置为支持 VLAN。
AGG1 和 AGG2
set vlans SP-VLAN-3000 vlan-id 3000 set vlans SP-VLAN-3000 interface ae1.3000 set vlans SP-VLAN-3000 interface ae4.3000 set vlans SP-VLAN-3001 vlan-id 3001 set vlans SP-VLAN-3001 interface ae1.3001 set vlans SP-VLAN-3001 interface ae4.3001 set interfaces ae1 unit 3000 encapsulation vlan-bridge set interfaces ae1 unit 3000 vlan-id 3000 set interfaces ae1 unit 3001 encapsulation vlan-bridge set interfaces ae1 unit 3001 vlan-id 3001 set interfaces ae4 description "Connected to TOR1 TOR2" set interfaces ae4 flexible-vlan-tagging set interfaces ae4 encapsulation flexible-ethernet-services set interfaces ae4 unit 3000 encapsulation vlan-bridge set interfaces ae4 unit 3000 vlan-id 3000 set interfaces ae4 unit 3001 encapsulation vlan-bridge set interfaces ae4 unit 3001 vlan-id 3001
- 配置主干设备的接口。
AGG1型
set interfaces ae3 description "Connected to Spine-1" set interfaces ae3 unit 0 family inet address 192.168.100.2/24
AGG2型
set interfaces ae3 description "Connected to Spine-1" set interfaces ae3 unit 0 family inet address 192.168.200.2/24
- 通过配置本地和远程 VTEP 接口来启用静态 VXLAN。
AGG1 和 AGG2
set switch-options vtep-source-interface lo0.0 set switch-options remote-vtep-list 192.168.1.6
- 将 VLAN 映射到远程 VTEP。
AGG1 和 AGG2
set vlans SP-VLAN-3000 vxlan vni 103000 set vlans SP-VLAN-3000 vxlan ingress-node-replication set vlans SP-VLAN-3000 vxlan static-remote-vtep-list 192.168.1.6 set vlans SP-VLAN-3001 vxlan vni 103001 set vlans SP-VLAN-3001 vxlan ingress-node-replication set vlans SP-VLAN-3001 vxlan static-remote-vtep-list 192.168.1.6
配置 TOR 设备
以下部分介绍如何配置 TOR 设备。
- 将 TOR 设备配置为支持聚合以太网和 MC-LAG。
-
设置聚合以太网接口的最大数量。
-
设置 LAG 的 SID。
-
配置环路地址。
-
配置管理端口。我们将管理接口用作“始终运行”端口,以支持 ICCP 对等方之间的激活通信。
TOR1 和 TOR2
set chassis aggregated-devices ethernet device-count 64 set switch-options service-id 1
TOR1
set interfaces lo0 unit 0 family inet address 192.168.1.8/32 primary set interfaces em0 unit 0 family inet address 10.48.49.197/22
TOR2
set interfaces lo0 unit 0 family inet address 192.168.1.9/32 primary set interfaces em0 unit 0 family inet address 10.48.49.196/22
-
- 分配聚合以太网接口。
-
ae0 和 ae1 构成 TOR 设备之间的 ICL 和 ICCP 链路。
-
ae4 将 TOR 设备连接到聚合器。
-
ae7 和 ae8 将 TOR 设备连接到服务器。
TOR1 和 TOR2
set interfaces xe-0/0/0:0 ether-options 802.3ad ae0 set interfaces xe-0/0/0:1 ether-options 802.3ad ae0 set interfaces xe-0/0/0:2 ether-options 802.3ad ae1 set interfaces xe-0/0/0:3 ether-options 802.3ad ae1 set interfaces xe-0/0/1:0 ether-options 802.3ad ae4 set interfaces xe-0/0/1:1 ether-options 802.3ad ae4 set interfaces xe-0/0/2:1 ether-options 802.3ad ae7 set interfaces xe-0/0/2:3 ether-options 802.3ad ae8
-
- 在聚合以太网接口上启用 LACP。以快速的周期间隔启用 LACP,每秒发送一个数据包。
TOR1 和 TOR2
set interfaces ae4 aggregated-ether-options minimum-links 1 set interfaces ae4 aggregated-ether-options lacp active set interfaces ae4 aggregated-ether-options lacp periodic fast set interfaces ae4 aggregated-ether-options lacp system-id 00:00:00:00:04:02 set interfaces ae4 aggregated-ether-options lacp admin-key 4 set interfaces ae7 aggregated-ether-options minimum-links 1 set interfaces ae7 aggregated-ether-options lacp active set interfaces ae7 aggregated-ether-options lacp periodic fast set interfaces ae7 aggregated-ether-options lacp system-id 00:00:00:00:07:01 set interfaces ae7 aggregated-ether-options lacp admin-key 7 set interfaces ae8 aggregated-ether-options minimum-links 1 set interfaces ae8 aggregated-ether-options lacp active set interfaces ae8 aggregated-ether-options lacp periodic fast set interfaces ae8 aggregated-ether-options lacp system-id 00:00:00:00:08:01 set interfaces ae8 aggregated-ether-options lacp admin-key 8
- 将接口配置为支持 VLAN 和 Q-in-Q 转换。
注意:
配置 Q-in-Q 映射时,设备会选择 VLAN ID 范围内的最低值作为外部标记。例如,当 VLAN-id 列表中的范围为 3000-3001 时,我们的设备使用 VLAN 3000 作为外部标记。当设备收到 VLAN 在 3000 到 3001 范围内的传出数据包时,设备将推送 VLAN ID 为 3000 的外部标记。相反,设备会去除其外部标记中 VLAN ID 为 3000 的传入数据包的外部标记。
TOR1 和 TOR2
set interfaces ae1 description "ICL link Connected to MCLAG peer" set interfaces ae1 flexible-vlan-tagging set interfaces ae1 encapsulation encapsulation extended-vlan-bridge set interfaces ae1 unit 3000 vlan-id 3000 set interfaces ae4 description "Connected to AGG1 AGG2" set interfaces ae4 flexible-vlan-tagging set interfaces ae4 encapsulation extended-vlan-bridge set interfaces ae4 unit 3000 vlan-id-list 3000-3001 set interfaces ae4 unit 3000 input-vlan-map push set interfaces ae4 unit 3000 output-vlan-map pop set interfaces ae7 description "Connected to Server1" set interfaces ae7 flexible-vlan-tagging set interfaces ae7 encapsulation extended-vlan-bridge set interfaces ae7 unit 3000 vlan-id-list 3000-3001 set interfaces ae7 unit 3000 input-vlan-map push set interfaces ae7 unit 3000 output-vlan-map pop set interfaces ae8 description "Connected to Server2" set interfaces ae8 flexible-vlan-tagging set interfaces ae8 encapsulation extended-vlan-bridge set interfaces ae8 unit 3000 vlan-id-list 3000-3001 set interfaces ae8 unit 3000 input-vlan-map push set interfaces ae8 unit 3000 output-vlan-map pop set vlans SP-VLAN-3000 interface ae1.3000 set vlans SP-VLAN-3000 interface ae4.3000 set vlans SP-VLAN-3000 interface ae7.3000 set vlans SP-VLAN-3000 interface ae8.3000 set vlans SP-VLAN-3000 service-id 3000 set vlans SP-VLAN-3001 interface ae4.3001 set vlans SP-VLAN-3001 interface ae7.3001 set vlans SP-VLAN-3001 interface ae8.3001 set vlans SP-VLAN-3001 service-id 3001
- 配置从 TOR 设备到聚合器和服务器的 MC-LAG 接口,并将其设置为 A/A 模式。为每个对等方设置唯一的机箱 ID。
TOR1 和 TOR2
set interfaces ae4 aggregated-ether-options mc-ae mc-ae-id 4 set interfaces ae4 aggregated-ether-options mc-ae redundancy-group 1 set interfaces ae4 aggregated-ether-options mc-ae mode active-active set interfaces ae4 aggregated-ether-options mc-ae status-control active set interfaces ae4 aggregated-ether-options mc-ae init-delay-time 300 set interfaces ae7 aggregated-ether-options mc-ae mc-ae-id 7 set interfaces ae7 aggregated-ether-options mc-ae redundancy-group 1 set interfaces ae7 aggregated-ether-options mc-ae mode active-active set interfaces ae7 aggregated-ether-options mc-ae status-control active set interfaces ae7 aggregated-ether-options mc-ae init-delay-time 300 set interfaces ae8 aggregated-ether-options mc-ae mc-ae-id 8 set interfaces ae8 aggregated-ether-options mc-ae redundancy-group 1 set interfaces ae8 aggregated-ether-options mc-ae mode active-active set interfaces ae8 aggregated-ether-options mc-ae status-control active set interfaces ae8 aggregated-ether-options mc-ae init-delay-time 300
TOR1
set interfaces ae4 aggregated-ether-options mc-ae chassis-id 0 set interfaces ae7 aggregated-ether-options mc-ae chassis-id 0 set interfaces ae8 aggregated-ether-options mc-ae chassis-id 0
TOR2
set interfaces ae4 aggregated-ether-options mc-ae chassis-id 1 set interfaces ae7 aggregated-ether-options mc-ae chassis-id 1 set interfaces ae8 aggregated-ether-options mc-ae chassis-id 1
- 在两个 TOR 对等方(TOR1 和 TOR2)之间的 ICL 上配置 ICCP。在配置
backup-liveness-detection
交换激活消息时,我们将使用管理链路的 IP 地址。TOR1
set interfaces ae0 description "ICCP link Connected to MCLAG peer" set interfaces ae0 unit 0 family inet address 172.16.20.1/30 set multi-chassis multi-chassis-protection 172.16.20.2 interface ae1 set protocols iccp local-ip-addr 172.16.2.1 set protocols iccp peer 172.16.20.2 session-establishment-hold-time 600 set protocols iccp peer 172.16.20.2 redundancy-group-id-list 1 set protocols iccp peer 172.16.20.2 backup-liveness-detection backup-peer-ip 10.48.49.197 set protocols iccp peer 172.16.20.2 liveness-detection minimum-interval 1000
TOR2
set interfaces ae0 description "ICCP link Connected to MCLAG peer" set interfaces ae0 unit 0 family inet address 172.16.20.2/30 set multi-chassis multi-chassis-protection 172.16.20.1 interface ae1 set protocols iccp local-ip-addr 172.16.20.2 set protocols iccp peer 172.16.20.1 session-establishment-hold-time 600 set protocols iccp peer 172.16.20.1 redundancy-group-id-list 1 set protocols iccp peer 17.16.20.1 backup-liveness-detection backup-peer-ip 10.48.49.196 set protocols iccp peer 172.16.20.1 liveness-detection minimum-interval 1000
在聚合器上使用 Q-in-Q 验证静态 VXLAN 隧道
本节介绍如何验证聚合器在通过静态 VXLAN 隧道管理 VLAN 时的操作。所有命令都在 AGG1 上发出。
- 显示 VLAN 信息。
user@agg1> show vlans default-switch SP-VLAN-3000 NA ae1.3000* ae4.3000* vtep.32769*
- 验证多机箱聚合以太网链路的运行状态。
user@agg1> show interfaces mc-ae Member Link : ae4 Current State Machine's State: mcae active state Configuration Error Status : No Error Local Status : active Local State : up Peer Status : active Peer State : up Logical Interface : ae4.3000 Topology Type : bridge Local State : up Peer State : up Peer Ip/MCP/State : 172.16.10.2 ae1.3000 up
- 验证 AGG1 和 AGG2 之间的 MC-LAG 状态。
user@agg1> show iccp Redundancy Group Information for peer 172.16.10.2 TCP Connection : Established Liveliness Detection : Up Backup liveness peer status: Up Redundancy Group ID Status 1 Up Client Application: lacpd Redundancy Group IDs Joined: 1 Client Application: l2ald_iccpd_client Redundancy Group IDs Joined: 1
- 验证聚合以太网接口上的 LACP 状态。
user@agg1> show lacp interfaces Aggregated interface: ae4 LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity xe-0/0/50:0 Actor No No Yes Yes Yes Yes Fast Active xe-0/0/50:0 Partner No No Yes Yes Yes Yes Fast Active xe-0/0/50:1 Actor No No Yes Yes Yes Yes Fast Active xe-0/0/50:1 Partner No No Yes Yes Yes Yes Fast Active LACP protocol: Receive State Transmit State Mux State xe-0/0/50:0 Current Fast periodic Collecting distributing xe-0/0/50:1 Current Fast periodic Collecting distributing
- 验证静态 VXLAN 的本地和远程 VTEP 接口是否正常运行。
user@agg1> show ethernet-switching vxlan-tunnel-end-point source Logical System Name Id SVTEP-IP IFL L3-Idx SVTEP-Mode ELP-SVTEP-IP <default> 0 192.168.1.4 lo0.0 0 L2-RTT Bridge Domain VNID Translation-VNID MC-Group-IP Interface default-switch SP-VLAN-3000 103000 0.0.0.0 vtep.32768 user@agg1> show ethernet-switching vxlan-tunnel-end-point remote Logical System Name Id SVTEP-IP IFL L3-Idx SVTEP-Mode ELP-SVTEP-IP <default> 0 192.168.1.4 lo0.0 0 RVTEP-IP L2-RTT IFL-Idx Interface NH-Id RVTEP-Mode ELP-IP Flags 192.168.1.5 default-switch 825 vtep.32769 1784 RNVE VNID MC-Group-IP 103000 0.0.0.0
验证 TOR 设备上的 Q-in-Q 隧道
本节介绍如何验证其中一个 TOR 设备上 VLAN 的操作。所有命令均在 TOR1 上发出
- 显示 VLAN 信息。
user@tor1> show vlans default-switch SP-VLAN-3000 NA ae1.3000* ae4.3000* ae7.3000* ae8.3000*
- 验证多机箱聚合以太网链路的运行状态。
user@tor1> show interfaces mc-ae Member Link : ae4 Current State Machine's State: mcae active state Configuration Error Status : No Error Local Status : active Local State : up Peer Status : active Peer State : up Logical Interface : ae4.3000 Topology Type : bridge Local State : up Peer State : up Peer Ip/MCP/State : 172.16.20.2 ae1.3000 up Member Link : ae7 Current State Machine's State: mcae active state Configuration Error Status : No Error Local Status : active Peer State : up Logical Interface : ae7.3000 Topology Type : bridge Local State : up Peer State : up Peer Ip/MCP/State : 172.16.20.2 ae1.3000 up Member Link : ae8 Current State Machine's State: mcae active state Configuration Error Status : No Error Local Status : active Local State : up Peer Status : active Peer State : up Logical Interface : ae8.3000 Topology Type : bridge Local State : up Peer State : up Peer Ip/MCP/State : 172.16.20.1 ae1.3000 up
- 验证聚合以太网接口上的 LACP 状态。
user@tor1> show lacp interfaces Aggregated interface: ae4 LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity xe-0/0/1:0 Actor No No Yes Yes Yes Yes Fast Active xe-0/0/1:0 Partner No No Yes Yes Yes Yes Fast Active xe-0/0/1:1 Actor No No Yes Yes Yes Yes Fast Active xe-0/0/1:1 Partner No No Yes Yes Yes Yes Fast Active LACP protocol: Receive State Transmit State Mux State xe-0/0/1:0 Current Fast periodic Collecting distributing xe-0/0/1:1 Current Fast periodic Collecting distributing Aggregated interface: ae7 LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity xe-0/0/2:1 Actor No No Yes Yes Yes Yes Fast Active xe-0/0/2:1 Partner No No Yes Yes Yes Yes Fast Active LACP protocol: Receive State Transmit State Mux State xe-0/0/2:1 Current Fast periodic Collecting distributing Aggregated interface: ae8 LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity xe-0/0/2:3 Actor No No Yes Yes Yes Yes Fast Active xe-0/0/2:3 Partner No No Yes Yes Yes Yes Fast Active LACP protocol: Receive State Transmit State Mux State xe-0/0/2:3 Current Fast periodic Collecting distributing
- 验证 TOR1 和 TOR2 之间的 MC-LAG 状态。
user@tor1> show iccp Redundancy Group Information for peer 172.16.20.2 TCP Connection : Established Liveliness Detection : Up Backup liveness peer status: Up Redundancy Group ID Status 1 Up Client Application: l2ald_iccpd_client Redundancy Group IDs Joined: 1 Client Application: lacpd Redundancy Group IDs Joined: 1