采用 Q-in-Q 的静态 VXLAN 隧道
对于小型 MC-LAG 网络，您可以使用静态 VXLAN 来降低网络中的控制平面复杂性。在静态 VXLAN 上配置 VTEP 非常简单。使用此示例可在数据中心之间配置具有 Q-in-Q 标记（VLAN 转换）的静态 VXLAN 隧道。在本示例中，我们重点介绍以下功能：
-
静态 VXLAN — 静态 VXLAN 通过创建第 2 层路径（隧道）连接不同数据中心的服务器。有关静态 VXLAN 的详细信息，请参阅 静态 VXLAN。
-
Q-in-Q 隧道 — Q-in-Q 隧道将不同的客户 VLAN （C-VLAN） 流量隔离并捆绑到单个服务提供商 VLAN 中。
有关 Q-in-Q 隧道的更多信息，请参阅 配置 Q-in-Q 隧道和 VLAN、Q-in-Q 隧道和 VLAN 转换。
-
MC-LAG — MC LAG 提供冗余和负载平衡。我们在两个对等设备之间配置 ICL 和 ICCP 连接以创建 MC-LAG。有关 MC-LAG 的详细信息，请参阅 了解多机箱链路聚合组
图 1 显示了主干-分叶数据中心 （POD） 的一部分。在 POD 中，TOR 设备（TOR1 和 TOR2）从以下服务器收集 VLAN，还管理 VLAN 转换（Q-in-Q 隧道）。聚合器从不同的 TOR 设备收集 VLAN，并充当 POD 的网关。我们使用静态 VXLAN 隧道作为两个 POD 之间的网关。我们在对等 TOR 设备和对等聚合器之间配置 MC-LAG。在我们的参考测试环境中，我们测试了包含 64 个 pod 的配置。对于此示例，我们将介绍如何在单个 Pod 中配置聚合器和 TOR 设备。
图 1：采用 Q-in-Q 和静态 VXLAN 隧道的数据中心 POD
此示例配置在现有 IP 交换矩阵之上。请参阅 IP 交换矩阵底层网络设计和实施。
配置聚合器
以下部分介绍如何配置聚合器。
- 将 聚合器配置为支持聚合以太网和 MC-LAG。
-
设置聚合以太网接口的最大数量。
-
为 LAG 设置服务标识符 （SID）。
-
配置环路地址。
-
配置管理端口。我们将管理接口用作“始终开启”的端口，以支持 ICCP 对等方之间的保持活动通信。
AGG1 和 AGG2
set chassis aggregated-devices ethernet device-count 64 set switch-options service-id 1
AGG1
set interfaces lo0 unit 0 family inet address 192.168.1.4/32 primary set interfaces em0 unit 0 family inet address 10.48.49.69/221
AGG2
set interfaces lo0 unit 0 family inet address 192.168.1.5/32 primary set nterfaces em0 unit 0 family inet address 10.48.49.117/22
-
- 分配聚合以太网接口。
-
ae0 和 ae1 在聚合器之间形成 ICL 和 ICCP 链路。
-
AE3 将聚合器连接到主干设备。
-
ae4 将聚合器连接到 TOR 设备。
AGG1 和 AGG2
set interfaces xe-0/0/49:0 ether-options 802.3ad ae0 set interfaces xe-0/0/49:1 ether-options 802.3ad ae1 set interfaces xe-0/0/48:0 ether-options 802.3ad ae3 set interfaces xe-0/0/50:0 ether-options 802.3ad ae4 set interfaces xe-0/0/50:1 ether-options 802.3ad ae4
-
- 在聚合以太网接口上启用 LACP。使用快速周期间隔启用 LACP，每秒发送一个数据包。
AGG1 和 AGG2
set interfaces ae4 aggregated-ether-options minimum-links 1 set interfaces ae4 aggregated-ether-options lacp active set interfaces ae4 aggregated-ether-options lacp periodic fast set interfaces ae4 aggregated-ether-options lacp system-id 00:00:00:00:04:01 set interfaces ae4 aggregated-ether-options lacp admin-key 4
- 配置从聚合器到 TOR 设备的 MC-LAG 接口，并将其设置为主动-主动模式。为每个对等方设置唯一的机箱 ID。
AGG1 和 AGG2
set interfaces ae4 aggregated-ether-options mc-ae mc-ae-id 4 set interfaces ae4 aggregated-ether-options mc-ae redundancy-group 1 set interfaces ae4 aggregated-ether-options mc-ae mode active-active set interfaces ae4 aggregated-ether-options mc-ae status-control active set interfaces ae4 aggregated-ether-options mc-ae init-delay-time 300
AGG1
set interfaces ae4 aggregated-ether-options mc-ae chassis-id 0
AGG2
set interfaces ae4 aggregated-ether-options mc-ae chassis-id 1
- 跨 ICL 配置 ICCP 对等方（AGG1 和 AGG2）。配置交换激活消息时
backup-liveness-detection，我们使用管理链路的 IP 地址。AGG1
set interfaces ae0 description "ICCP link Connected to MCLAG peer" set interfaces ae0 unit 0 family inet address 172.16.10.1/30 set multi-chassis multi-chassis-protection 172.16.10.2 interface ae1 set protocols iccp local-ip-addr 172.16.10.1 set protocols iccp peer 172.16.10.2 session-establishment-hold-time 600 set protocols iccp peer 172.16.10.2 redundancy-group-id-list 1 set protocols iccp peer 172.16.10.2 backup-liveness-detection backup-peer-ip 10.48.49.69 set protocols iccp peer 172.16.10.2 liveness-detection minimum-interval 1000
AGG2
set interfaces ae0 description "ICCP link Connected to MCLAG peer" set interfaces ae0 unit 0 family inet address 172.16.10.2/30 set multi-chassis multi-chassis-protection 172.16.10.1 interface ae1 set protocols iccp local-ip-addr 172.16.10.2 set protocols iccp peer 172.16.10.1 session-establishment-hold-time 600 set protocols iccp peer 172.16.10.1 redundancy-group-id-list 1 set protocols iccp peer 172.16.10.1 backup-liveness-detection backup-peer-ip 10.48.49.117 set protocols iccp peer 172.16.10.1 liveness-detection minimum-interval 1000
- 配置 接口以支持 VLAN。
AGG1 和 AGG2
set vlans SP-VLAN-3000 vlan-id 3000 set vlans SP-VLAN-3000 interface ae1.3000 set vlans SP-VLAN-3000 interface ae4.3000 set vlans SP-VLAN-3001 vlan-id 3001 set vlans SP-VLAN-3001 interface ae1.3001 set vlans SP-VLAN-3001 interface ae4.3001 set interfaces ae1 unit 3000 encapsulation vlan-bridge set interfaces ae1 unit 3000 vlan-id 3000 set interfaces ae1 unit 3001 encapsulation vlan-bridge set interfaces ae1 unit 3001 vlan-id 3001 set interfaces ae4 description "Connected to TOR1 TOR2" set interfaces ae4 flexible-vlan-tagging set interfaces ae4 encapsulation flexible-ethernet-services set interfaces ae4 unit 3000 encapsulation vlan-bridge set interfaces ae4 unit 3000 vlan-id 3000 set interfaces ae4 unit 3001 encapsulation vlan-bridge set interfaces ae4 unit 3001 vlan-id 3001
- 配置主干设备的接口。
AGG1
set interfaces ae3 description "Connected to Spine-1" set interfaces ae3 unit 0 family inet address 192.168.100.2/24
AGG2
set interfaces ae3 description "Connected to Spine-1" set interfaces ae3 unit 0 family inet address 192.168.200.2/24
- 通过配置本地和远程 VTEP 接口来启用静态 VXLAN。
AGG1 和 AGG2
set switch-options vtep-source-interface lo0.0 set switch-options remote-vtep-list 192.168.1.6
- 将 VLAN 映射到远程 VTEP。
AGG1 和 AGG2
set vlans SP-VLAN-3000 vxlan vni 103000 set vlans SP-VLAN-3000 vxlan ingress-node-replication set vlans SP-VLAN-3000 vxlan static-remote-vtep-list 192.168.1.6 set vlans SP-VLAN-3001 vxlan vni 103001 set vlans SP-VLAN-3001 vxlan ingress-node-replication set vlans SP-VLAN-3001 vxlan static-remote-vtep-list 192.168.1.6
配置 TOR 设备
以下部分介绍如何配置 TOR 设备。
- 将 TOR 设备配置为支持聚合以太网和 MC-LAG。
-
设置聚合以太网接口的最大数量。
-
设置 LAG 的 SID。
-
配置环路地址。
-
配置管理端口。我们将管理接口用作“始终开启”的端口，以支持 ICCP 对等方之间的保持活动通信。
TOR1 和 TOR2
set chassis aggregated-devices ethernet device-count 64 set switch-options service-id 1
TOR1
set interfaces lo0 unit 0 family inet address 192.168.1.8/32 primary set interfaces em0 unit 0 family inet address 10.48.49.197/22
TOR2
set interfaces lo0 unit 0 family inet address 192.168.1.9/32 primary set interfaces em0 unit 0 family inet address 10.48.49.196/22
-
- 分配聚合以太网接口。
-
ae0 和 ae1 在 TOR 设备之间形成 ICL 和 ICCP 链路。
-
ae4 将 TOR 设备连接到聚合器。
-
AE7 和 AE8 将 TOR 设备连接到服务器。
TOR1 和 TOR2
set interfaces xe-0/0/0:0 ether-options 802.3ad ae0 set interfaces xe-0/0/0:1 ether-options 802.3ad ae0 set interfaces xe-0/0/0:2 ether-options 802.3ad ae1 set interfaces xe-0/0/0:3 ether-options 802.3ad ae1 set interfaces xe-0/0/1:0 ether-options 802.3ad ae4 set interfaces xe-0/0/1:1 ether-options 802.3ad ae4 set interfaces xe-0/0/2:1 ether-options 802.3ad ae7 set interfaces xe-0/0/2:3 ether-options 802.3ad ae8
-
- 在聚合以太网接口上启用 LACP。以快速周期性的时间间隔启用 LACP，以每秒发送一个数据包。
TOR1 和 TOR2
set interfaces ae4 aggregated-ether-options minimum-links 1 set interfaces ae4 aggregated-ether-options lacp active set interfaces ae4 aggregated-ether-options lacp periodic fast set interfaces ae4 aggregated-ether-options lacp system-id 00:00:00:00:04:02 set interfaces ae4 aggregated-ether-options lacp admin-key 4 set interfaces ae7 aggregated-ether-options minimum-links 1 set interfaces ae7 aggregated-ether-options lacp active set interfaces ae7 aggregated-ether-options lacp periodic fast set interfaces ae7 aggregated-ether-options lacp system-id 00:00:00:00:07:01 set interfaces ae7 aggregated-ether-options lacp admin-key 7 set interfaces ae8 aggregated-ether-options minimum-links 1 set interfaces ae8 aggregated-ether-options lacp active set interfaces ae8 aggregated-ether-options lacp periodic fast set interfaces ae8 aggregated-ether-options lacp system-id 00:00:00:00:08:01 set interfaces ae8 aggregated-ether-options lacp admin-key 8
- 配置 接口以支持 VLAN 和 Q-in-Q 转换。
TOR1 和 TOR2
set interfaces ae1 description "ICL link Connected to MCLAG peer" set interfaces ae1 flexible-vlan-tagging set interfaces ae1 encapsulation encapsulation extended-vlan-bridge set interfaces ae1 unit 3000 vlan-id 3000 set interfaces ae4 description "Connected to AGG1 AGG2" set interfaces ae4 flexible-vlan-tagging set interfaces ae4 encapsulation extended-vlan-bridge set interfaces ae4 unit 3000 vlan-id-list 3000-3001 set interfaces ae4 unit 3000 input-vlan-map push set interfaces ae4 unit 3000 output-vlan-map pop set interfaces ae7 description "Connected to Server1" set interfaces ae7 flexible-vlan-tagging set interfaces ae7 encapsulation extended-vlan-bridge set interfaces ae7 unit 3000 vlan-id-list 3000-3001 set interfaces ae7 unit 3000 input-vlan-map push set interfaces ae7 unit 3000 output-vlan-map pop set interfaces ae8 description "Connected to Server2" set interfaces ae8 flexible-vlan-tagging set interfaces ae8 encapsulation extended-vlan-bridge set interfaces ae8 unit 3000 vlan-id-list 3000-3001 set interfaces ae8 unit 3000 input-vlan-map push set interfaces ae8 unit 3000 output-vlan-map pop set vlans SP-VLAN-3000 interface ae1.3000 set vlans SP-VLAN-3000 interface ae4.3000 set vlans SP-VLAN-3000 interface ae7.3000 set vlans SP-VLAN-3000 interface ae8.3000 set vlans SP-VLAN-3000 service-id 3000 set vlans SP-VLAN-3001 interface ae4.3001 set vlans SP-VLAN-3001 interface ae7.3001 set vlans SP-VLAN-3001 interface ae8.3001 set vlans SP-VLAN-3001 service-id 3001
- 配置从 TOR 设备到聚合器和服务器的 MC-LAG 接口，并将其设置为主动-主动模式。为每个对等方设置唯一的机箱 ID。
TOR1 和 TOR2
set interfaces ae4 aggregated-ether-options mc-ae mc-ae-id 4 set interfaces ae4 aggregated-ether-options mc-ae redundancy-group 1 set interfaces ae4 aggregated-ether-options mc-ae mode active-active set interfaces ae4 aggregated-ether-options mc-ae status-control active set interfaces ae4 aggregated-ether-options mc-ae init-delay-time 300 set interfaces ae7 aggregated-ether-options mc-ae mc-ae-id 7 set interfaces ae7 aggregated-ether-options mc-ae redundancy-group 1 set interfaces ae7 aggregated-ether-options mc-ae mode active-active set interfaces ae7 aggregated-ether-options mc-ae status-control active set interfaces ae7 aggregated-ether-options mc-ae init-delay-time 300 set interfaces ae8 aggregated-ether-options mc-ae mc-ae-id 8 set interfaces ae8 aggregated-ether-options mc-ae redundancy-group 1 set interfaces ae8 aggregated-ether-options mc-ae mode active-active set interfaces ae8 aggregated-ether-options mc-ae status-control active set interfaces ae8 aggregated-ether-options mc-ae init-delay-time 300
TOR1
set interfaces ae4 aggregated-ether-options mc-ae chassis-id 0 set interfaces ae7 aggregated-ether-options mc-ae chassis-id 0 set interfaces ae8 aggregated-ether-options mc-ae chassis-id 0
TOR2
set interfaces ae4 aggregated-ether-options mc-ae chassis-id 1 set interfaces ae7 aggregated-ether-options mc-ae chassis-id 1 set interfaces ae8 aggregated-ether-options mc-ae chassis-id 1
- 在 两个 TOR 对等方（TOR1 和 TOR2）之间的 ICL 上配置 ICCP。配置交换激活消息时
backup-liveness-detection，我们使用管理链路的 IP 地址。TOR1
set interfaces ae0 description "ICCP link Connected to MCLAG peer" set interfaces ae0 unit 0 family inet address 172.16.20.1/30 set multi-chassis multi-chassis-protection 172.16.20.2 interface ae1 set protocols iccp local-ip-addr 172.16.2.1 set protocols iccp peer 172.16.20.2 session-establishment-hold-time 600 set protocols iccp peer 172.16.20.2 redundancy-group-id-list 1 set protocols iccp peer 172.16.20.2 backup-liveness-detection backup-peer-ip 10.48.49.197 set protocols iccp peer 172.16.20.2 liveness-detection minimum-interval 1000
TOR2
set interfaces ae0 description "ICCP link Connected to MCLAG peer" set interfaces ae0 unit 0 family inet address 172.16.20.2/30 set multi-chassis multi-chassis-protection 172.16.20.1 interface ae1 set protocols iccp local-ip-addr 172.16.20.2 set protocols iccp peer 172.16.20.1 session-establishment-hold-time 600 set protocols iccp peer 172.16.20.1 redundancy-group-id-list 1 set protocols iccp peer 17.16.20.1 backup-liveness-detection backup-peer-ip 10.48.49.196 set protocols iccp peer 172.16.20.1 liveness-detection minimum-interval 1000
在聚合器上使用 Q-in-Q 验证静态 VXLAN 隧道
本节介绍如何在聚合器通过静态 VXLAN 隧道管理 VLAN 时验证 VXLAN 的操作。所有命令均在 AGG1 上发出。
- 显示 VLAN 信息。
user@agg1> show vlans default-switch SP-VLAN-3000 NA ae1.3000* ae4.3000* vtep.32769* - 验证多机箱聚合以太网链路的运行状态。
user@agg1> show interfaces mc-ae Member Link : ae4 Current State Machine's State: mcae active state Configuration Error Status : No Error Local Status : active Local State : up Peer Status : active Peer State : up Logical Interface : ae4.3000 Topology Type : bridge Local State : up Peer State : up Peer Ip/MCP/State : 172.16.10.2 ae1.3000 up - 验证 AGG1 和 AGG2 之间的 MC-LAG 状态。
user@agg1> show iccp Redundancy Group Information for peer 172.16.10.2 TCP Connection : Established Liveliness Detection : Up Backup liveness peer status: Up Redundancy Group ID Status 1 Up Client Application: lacpd Redundancy Group IDs Joined: 1 Client Application: l2ald_iccpd_client Redundancy Group IDs Joined: 1 - 验证聚合以太网接口上的 LACP 状态。
user@agg1> show lacp interfaces Aggregated interface: ae4 LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity xe-0/0/50:0 Actor No No Yes Yes Yes Yes Fast Active xe-0/0/50:0 Partner No No Yes Yes Yes Yes Fast Active xe-0/0/50:1 Actor No No Yes Yes Yes Yes Fast Active xe-0/0/50:1 Partner No No Yes Yes Yes Yes Fast Active LACP protocol: Receive State Transmit State Mux State xe-0/0/50:0 Current Fast periodic Collecting distributing xe-0/0/50:1 Current Fast periodic Collecting distributing - 验证静态 VXLAN 的本地和远程 VTEP 接口是否正常运行。
user@agg1> show ethernet-switching vxlan-tunnel-end-point source Logical System Name Id SVTEP-IP IFL L3-Idx SVTEP-Mode ELP-SVTEP-IP <default> 0 192.168.1.4 lo0.0 0 L2-RTT Bridge Domain VNID Translation-VNID MC-Group-IP Interface default-switch SP-VLAN-3000 103000 0.0.0.0 vtep.32768 user@agg1> show ethernet-switching vxlan-tunnel-end-point remote Logical System Name Id SVTEP-IP IFL L3-Idx SVTEP-Mode ELP-SVTEP-IP <default> 0 192.168.1.4 lo0.0 0 RVTEP-IP L2-RTT IFL-Idx Interface NH-Id RVTEP-Mode ELP-IP Flags 192.168.1.5 default-switch 825 vtep.32769 1784 RNVE VNID MC-Group-IP 103000 0.0.0.0
验证 TOR 设备上的 Q-in-Q 隧道
本节介绍如何验证其中一个 TOR 设备上的 VLAN 操作。所有命令均在 TOR1 上发出
- 显示 VLAN 信息。
user@tor1> show vlans default-switch SP-VLAN-3000 NA ae1.3000* ae4.3000* ae7.3000* ae8.3000* - 验证多机箱聚合以太网链路的运行状态。
user@tor1> show interfaces mc-ae Member Link : ae4 Current State Machine's State: mcae active state Configuration Error Status : No Error Local Status : active Local State : up Peer Status : active Peer State : up Logical Interface : ae4.3000 Topology Type : bridge Local State : up Peer State : up Peer Ip/MCP/State : 172.16.20.2 ae1.3000 up Member Link : ae7 Current State Machine's State: mcae active state Configuration Error Status : No Error Local Status : active Peer State : up Logical Interface : ae7.3000 Topology Type : bridge Local State : up Peer State : up Peer Ip/MCP/State : 172.16.20.2 ae1.3000 up Member Link : ae8 Current State Machine's State: mcae active state Configuration Error Status : No Error Local Status : active Local State : up Peer Status : active Peer State : up Logical Interface : ae8.3000 Topology Type : bridge Local State : up Peer State : up Peer Ip/MCP/State : 172.16.20.1 ae1.3000 up - 验证聚合以太网接口上的 LACP 状态。
user@tor1> show lacp interfaces Aggregated interface: ae4 LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity xe-0/0/1:0 Actor No No Yes Yes Yes Yes Fast Active xe-0/0/1:0 Partner No No Yes Yes Yes Yes Fast Active xe-0/0/1:1 Actor No No Yes Yes Yes Yes Fast Active xe-0/0/1:1 Partner No No Yes Yes Yes Yes Fast Active LACP protocol: Receive State Transmit State Mux State xe-0/0/1:0 Current Fast periodic Collecting distributing xe-0/0/1:1 Current Fast periodic Collecting distributing Aggregated interface: ae7 LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity xe-0/0/2:1 Actor No No Yes Yes Yes Yes Fast Active xe-0/0/2:1 Partner No No Yes Yes Yes Yes Fast Active LACP protocol: Receive State Transmit State Mux State xe-0/0/2:1 Current Fast periodic Collecting distributing Aggregated interface: ae8 LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity xe-0/0/2:3 Actor No No Yes Yes Yes Yes Fast Active xe-0/0/2:3 Partner No No Yes Yes Yes Yes Fast Active LACP protocol: Receive State Transmit State Mux State xe-0/0/2:3 Current Fast periodic Collecting distributing - 验证 TOR1 和 TOR2 之间的 MC-LAG 状态。
user@tor1> show iccp Redundancy Group Information for peer 172.16.20.2 TCP Connection : Established Liveliness Detection : Up Backup liveness peer status: Up Redundancy Group ID Status 1 Up Client Application: l2ald_iccpd_client Redundancy Group IDs Joined: 1 Client Application: lacpd Redundancy Group IDs Joined: 1