패브릭 OAM에 의한 오류 처리
패브릭 운영, 관리, 유지 관리(OAM)는 패브릭 경로의 장애를 감지하는 데 도움이 됩니다. 패브릭 OAM은 PFE에 대한 새 패브릭 경로가 생성될 때마다 패브릭 플레인에서 트래픽을 전송하기 전에 패브릭 연결을 검증합니다. 장애가 감지되면 소프트웨어는 장애를 보고하고 해당 PFE에 해당 패브릭 플레인을 사용하지 않습니다. 이 기능은 사용 가능한 각 패브릭 플레인을 통해 초당 매우 낮은 패킷(PPS) 자체 목적지 OAM 트래픽을 전송하고 엔드포인트에서 트래픽 손실을 감지하는 방식으로 작동합니다(패브릭 자체 핑 확인).
- Junos OS Evolved 릴리스 20.4R1에서는 패브릭 OAM 기능이 기본적으로 활성화되어 있습니다. CLI 명령을
set chassis fabric oam detection-disable사용하여 기능을 비활성화할 수 있습니다. - Junos OS 진화한 릴리스 20.4R2 및 21.1R1에서는 패브릭 OAM 기능이 기본적으로 비활성화되어 있습니다.
- Junos OS Evolved 릴리스 22.1R1에서는 런타임 패브릭 OAM 기능이 기본적으로 활성화되어 있습니다. CLI 명령을
edit chassis fabric oam runtime-disable사용하여 기능을 비활성화할 수 있습니다. 런타임 패브릭 OAM 기능은 PTX10004, PTX10008 및 PTX10016 라우터에서 지원됩니다.
패브릭 OAM 검사는 부팅 시 수행됩니다. 실패한 경로는 비활성화됩니다. 시스템은 복구 조치를 수행하지 않습니다. 그러나 SIB를 다시 시작하여 영향을 받는 패브릭 플레인을 복구할 수 있습니다. 복구 단계는 장애의 특성에 따라 다릅니다.
패브릭 플레인은 PFE와 패브릭 ASIC 간의 독립적인 양방향 경로를 나타냅니다. 런타임 패브릭 OAM은 패브릭 연결을 주기적으로 검사하고 시스템 런타임 동안 패브릭 플레인의 장애를 감지하고 보고하는 데 도움이 됩니다. 런타임 패브릭 OAM은 각 PFE의 패브릭 도달 가능성을 감지합니다.
단일 또는 여러 FPC에서 동일한 패브릭 플레인이 실패하면 다음 명령을 사용하여 실패한 플레인이 포함된 SIB를 다시 시작합니다.
user@host> request chassis sib slot slot-number offline
user@host> request chassis sib slot slot-number online
여러 FPC에서 랜덤 패브릭 플레인이 실패하면 결함을 특정 FPC 또는 SIB로 격리할 수 없습니다. 그러나 영향을 받는 플레인을 포함하는 SIB를 순차적으로 다시 시작하여 플레인을 복구할 수 있습니다.
패브릭 OAM 기능에서 감지된 각 오류에 대해 운영자가 필요한 정보에 빠르고 효율적으로 액세스하는 데 도움이 되는 syslog가 생성됩니다.
Junos 릴리스 버전에 따른 관련 기능에 대한 syslog 메시지 세부 정보를 보려면 Syslog 탐색기를 참조하십시오. 패브릭 OAM의 패브릭 링크 결함과 관련된 로그, syslog 및 기타 진단 메시지 목록을 보려면 패브릭 OAM Syslog 메시지를 참조하십시오.
다음은 오류 및 syslog 메시지의 예입니다.
Oct 29 23:02:46 router-dvi resiliencyd[12921]: Error: /fpc/0/fabspoked-pfe/0/cm/0/pfe/0/fabric_link_foam_fault (0x410009), scope: board, category: internal, severity: major, module: fab-pfe@0, type: fabric link foam fault
다음 syslog 메시지는 패브릭 OAM 관련 오류가 해결되었음을 나타냅니다.
Oct 29 23:25:14 router-dvi resiliencyd[12921]: Performing action clear-cmalarm for error /fpc/0/fabspoked-pfe/0/cm/0/pfe/0/fabric_link_foam_fault (0x410009) in module: fab-pfe@0 with scope: board category: internal level: major
또한 CLI 명령을 show system errors active detail show system alarms 사용하고 패브릭 OAM 관련 오류를 볼 수 있습니다.
user@router> show system alarms
20 alarms currently active
Alarm time Class Description
2020-08-20 10:32:02 UTC Major FPC 0 Ideeprom read failure
2020-08-20 10:58:07 UTC Major FPC 0 Self_FOAM fault detected
[...Output truncated...]
user@router> show system alarms
14 alarms currently active
Alarm time Class Description
2022-02-15 23:45:28 PST Minor FPC 1 Volt Sensor Fail
2022-02-16 00:02:03 PST Major FPC 1 Self_Fabric OAM Runtime fault detected
2022-02-15 23:43:04 PST Minor FPC 1 Secure boot disabled or not enforced
2022-02-15 23:55:50 PST Minor FPC 3 Secure boot disabled or not enforced
[...Output truncated...]
다음 출력은 단일 패브릭 플레인 실패(패킷 포워딩 엔진 0) 및 모든 패브릭 플레인 실패(패킷 포워딩 엔진 1) 모두에 대한 세부 정보를 보여줍니다.
user@router> show system errors active detail
System Active Errors Detail Information
FPC 0
----------------------------------------------------------------
Error Name : fabric_down_condition_on_pfe
Identifier : /fpc/0/fabricHub/0/cm/0/fabrichub/1/fabric_down_condition_on_pfe
Description : fabric_down_condition_on_pfe
State : enabled
Scope : pfe
Category : functional
Level : major
Threshold : 1
Error limit : 0
Occur count : 3
Clear count : 2
Last occurred(ms ago) : 103158
System Active Errors Detail Information
FPC 0
----------------------------------------------------------------
Error Name : fabric_link_foam_fault
Identifier : /fpc/0/fabspoked-pfe/0/cm/0/pfe/0/fabric_link_foam_fault
Description : fabric link foam fault
State : enabled
Scope : board
Category : internal
Level : major
Threshold : 1
Error limit : 100
Occur count : 2
Clear count : 0
Last occurred(ms ago) : 113277
System Active Errors Detail Information
FPC 0
----------------------------------------------------------------
Error Name : fabric_link_foam_fault
Identifier : /fpc/0/fabspoked-pfe/0/cm/0/pfe/1/fabric_link_foam_fault
Description : fabric link foam fault
State : enabled
Scope : board
Category : internal
Level : major
Threshold : 1
Error limit : 100
Occur count : 12
Clear count : 0
Last occurred(ms ago) : 103267
System Active Errors Detail Information
RE 0
----------------------------------------------------------------
Error Name : fpga_min_supported_fw_ver_mismatch
Identifier : /re/0/hwdre/0/cm/0/fpga_fw_events/UBAM FPGA/fpga_min_supported_fw_ver_mismatch
Description : firmware_version_lower_than_minimum_expected
State : enabled
Scope : board
Category : functional
Level : minor
Threshold : 10
Error limit : 1
Occur count : 1
Clear count : 0
Last occurred(ms ago) : 68886367
FPC 1
----------------------------------------------------------------
Error Name : fabric_link_self_fabric_oam_runtime_fault
Identifier : /fpc/1/fabspoked-pfe/0/cm/0/pfe/0/fabric_link_self_fabric_oam_runtime_fault
Description : fabric link self fabric oam runtime fault
State : enabled
Scope : board
Category : internal
Level : major
Threshold : 1
Error limit : 36
Occur count : 1
Clear count : 0
Last occurred(ms ago) : 2022-02-16 00:02:03 PST (448108 ms ago) System Active Errors Detail Information
CLI 명령을 show chassis fabric fpcs 사용하여 각 패브릭 플레인의 패브릭 OAM 셀프 핑 상태를 볼 수 있습니다.
user@router> show chassis fabric fpcs
Fabric management FPC state:
FPC #0
PFE #0
SIB0_Asic0_Fcore0 (plane 0) Plane Disabled, Links ok Fabric OAM failed
SIB0_Asic0_Fcore0 (plane 1) Plane Enabled, Links ok Fabric OAM success
SIB0_Asic0_Fcore0 (plane 2) Plane Enabled, Links ok Fabric OAM success
SIB0_Asic0_Fcore0 (plane 3) Plane Enabled, Links ok Fabric OAM success
SIB0_Asic0_Fcore0 (plane 4) Plane Enabled, Links ok Fabric OAM success
SIB0_Asic0_Fcore0 (plane 5) Plane Enabled, Links ok Fabric OAM success
SIB1_Asic0_Fcore0 (plane 6) Plane Enabled, Links ok Fabric OAM success
SIB1_Asic0_Fcore0 (plane 7) Plane Enabled, Links ok Fabric OAM success
SIB1_Asic0_Fcore0 (plane 8) Plane Enabled, Links ok Fabric OAM success
SIB1_Asic0_Fcore0 (plane 9) Plane Enabled, Links ok Fabric OAM success
SIB1_Asic0_Fcore0 (plane 10) Plane Enabled, Links ok Fabric OAM success
SIB1_Asic0_Fcore0 (plane 11) Plane Enabled, Links ok Fabric OAM success
PFE #1
SIB0_Asic0_Fcore0 (plane 0) Plane Enabled, Links ok Fabric OAM success
SIB0_Asic0_Fcore0 (plane 1) Plane Enabled, Links ok Fabric OAM success
user@router> show chassis fabric fpcs
Fabric management FPC state:
FPC #1
PFE #0
SIB0_Asic0_Fcore0 (plane 0) Plane Enabled, Links ok Fabric OAM Runtime success
SIB0_Asic0_Fcore0 (plane 1) Plane Disabled, Links ok Fabric OAM Runtime failed
SIB0_Asic1_Fcore0 (plane 2) Plane Enabled, Links ok Fabric OAM Runtime success
SIB0_Asic1_Fcore0 (plane 3) Plane Enabled, Links ok Fabric OAM Runtime success
SIB0_Asic2_Fcore0 (plane 4) Plane Enabled, Links ok Fabric OAM Runtime success
SIB0_Asic2_Fcore0 (plane 5) Plane Enabled, Links ok Fabric OAM Runtime success
SIB1_Asic0_Fcore0 (plane 6) Plane Enabled, Links ok Fabric OAM Runtime success
SIB1_Asic0_Fcore0 (plane 7) Plane Enabled, Links ok Fabric OAM Runtime success
SIB1_Asic1_Fcore0 (plane 8) Plane Enabled, Links ok Fabric OAM Runtime success
SIB1_Asic1_Fcore0 (plane 9) Plane Enabled, Links ok Fabric OAM Runtime success
SIB1_Asic2_Fcore0 (plane 10) Plane Enabled, Links ok Fabric OAM Runtime success
SIB1_Asic2_Fcore0 (plane 11) Plane Enabled, Links ok Fabric OAM Runtime success
SIB2_Asic0_Fcore0 (plane 12) Plane Enabled, Links ok Fabric OAM Runtime success
SIB2_Asic0_Fcore0 (plane 13) Plane Enabled, Links ok Fabric OAM Runtime success
SIB2_Asic1_Fcore0 (plane 14) Plane Enabled, Links ok Fabric OAM Runtime success
SIB2_Asic1_Fcore0 (plane 15) Plane Enabled, Links ok Fabric OAM Runtime success
이 명령은 show chassis fabric fpcs 패브릭 OAM 기능이 비활성화될 때 다음 출력을 표시합니다.
user@router> show chassis fabric fpcs
Fabric management FPC state:
FPC #0
PFE #0
SIB0_Asic0_Fcore0 (plane 0) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 1) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 2) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 3) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 4) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 5) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 6) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 7) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 8) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 9) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 10) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 11) Plane Enabled, Links ok
PFE #1
SIB0_Asic0_Fcore0 (plane 0) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 1) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 2) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 3) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 4) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 5) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 6) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 7) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 8) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 9) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 10) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 11) Plane Enabled, Links ok
PFE #2
SIB0_Asic0_Fcore0 (plane 0) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 1) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 2) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 3) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 4) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 5) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 6) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 7) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 8) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 9) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 10) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 11) Plane Enabled, Links ok
PFE #3