패브릭 OAM에 의한 오류 처리
패브릭 OAM(Operation, Administration, Maintenance)은 패브릭 경로의 실패를 감지하는 데 도움이 됩니다. 패브릭 OAM은 PFE에 대한 새로운 패브릭 경로가 제공될 때마다 패브릭 플레인에서 트래픽을 전송하기 전에 패브릭 연결을 검증합니다. 장애가 감지되면 소프트웨어는 결함을 보고하고 해당 PFE에 해당 패브릭 플레인을 사용하지 않습니다. 이 기능은 사용 가능한 각 패브릭 플레인에 대해 매우 낮은 PPS(초당 패킷 수) 자체 목적지 OAM 트래픽을 전송하고 엔드포인트에서 트래픽 손실을 감지하는 방식으로 작동합니다(패브릭 셀프 핑 확인).
- 진화한 Junos OS 릴리스 20.4R1에서는 패브릭 OAM 기능이 기본적으로 활성화되어 있습니다. CLI 명령을 사용하여 기능을 비활성화할 수 있습니다
set chassis fabric oam detection-disable. - Junos OS 진화한 릴리스 20.4R2 및 21.1R1에서 패브릭 OAM 기능은 기본적으로 비활성화되어 있습니다.
- 진화한 Junos OS 릴리스 22.1R1에서는 런타임 패브릭 OAM 기능이 기본적으로 활성화되어 있습니다. CLI 명령을 사용하여 기능을 비활성화할 수 있습니다
edit chassis fabric oam runtime-disable. 런타임 패브릭 OAM 기능은 PTX10004, PTX10008 및 PTX10016 라우터에서 지원됩니다.
패브릭 OAM 검사는 부팅 시 수행됩니다. 장애가 발생한 경로는 비활성화됩니다. 시스템은 복구 작업을 수행하지 않습니다. 그러나 SIB를 다시 시작하여 영향을 받는 패브릭 플레인의 복구를 시도할 수 있습니다. 복구 단계는 오류의 특성에 따라 달라집니다.
패브릭 평면은 PFE와 패브릭 ASIC 간의 독립적인 양방향 경로를 나타냅니다. 런타임 패브릭 OAM은 패브릭 연결을 주기적으로 검사하고 시스템 런타임 동안 패브릭 플레인의 장애를 감지하고 보고하는 데 도움이 됩니다. 런타임 패브릭 OAM은 각 PFE의 패브릭 도달 가능성을 감지합니다.
동일한 패브릭 플레인이 단일 또는 여러 FPC에서 실패하면, 다음 명령을 사용하여 실패한 플레인을 포함하는 SIB를 다시 시작합니다.
user@host> request chassis sib slot slot-number offline
user@host> request chassis sib slot slot-number online
무작위 패브릭 플레인이 여러 FPC에서 실패하면 결함을 특정 FPC 또는 SIB로 격리할 수 없습니다. 그러나 영향을 받는 플레인을 포함하는 SIB를 순차적으로 다시 시작하여 플레인 복구를 시도할 수 있습니다.
패브릭 OAM 기능에서 감지된 각 오류에 대해 syslog가 생성되어 운영자가 필요한 정보에 빠르고 효율적으로 액세스할 수 있도록 지원합니다.
Junos 릴리스 버전에 따른 관련 기능에 대한 syslog 메시지 세부 정보를 보려면 Syslog 탐색기를 참조하십시오. 패브릭 OAM의 패브릭 링크 결함과 관련된 로그, syslog 및 기타 진단 메시지 목록을 보려면 패브릭 OAM syslog 메시지를 참조하십시오.
다음은 오류 및 syslog 메시지의 예입니다.
Oct 29 23:02:46 router-dvi resiliencyd[12921]: Error: /fpc/0/fabspoked-pfe/0/cm/0/pfe/0/fabric_link_foam_fault (0x410009), scope: board, category: internal, severity: major, module: fab-pfe@0, type: fabric link foam fault
다음 syslog 메시지는 패브릭 OAM 관련 오류가 해결되었음을 나타냅니다.
Oct 29 23:25:14 router-dvi resiliencyd[12921]: Performing action clear-cmalarm for error /fpc/0/fabspoked-pfe/0/cm/0/pfe/0/fabric_link_foam_fault (0x410009) in module: fab-pfe@0 with scope: board category: internal level: major
또한 CLI 명령 show system errors active detail 및 show system alarms 을 사용하여 패브릭 OAM 관련 오류를 확인할 수 있습니다.
user@router> show system alarms
20 alarms currently active
Alarm time Class Description
2020-08-20 10:32:02 UTC Major FPC 0 Ideeprom read failure
2020-08-20 10:58:07 UTC Major FPC 0 Self_FOAM fault detected
[...Output truncated...]
user@router> show system alarms
14 alarms currently active
Alarm time Class Description
2022-02-15 23:45:28 PST Minor FPC 1 Volt Sensor Fail
2022-02-16 00:02:03 PST Major FPC 1 Self_Fabric OAM Runtime fault detected
2022-02-15 23:43:04 PST Minor FPC 1 Secure boot disabled or not enforced
2022-02-15 23:55:50 PST Minor FPC 3 Secure boot disabled or not enforced
[...Output truncated...]
다음 출력은 단일 패브릭 플레인 실패(패킷 포워딩 엔진 0)와 모든 패브릭 플레인 실패(패킷 포워딩 엔진 1)에 대한 세부 정보를 보여줍니다.
user@router> show system errors active detail
System Active Errors Detail Information
FPC 0
----------------------------------------------------------------
Error Name : fabric_down_condition_on_pfe
Identifier : /fpc/0/fabricHub/0/cm/0/fabrichub/1/fabric_down_condition_on_pfe
Description : fabric_down_condition_on_pfe
State : enabled
Scope : pfe
Category : functional
Level : major
Threshold : 1
Error limit : 0
Occur count : 3
Clear count : 2
Last occurred(ms ago) : 103158
System Active Errors Detail Information
FPC 0
----------------------------------------------------------------
Error Name : fabric_link_foam_fault
Identifier : /fpc/0/fabspoked-pfe/0/cm/0/pfe/0/fabric_link_foam_fault
Description : fabric link foam fault
State : enabled
Scope : board
Category : internal
Level : major
Threshold : 1
Error limit : 100
Occur count : 2
Clear count : 0
Last occurred(ms ago) : 113277
System Active Errors Detail Information
FPC 0
----------------------------------------------------------------
Error Name : fabric_link_foam_fault
Identifier : /fpc/0/fabspoked-pfe/0/cm/0/pfe/1/fabric_link_foam_fault
Description : fabric link foam fault
State : enabled
Scope : board
Category : internal
Level : major
Threshold : 1
Error limit : 100
Occur count : 12
Clear count : 0
Last occurred(ms ago) : 103267
System Active Errors Detail Information
RE 0
----------------------------------------------------------------
Error Name : fpga_min_supported_fw_ver_mismatch
Identifier : /re/0/hwdre/0/cm/0/fpga_fw_events/UBAM FPGA/fpga_min_supported_fw_ver_mismatch
Description : firmware_version_lower_than_minimum_expected
State : enabled
Scope : board
Category : functional
Level : minor
Threshold : 10
Error limit : 1
Occur count : 1
Clear count : 0
Last occurred(ms ago) : 68886367
FPC 1
----------------------------------------------------------------
Error Name : fabric_link_self_fabric_oam_runtime_fault
Identifier : /fpc/1/fabspoked-pfe/0/cm/0/pfe/0/fabric_link_self_fabric_oam_runtime_fault
Description : fabric link self fabric oam runtime fault
State : enabled
Scope : board
Category : internal
Level : major
Threshold : 1
Error limit : 36
Occur count : 1
Clear count : 0
Last occurred(ms ago) : 2022-02-16 00:02:03 PST (448108 ms ago) System Active Errors Detail Information
CLI 명령을 show chassis fabric fpcs 사용하여 각 패브릭 플레인의 패브릭 OAM 셀프 핑 상태를 확인할 수 있습니다.
user@router> show chassis fabric fpcs
Fabric management FPC state:
FPC #0
PFE #0
SIB0_Asic0_Fcore0 (plane 0) Plane Disabled, Links ok Fabric OAM failed
SIB0_Asic0_Fcore0 (plane 1) Plane Enabled, Links ok Fabric OAM success
SIB0_Asic0_Fcore0 (plane 2) Plane Enabled, Links ok Fabric OAM success
SIB0_Asic0_Fcore0 (plane 3) Plane Enabled, Links ok Fabric OAM success
SIB0_Asic0_Fcore0 (plane 4) Plane Enabled, Links ok Fabric OAM success
SIB0_Asic0_Fcore0 (plane 5) Plane Enabled, Links ok Fabric OAM success
SIB1_Asic0_Fcore0 (plane 6) Plane Enabled, Links ok Fabric OAM success
SIB1_Asic0_Fcore0 (plane 7) Plane Enabled, Links ok Fabric OAM success
SIB1_Asic0_Fcore0 (plane 8) Plane Enabled, Links ok Fabric OAM success
SIB1_Asic0_Fcore0 (plane 9) Plane Enabled, Links ok Fabric OAM success
SIB1_Asic0_Fcore0 (plane 10) Plane Enabled, Links ok Fabric OAM success
SIB1_Asic0_Fcore0 (plane 11) Plane Enabled, Links ok Fabric OAM success
PFE #1
SIB0_Asic0_Fcore0 (plane 0) Plane Enabled, Links ok Fabric OAM success
SIB0_Asic0_Fcore0 (plane 1) Plane Enabled, Links ok Fabric OAM success
user@router> show chassis fabric fpcs
Fabric management FPC state:
FPC #1
PFE #0
SIB0_Asic0_Fcore0 (plane 0) Plane Enabled, Links ok Fabric OAM Runtime success
SIB0_Asic0_Fcore0 (plane 1) Plane Disabled, Links ok Fabric OAM Runtime failed
SIB0_Asic1_Fcore0 (plane 2) Plane Enabled, Links ok Fabric OAM Runtime success
SIB0_Asic1_Fcore0 (plane 3) Plane Enabled, Links ok Fabric OAM Runtime success
SIB0_Asic2_Fcore0 (plane 4) Plane Enabled, Links ok Fabric OAM Runtime success
SIB0_Asic2_Fcore0 (plane 5) Plane Enabled, Links ok Fabric OAM Runtime success
SIB1_Asic0_Fcore0 (plane 6) Plane Enabled, Links ok Fabric OAM Runtime success
SIB1_Asic0_Fcore0 (plane 7) Plane Enabled, Links ok Fabric OAM Runtime success
SIB1_Asic1_Fcore0 (plane 8) Plane Enabled, Links ok Fabric OAM Runtime success
SIB1_Asic1_Fcore0 (plane 9) Plane Enabled, Links ok Fabric OAM Runtime success
SIB1_Asic2_Fcore0 (plane 10) Plane Enabled, Links ok Fabric OAM Runtime success
SIB1_Asic2_Fcore0 (plane 11) Plane Enabled, Links ok Fabric OAM Runtime success
SIB2_Asic0_Fcore0 (plane 12) Plane Enabled, Links ok Fabric OAM Runtime success
SIB2_Asic0_Fcore0 (plane 13) Plane Enabled, Links ok Fabric OAM Runtime success
SIB2_Asic1_Fcore0 (plane 14) Plane Enabled, Links ok Fabric OAM Runtime success
SIB2_Asic1_Fcore0 (plane 15) Plane Enabled, Links ok Fabric OAM Runtime success
show chassis fabric fpcs 명령은 패브릭 OAM 기능이 비활성화된 경우 다음 출력을 표시합니다.
user@router> show chassis fabric fpcs
Fabric management FPC state:
FPC #0
PFE #0
SIB0_Asic0_Fcore0 (plane 0) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 1) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 2) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 3) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 4) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 5) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 6) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 7) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 8) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 9) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 10) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 11) Plane Enabled, Links ok
PFE #1
SIB0_Asic0_Fcore0 (plane 0) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 1) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 2) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 3) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 4) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 5) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 6) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 7) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 8) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 9) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 10) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 11) Plane Enabled, Links ok
PFE #2
SIB0_Asic0_Fcore0 (plane 0) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 1) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 2) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 3) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 4) Plane Enabled, Links ok
SIB0_Asic0_Fcore0 (plane 5) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 6) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 7) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 8) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 9) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 10) Plane Enabled, Links ok
SIB1_Asic0_Fcore0 (plane 11) Plane Enabled, Links ok
PFE #3