恢复 CSO 服务
恢复 CSO 服务
components_health.sh如果脚本检测到任何服务运行不正常,则可以使用脚本recovery.sh恢复该服务。
恢复脚本启动指定组件(本例中为 saltstack)的恢复过程。以下是显示的消息的示例输出。恢复组件后,将显示恢复完成消息。
INFO Started recovering saltstack component at 2021-07-22 03:00:08.989767 ... INFO Saltstack failure recovery is initiated... INFO Saltstack check() INFO Salt Master is running INFO Deleting unreachable minion key csp-central-proxy_sblb2.N6RGW8.central INFO Deleting unreachable minion key csp-central-k8-microservices3.N6RGW8.central INFO Deleting unreachable minion key csp-central-k8-microservices2.N6RGW8.central INFO Deleting unreachable minion key csp-central-k8-infra3.N6RGW8.central INFO Completed recovering saltstack component at 2021-07-22 03:00:27.816847 . INFO Time taken to recover 0:00:18.827080
如果 recovery.sh 脚本在恢复 kubernetes 组件时检测到 k8 虚拟机存在问题,则会显示一条错误消息,如以下示例输出所示:
Kubernetes recovery failed, Please refer logs/recovery.log for more details Failed to recover Kubernetes Please run replace_vm for k8-master1
您可以运行 deploy.sh 脚本来替换 k8 虚拟机。
替换 KVM 虚拟机管理程序的虚拟机
您只能替换 k8 虚拟机。要替换 k8 虚拟机,请执行以下作:
注意:
如果物理服务器上发生电源重启,VRR 将变得不正常。
以下是将 vrr 状态显示为不正常的示例输出:
INFO Health Check for Infrastructure Component Vrr Started INFO Attempt: 1 - Retrying Health Check for Component Vrr INFO Attempt: 2 - Retrying Health Check for Component Vrr ERROR The Infra Component : Vrr is Unhealthy
再次运行 recovery.sh 脚本以使 vrr 重新联机。以下是恢复过程的示例输出:
root@startupserver1:~/Contrail_Service_Orchestration_6.1.0# ./recovery.sh *************** This tool assists you recover your CSO setup. *************** Following components can be recovered 1: contrailanalytics 2: cassandra 3: mariadb 4: etcd 5: kubernetes 6: vrr 7: saltstack 8: arangodb 9: microservices 10: icinga 11: rabbitmq Specify one of the component to recover (In Number) : 6 INFO Started recovering vrr component at 2021-07-22 07:38:23.504490 ... INFO VRR recovery is initiated... INFO Vrr - 192.168.10.29 is healthy ERROR Vrr - 192.168.10.30 is unhealthy INFO Recovery takes time, please be patient INFO VRR recovery started. Please wait... INFO Vrr console recovered for vrr2 INFO VRR config sync completed successfully INFO Completed recovering vrr component at 2021-07-22 07:48:31.324598 . INFO Time taken to recover 0:10:07.820108