Troubleshooting CSO Installation Issues
Salt Key Issue During CSO Installation
Problem
Description: The infrastructure services for the central infrastructure virtual machine (infravm) and the central microservices virtual machine (msvm) ran successfully. However, the Contrail Analytics virtual machine (contrail-analytics-vms) failed with the following key exchange error.
root@cso-central-2:~#salt-key -L Accepted Keys: csp-central-infravm.M8DLI0.central csp-central-msvm.M8DLI0.central Denied Keys: Unaccepted Keys: Rejected Keys: 2017-06-09 13:09:37 INFO utils.core Minion csp-contrailanalytics-vm.M8DLI0.central pointed to the salt master 2017-06-09 13:09:37 INFO utils.core RP: salt_master accepted keys are: ['csp-central-infravm.M8DLI0.central', 'csp-central-msvm.M8DLI0.central'] 2017-06-09 13:09:47 ERROR utils.core key exchange did not go through by reactor for csp-contrailanalytics-vm 2017-06-09 13:09:47 INFO utils.core Pinging minion csp-contrailanalytics-vm.M8DLI0.central 2017-06-09 13:09:47 INFO utils.core timeout 5 2017-06-09 13:09:47 INFO utils.core calling test.ping on csp-contrailanalytics-vm.M8DLI0.central 2017-06-09 13:09:47 INFO utils.core timeout 10 2017-06-09 13:09:47 INFO utils.core calling test.ping on csp-contrailanalytics-vm.M8DLI0.central 2017-06-09 13:09:47 INFO utils.core timeout 15 2017-06-09 13:09:47 INFO utils.core calling test.ping on csp-contrailanalytics-vm.M8DLI0.central 2017-06-09 13:09:47 INFO utils.core timeout 20 2017-06-09 13:09:47 INFO utils.core calling test.ping on csp-contrailanalytics-vm.M8DLI0.central 2017-06-09 13:09:47 INFO utils.core timeout 25 2017-06-09 13:09:47 INFO utils.core calling test.ping on csp-contrailanalytics-vm.M8DLI0.central 2017-06-09 13:09:47 INFO utils.core Ping to minion csp-contrailanalytics-vm.M8DLI0.central failed 2017-06-09 13:09:48 INFO utils.core setting up salt agent on csp-contrailanalytics-vm 2017-06-09 13:09:50 INFO utils.core Salt minion already installed on server csp-contrailanalytics-vm 2017-06-09 13:09:50 INFO utils.core Pointing minion to installer host 2017-06-09 13:09:57 INFO utils.core attaching to master 192.168.255.53
Solution
- Verify that the deployment script is running on the installer virtual machine and on the analytics virtual machine.
- Execute the following commands:
root@cso-central-2:~#rm –rf /var/cache/salt*root@cso-central-2:~#sudo apt-get remove salt-minionroot@cso-central-2:~#sudo apt-get purge salt-minion
TimeZone Error
Problem
Description: The following error message is displayed when installing the central microservices virtual machine.
ERROR State timezone_|-time_zone_|-America/Los_Angeles_|-system on csp-central-msvm.D2IHJM.central is in error ERROR Some errors while deploying the roles set(['ntp']) ERROR Please check the logs and correct
Solution
This issue occurs only when the NTP server input is not synchronized with the virtual machine. Check the NTP server details in the setup assistant.
SSL Handshake Failure
Problem
Description: SSL handshake failures are reported on both regional services (regionalmsvm1 and regionalmsvm2) in Icinga.
CHECK_NRPE: Error - Could not complete SSL handshake. Check execution Command check_nrpe Check Source centralmsvm2.sst.net.cn is reachable Check execution Command check_nrpe Check Source centralmsvm1.sst.net.cn is reachable 192.1.1.1 regionalmsvm1.sst.net.cn regionalmsvm1 192.1.1.2 regionalmsvm2.sst.net.cn regionalmsvm2
Solution
- Edit the
/etc/nagios/nrpe.cfgfile at the regional microservices virtual machine. The allowed_hosts might have the central microservices virtual machine management address instead of the Operation, Administration, and Maintenance (OAM) address. Add the OAM IP address. For example:allowed_hosts=192.0.2.0, 192.0.3.0 - Send the kill –HUP command to the Nagios
Remote Plugin Executor (NRPE) process.
root@regional-msvm:/etc/nagios#ps aux | grep nrpenagios 1428 0.0 0.0 23472 1204 ? Ss Aug14 0:38 /usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -d root 6535 0.0 0.0 10476 888 pts/6 S+ 15:40 0:00 grep --color=auto nrpe
root@regional-msvm:/etc/nagios#kill -HUP 1428
Missing Interface on CSO VM
Problem
Description: When two virtual bridge interfaces(virbr0 and virbr1) on the server are mapped to eth0 and eth1, only one interface is listed in the CSO virtual machines. The second interface is not visible to the CSO virtual machines.
root@host:~# brctl show
bridge name bridge id STP enabled interfaces
virbr0 8000.0cc47a6efbfa no eth0
vnet0
vnet1
vnet2
vnet3
vnet4
vnet5
vnet6
vnet7
virbr1 8000.0cc47a6efbfb no eth1
Solution
Assign an IP address to the second interface in
/etc/network/interfaces.Set enable_data_interface to true for all virtual machines and set data_interface to virbr1 for cso-host in the provision_vm.conf file.