Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
 

Troubleshoot Deployment

Learn how to troubleshoot your Cloud-Native Router deployment.

Note:

We use the JSON processor jq to format the output of some commands. If jq is not installed on your OS by default, install it using the appropriate package manager for your OS. For example:

Common Problems

Table 1 lists some common deployment problems and remedies.

Table 1: Common Problems
Potential issue What to check Related Commands
Image not found Check if the images are uploaded to the local docker using the command docker images. If not, then the registry configured in values.yaml should be accessible. Ensure image tags are correct.
kubectl -n jcnr describe pod <crpd-pod-name>
Initialization errors Check if jcnr-secrets is loaded and has a valid license key
[root@jcnr-01]# kubectl get secrets -n jcnr
NAME                  TYPE                                  DATA   AGE
crpd-token-zp8kc      kubernetes.io/service-account-token   3      29d
default-token-zn6p9   kubernetes.io/service-account-token   3      29d
jcnr-secrets          Opaque                                2      29d
Confirm that root password and license key are present in /var/run/jcnr/juniper.conf
cRPD Pod in CrashLoopBackOff state
  • Check if startup/liveness probe is failing or vrouter pod not running
  • rpd-vrouter-agent gRPC connection not UP
  • Composed configuration is invalid or config template is invalid
  • kubectl get pods -A
    kubectl -n jcnr describe pod <crpd-pod-name>
    tail –f /var/log/jcnr/jcnr-cni.log​
    tail –f /var/log/jcnr/jcnr_notifications.json
  • See Access cRPD CLI to enter the cRPD CLI and run the following command:

    show krt state channel vrouter
  • cat /var/run/jcnr/juniper.conf

vRouter Pod in CrashLoopBackOff state

Check the contail-k8s-deployer logs for errors See Check Deployer Logs.

Check Deployer Logs

The deployer logs should be one of the first places you look when you run into installation problems.
To check the deployer logs:
  1. List the deployer pod.

    Sample output:

  2. View the deployer logs.

Verify vRouter and cRPD Health

  1. Check the vRouter daemonset.
    1. List the daemonsets.

      Sample output:

    2. Get vRouter daemonset details.

      Sample output:

  2. Check the cRPD stateful set.
    1. List the stateful sets.

      Sample output:

    2. Get the cRPD stateful set details.

      Sample output:

Verify cRPD Configuration

The Cloud-Native Router deployment process creates a cRPD configuration file from the parameters in values.yaml for L2 mode and custom configuration via node annotations in L3 mode. This cRPD configuration file is at /var/run/jcnr/juniper.conf on any node running JCNR.

The cRPD configuration can be customized using node annotations. The cRPD pod will stay in pending state if the applied configuration is invalid.

The rendered custom configuration is in /etc/crpd/juniper.conf.master.

In an AWS EKS deployment you can see the rendered custom configuration by accessing the cRPD CLI and navigating to the /config directory.

View Log Files

You can find the Cloud-Native Router log files in the default log_path directory (/var/log/jcnr/) on any node running JCNR. You can change this location by changing the value of the log_path or syslog_notifications parameters in the values.yaml file prior to deployment.

Here's an example of some of the log files that Cloud-Native Router keeps.

Note:

If your deployment fails, check the applier logs in applier/applier.log for more information.