Collect Crash Data for a Routing Engine Kernel
Purpose
When a Routing Engine kernel crashes, the Routing Engine automatically reboots. By default, the Juniper Networks router does not attempt to dump a core if the Routing Engine kernel crashes. As a result, there is no crash data on the router to help investigate the crash. In addition, the system log messages are similar to those generated when the router is powered down and restarted, so you cannot tell if the Routing Engine restart was caused by a kernel crash or a normal power restart.
To collect crash data for a Routing Engine kernel crash, follow these steps:
- Check the Routing Engine Core Files
- List the Core Files
- Compress the vmcore File
- Log Software Version Information
- Open a Case with JTAC
Check the Routing Engine Core Files
Purpose
If you observe disruption to the Routing Engine kernel, check the /var/crash directory for any core files created around the time of the crash.
Action
To check the /var/crash directory, use the following Junos OS command-line interface (CLI) operational mode command:
Sample Output
user@host> file list detail /var/crash total 1577912 drwxr-x--- 2 root wheel 512 Sep 9 11:59 ./ drwxr-xr-x 22 root wheel 512 Oct 29 2001 ../ -rw-r--r-- 1 root wheel 2 Jul 20 01:11 bounds -rw-r--r-- 1 root wheel 2166913 Jul 20 01:11 kernel.0 -rw-r--r-- 1 root wheel 5 Feb 15 2002 minfree -rw------- 1 root wheel 805306368 Jul 20 01:11 vmcore.0
Meaning
The sample output lists the contents of the /var/crash/ directory. Check the date and timestamp for any kernel core files created around the time of the crash. In the example above, two core files are listed: kernel.0 and vmcore.0.
List the Core Files
Purpose
To list the core files, follow these steps:
Action
- Exit from the CLI environment and create a UNIX-level
shell by entering the start shell command:user@host> start shell
- Type su and the root password when prompted. You are now in
the shell and the prompt is % instead of >, for example:% su Password: ****
- Change the directory to /var/crash and type ls -l, for example:root@host% cd /var/crash root@host% ls -l
- Look for any core files created around the time of the crash.
Sample Output
user@host> start shell
% su Password: **** root@host% cd /var/crash root@host% ls -l total 1577908 -rw-r--r-- 1 root wheel 2 Jul 20 01:11 bounds -rw-r--r-- 1 root wheel 2166913 Jul 20 01:11 kernel.0 -rw-r--r-- 1 root wheel 5 Feb 15 2002 minfree -rw------- 1 root wheel 805306368 Jul 20 01:11 vmcore.0
Meaning
The sample output lists the contents of the /var/crash directory and shows the current core files kernel.0 and vm.core.0.
Compress the vmcore File
Purpose
The gzip compression utility is used to compress files. Compress the vmcore file if it is larger than 50 MB. Files created using the gzip command end with the file extension .gz.
 | Note: Use lowercase for the gzip command when you are in the shell. |
Action
To compress the vmcore file with gzip, use the following command from the shell:
To unzip the vmcore file with gzip, use the following command from the shell:
Meaning
The contents of the vmcore file are compressed into a single compressed file named vmcore.number.gz. The gzip command preserves the mode, ownership, and timestamps of files when compressing or decompressing them.
Log Software Version Information
Purpose
To log the Junos OS version information.
Action
To log the Junos OS version information, use the following Junos OS CLI operational mode command:
Sample Output
user@host> show version
Hostname: host Model: m10 JUNOS Base OS boot [5.0R5] JUNOS Base OS Software Suite [5.0R5] JUNOS Kernel Software Suite [5.0R5] JUNOS Routing Software Suite [5.0R5] JUNOS Packet Forwarding Engine Support [5.0R5] JUNOS Crypto Software Suite [5.0R5] JUNOS Online Documentation [5.0R5] KERNEL 5.0R5 #0 built by builder on 2002-03-02 05:10:28 UTC MGD release 5.0R5 built by builder on 2002-03-02 04:45:32 UTC CLI release 5.0R5 built by builder on 2002-03-02 04:44:22 UTC CHASSISD release 5.0R5 built by builder on 2002-03-02 04:43:37 UTC DCD release 5.0R5 built by builder on 2002-03-02 04:42:47 UTC RPD release 5.0R5 built by builder on 2002-03-02 04:46:17 UTC SNMPD release 5.0R5 built by builder on 2002-03-02 04:52:26 UTC MIB2D release 5.0R5 built by builder on 2002-03-02 04:45:37 UTC APSD release 5.0R5 built by builder on 2002-03-02 04:43:31 UTC VRRPD release 5.0R5 built by builder on 2002-03-02 04:52:34 UTC ALARMD release 5.0R5 built by builder on 2002-03-02 04:43:24 UTC PFED release 5.0R5 built by builder on 2002-03-02 04:46:06 UTC CRAFTD release 5.0R5 built by builder on 2002-03-02 04:44:30 UTC SAMPLED release 5.0R5 built by builder on 2002-03-02 04:52:20 UTC ILMID release 5.0R5 built by builder on 2002-03-02 04:45:21 UTC BPRELAYD release 5.0R5 built by builder on 2002-03-02 04:42:41 UTC RMOPD release 5.0R5 built by builder on 2002-03-02 04:46:11 UTC jkernel-dd release 5.0R5 built by builder on 2002-03-02 04:41:07 UTC jroute-dd release 5.0R5 built by builder on 2002-03-02 04:41:21 UTC jdocs-dd release 5.0R5 built by builder on 2002-03-02 04:39:11 UTC
Meaning
The sample output shows the hostname, router model, and the different Junos OS packages, processes, and documents.
Open a Case with JTAC
Problem
To open a case with JTAC, follow these steps:
Solution
- Send an e-mail to support@juniper.net, and include the information from the show version command.
- At the support engineer’s request, ftp the vmcore.number.gz file to a case-number directory at ftp.juniper.net. To ftp the core file to a directory, follow
these steps:
- At the shell prompt, enter ftp ftp.juniper.net.
- At the name prompt, enter anonymous.
- At the password prompt, enter your e-mail address as the password.
- At the ftp prompt, enter the cd pub/ incoming string.
- Enter the mkdir case-number command, where the case-number is the value of the case you opened with JTAC, for example, 1999-1231-9999. If a directory has already been created, continue with the next step.
- Enter the cd case-number command.
- Enter the binary command so that the file transfer is in binary and not ASCII.
- Enter the put vmcore.0.gz command.
Meaning
The following output is an example of copying a core file from the shell to an ftp directory at ftp.juniper.net:
Sample Output
root@host% ftp ftp.juniper.net Connected to colo-ftp.juniper.net. 220 colo-ftp.juniper.net FTP server (Version 6.00LS) ready. Name (ftp.juniper.net: root): anonymous 331 Guest login ok, send your email address as password. Password: **** 230 Guest login ok, access restrictions apply. ftp> cd pub/ incoming 250 CWD command successful ftp> mkdir 1999-1231-9999 257 MKD command successful. ftp> cd 1999-1231-9999 250 CWD command successful. ftp> bin 200 Type set to I. ftp> put vmcore.0.gz
Meaning
The sample output shows that there is a connection to ftp.juniper.net, that the login name and password were entered, and that the core file was successfully copied from the shell to an ftp directory at ftp.juniper.net.
