Overview of System Logging
SUMMARY This section describes the system log messages that identify the Junos OS process that generated the message and briefly describes the operation or error that occurred.
System Log Overview
Junos OS generates system log messages (also called syslog messages) to record events that occur on the device, including the following:
-
Routine operations, such as creation of an Open Shortest Path First (OSPF) protocol adjacency or a user login to the configuration database
-
Failure and error conditions, such as failure to access a configuration file or unexpected closure of a connection to a peer process
-
Emergency or critical conditions, such as router power-down due to excessive temperature
In Junos OS Release 17.3R1, the syslog-event daemon is able to handle the fxp0 in dedicated management routing instance for for IPv4 addressed remote host. As of Junos OS Release 18.1R1, the syslog-event daemon supports IPv6-based configuration when connecting to a remote host or an archival site and fxp0 is moved to dedicated management instance. In Junos OS Release 18.4R1, the syslog client can send messages through any routing instance you define at appropriate hierarchies. See routing-instance (Syslog).
This topic describes system log messages for Junos OS processes and libraries and not the system logging services on a Physical Interface Card (PIC) such as the Adaptive Services PIC.
In Junos OS Evolved, each node has the standard journalctl
tool, which
is an interface to retrieve and filter the system journal. System log messages are
parsed from the system journal. The relay-eventd
process runs on all
nodes and retrieves events (based on the syslog configuration) from the system journal
as well as error messages from the different applications and forwards them to the
master-eventd
process. The master-eventd
process
runs on the primary Routing Engine and writes the log messages and errors to disk.
Use the System Log Explorer application to view or compare system log messages in different releases.
In Junos OS Evolved there is no messages
file on the backup Routing
Engine. All backup Routing Engine logs are in the messages
file on the
primary Routing Engine node.
By default, Junos OS Evolved appends the node name to the hostname in system log messages; Junos OS does not. This action keeps Junos OS Evolved system log messages compliant with RFC5424. However, some monitoring systems may not identify a Junos OS Evolved hostname correctly, because the hostname-node name combination does not match any hostnames in the inventory of hostnames.
If your monitoring system is not identifying Junos OS Evolved hostnames correctly, as of
Junos OS Evolved Release 20.4R2, you should issue the set system syslog
alternate-format
configuration mode command. This command changes the
format of the Junos OS Evolved system log messages. The node name is prepended to the
process name in the message rather than appended to the hostname, thereby allowing the
monitoring system to identify the hostname correctly.
For example, Junos OS system log messages do not print the origin process in system log messages coming from an FPC:
user@mxhost> show log messages Dec 19 13:22:41.959 mxhost chassisd[5290]: CHASSISD_IFDEV_DETACH_FPC: ifdev_detach_fpc(0) Dec 19 13:23:22.900 mxhost fpc2 Ukern event counter Sock_tx init delayed
However, Junos OS Evolved messages append the node name to the hostname and do print the origin process for messages coming from a node, including FPCs:
user@ptxhost-re0> show log messages May 25 18:41:05.375 ptxhost-re0 mgd[16201]: UI_CHILD_STATUS: Cleanup child '/usr/sbin/dot1xd', PID 21322, status 0 May 25 18:42:34.632 ptxhost-fpc0 evo-cda-bt[14299]: Register bt.igp_misc.debug.hdr_length_cnt not found May 25 18:42:34.753 ptxhost-fpc1 evo-cda-bt[14427]: HBM: hbm_gf_register_inst May 25 18:47:14.498 ptxhost-re0 ehmd[5598]: SYSTEM_APP_READY: App is ready re0-ehmd
If you have configured the alternate format for Junos OS Evolved system log messages, the same set of system log messages would look like this instead, with the hostname by itself:
user@ptxhost-re0> show log messages May 25 18:41:05.375 ptxhost re0- mgd[16201]: UI_CHILD_STATUS: Cleanup child '/usr/sbin/dot1xd', PID 21322, status 0 May 25 18:42:34.632 ptxhost fpc0- evo-cda-bt[14299]: Register bt.igp_misc.debug.hdr_length_cnt not found May 25 18:42:34.753 ptxhost fpc1- evo-cda-bt[14427]: HBM: hbm_gf_register_inst May 25 18:47:14.498 ptxhost re0- ehmd[5598]: SYSTEM_APP_READY: App is ready re0-ehmd
Overview of Junos OS System Log Messages
The Junos OS generates system log messages (also called syslog messages) to record events that occur on the switch, including the following:
-
Routine operations, such as a user login into the configuration database.
-
Failure and error conditions, such as failure to access a configuration file.
-
Emergency or critical conditions, such as power-down of the switch due to excessive temperature.
Each system log message identifies the Junos OS process that generated the message and briefly describes the operation or error that occurred. For detailed information about specific system log messages, see the System Log Explorer.
OCX Series switches comprise both the Junos OS and the host operating system (OS). For information about system logging on the host OS, see Managing Host OS System Log and Core Files.
To configure the device to log system messages, configure the syslog statement at the [edit system] hierarchy level.
Junos OS System Logging Facilities and Message Severity Levels
Table 1 lists the Junos
OS system logging facilities that you can specify in configuration
statements at the [edit system syslog]
hierarchy level.
Facility (number) |
Type of Event or Error |
---|---|
|
Actions performed or errors encountered by the Junos OS kernel |
|
Actions performed or errors encountered by user-space processes |
|
Actions performed or errors encountered by system processes |
|
Authentication and authorization attempts |
|
Actions performed or errors encountered by the FTP process |
|
Actions performed or errors encountered by the Network Time Protocol processes. |
|
Security related events or errors. |
|
Events related to dynamic flow capture |
|
Actions performed or errors encountered by the local external applications. |
|
Packet filtering actions performed by a firewall filter |
|
Actions performed or errors encountered by the Packet Forwarding Engine |
|
Specified configuration is invalid on the router type |
|
Changes to the Junos OS configuration |
|
Commands issued at the Junos OS command-line interface (CLI) prompt or by a client application such as a Junos XML protocol or NETCONF XML client |
Table 2 lists the severity
levels that you can specify in configuration statements at the [edit system syslog]
hierarchy level. The levels from emergency
through info
are in order from highest
severity (greatest effect on functioning) to lowest.
Unlike the other severity levels, the none
level disables logging of a facility instead of indicating how seriously
a triggering event affects routing functions. For more information,
see Disabling the System Logging of a Facility.
Value |
Severity Level |
Description |
---|---|---|
N/A |
|
Disables logging of the associated facility to a destination |
0 |
|
System panic or other condition that causes the router to stop functioning |
1 |
|
Conditions that require immediate correction, such as a corrupted system database |
2 |
|
Critical conditions, such as hard errors |
3 |
|
Error conditions that generally have less serious consequences than errors at the emergency, alert, and critical levels |
4 |
|
Conditions that warrant monitoring |
5 |
|
Conditions that are not errors but might warrant special handling |
6 |
|
Events or nonerror conditions of interest |
7 |
|
Includes all severity levels |
Junos OS Default System Log Settings
Table 3 summarizes the default system log settings that apply to all routers that run the Junos OS, and specifies which statement to include in the configuration to override the default value.
Setting |
Default |
Overriding Statement |
Instructions |
---|---|---|---|
Alternative facility for message forwarded to a remote machine |
For For For For For For |
[edit system syslog] host hostname { facility-override facility; } |
Changing the Alternative Facility Name for System Log Messages Directed to a Remote Destination |
Format of messages logged to a file |
Standard Junos OS format, based on UNIX format |
[edit system syslog] file filename { structured-data; } |
|
Maximum number of files in the archived set |
10 |
[edit system syslog] archive { files number; } file filename { archive { files number; } } |
|
Maximum size of the log file |
M Series, MX Series, and T Series: 1 megabyte (MB) TX Matrix: 10 MB |
[edit system syslog] archive { size size; } file filename { archive { size size; } } |
|
Timestamp format |
Month, date, hour, minute, second For example: |
[edit system syslog] time-format format; |
|
Users who can read log files |
|
[edit system syslog] archive { world-readable; } file filename { archive { world-readable; } } |
Junos OS Platform-Specific Default System Log Messages
The following messages are generated by default on specific routers. To view any of these types of messages, you must configure at least one destination for messages as described in Junos OS Minimum System Logging Configuration.
To log the kernel process message on an M Series, MX Series, or T Series router, include the
kernel info
statement at the appropriate hierarchy level:[edit system syslog] (console | file filename | host destination | user username) { kernel info; }
On a routing matrix composed of a TX Matrix router and T640 routers, the primary Routing Engine on each T640 router forwards all messages with a severity of
info
and higher to the primary Routing Engine on the TX Matrix router. This is equivalent to the following configuration statement included on the TX Matrix router:[edit system syslog] host scc-master { any info; }
Starting in Junos OS Release 15.1X49-D10 and Junos OS Release 17.3R1, likewise on a routing matrix composed of a TX Matrix Plus router with connected T1600 or T4000 routers, the primary Routing Engine on each T1600 or T4000 LCC forwards to the primary Routing Engine on the TX Matrix Plus router all messages with a severity of
info
and higher. This is equivalent to the following configuration statement included on the TX Matrix Plus router:Note:From the perspective of the user interface, the routing matrix appears as a single router. The TX Matrix Plus router controls all the T1600 or T4000 routers connected to it in the routing matrix.
[edit system syslog] host sfc0-master { any info; }
Interpreting Messages Generated in Standard Format
The syntax of a standard-format message generated by a Junos OS process or subroutine library depends on whether it includes priority information:
When the
explicit-priority
statement is included at the [filename] or [hostname] hierarchy level, a system log message has the following syntax:timestamp message-source: %facility–severity–TAG: message-text
When directed to the console or to users, or when the
explicit-priority
statement is not included for files or remote hosts, a system log message has the following syntax:timestamp message-source: TAG: message-text
Table 4 describes the message fields.
Field | Description |
---|---|
timestamp |
Time at which the message was logged. |
message-source |
Identifier of the process or component that generated the message and the routing platform on
which the message was logged.
For
Junos OS, this field includes two or more
subfields: hostname, process and process ID (PID).
For
Junos OS Evolved, this field includes a hostname with an appended
node name, a process name, and PID. If the
hostname process[process-ID] |
facility |
Code that specifies the facility to which the system log message belongs. For a mapping of codes to facility names, see Table: Facility Codes Reported in Priority Information in Including Priority Information in System Log Messages. |
severity |
Numerical code that represents the severity level assigned to the system log message. For a mapping of codes to severity names, see Table: Numerical Codes for Severity Levels Reported in Priority Information in Including Priority Information in System Log Messages. |
TAG |
Text string that uniquely identifies the message, in all uppercase letters and using the underscore (_) to separate words. The tag name begins with a prefix that indicates the generating software process or library. The entries in this reference are ordered alphabetically by this prefix. Not all processes on a routing platform use tags, so this field does not always appear. |
message-text |
Text of the message. |
Managing Host OS System Log and Core Files
On Junos OS switches with a host OS, the Junos OS might generates system log messages (also called syslog messages) to record events that occur on the switch, including the following:
Routine operations, such as a user login into the configuration database.
Failure and error conditions.
Emergency or critical conditions, such as power-down of the switch due to excessive temperature.
On OCX Series switches:
System log messages are logged in the /var/log/dcpfe.log file in the host OS in the following scenarios:
When the forwarding daemon is initialized.
Messages are tagged as emergency (LOG_EMERG). A copy of the message is also sent to the /var/log directory on the switch.
Messages from processes are available on the host system in the /var/log directory. System log messages from the host chassis management process are recorded in the lcmd.log file in the /var/log directory.
On QFX switches with a host OS:
The Junos OS and host OS record log messages for system and process events, and generate core files upon certain system failures.
These files are stored in directories such as /var/log for log messages, and /var/tmp or /var/crash for core files, depending on the type of host OS running on the switch.
For diagnostic purposes, you can access these host OS system log and core files from the Junos OS CLI on the switch. You can also clean up directories where the host OS stores temporary log and other files.
This topic includes these sections:
- Viewing Log Files On the Host OS System
- Copying Log Files From the Host System To the Switch
- Viewing Core Files On the Host OS System
- Copying Core Files From the Host System To the Switch
- Cleaning Up Temporary Files on the Host OS
Viewing Log Files On the Host OS System
To view a list of the log files created on the host OS, enter the following command:
user@switch> show app-engine logs
Copying Log Files From the Host System To the Switch
To copy log files from the host OS to the switch, enter the following command:
user@switch> request app-engine file-copy log from-jhost source to-vjunos destination
For example, to copy the lcmd log file to the switch, enter the following command:
user@switch> request app-engine file-copy log from-jhost lcmd.log to-vjunos /var/tmp
Viewing Core Files On the Host OS System
To view the list of core files generated and stored on the host OS system, enter the following command:
user@switch> show app-engine crash
The list might look like this example output:
Compute cluster: default-cluster Compute node: default-node Crash Info ========== total 13480 -rw-r--r-- 1 root root 178046 Feb 14 23:08 localhost.lcmd.26653.1455520135.core.tgz -rw-r--r-- 1 root root 4330343 Feb 15 00:45 localhost.dcpfe.7155.1455525926.core.tgz -rw-r--r-- 1 root root 4285901 Feb 15 01:49 localhost.dcpfe.25876.1455529782.core.tgz -rw-r--r-- 1 root root 4288508 Feb 15 02:39 localhost.dcpfe.713.1455532774.core.tgz -rw-r--r-- 1 root root 264079 Feb 15 17:02 localhost.lcmd.1144.1455584540.core.tgz
Copying Core Files From the Host System To the Switch
To copy core files from the host OS to the switch, enter the following command:
user@switch> request app-engine file-copy crash from-jhost source to-vjunos destination-dir-or-file-path
When the destination Junos OS path is a directory, the source filename is used by default. To rename the file at the destination, enter the destination argument as a full path including the desired filename.
For example, to copy the localhost.lcmd.26653.1455520135.core.tgz core archive file to the switch, enter the following command:
user@switch> request app-engine file-copy crash from-jhost localhost.lcmd.26653.1455520135.core.tgz to-vjunos /var/tmp
To see the results on the switch, enter the following command:
user@switch> show system core-dumps re0: -------------------------------------------------------------------------- -rw-r--r-- 1 root field 178046 Feb 15 17:15 /var/tmp/localhost.lcmd.26653.1455520135.core.tgz total files: 1
Cleaning Up Temporary Files on the Host OS
To remove temporary files created on the host OS, enter the following command:
user@switch> request app-engine cleanup
For example, the following sample output on a switch with a Linux host OS shows cleanup of temporary files stored in /var/tmp:
Compute cluster: default-cluster Compute node: default-node Cleanup (/var/tmp) =======
info
and higher.