Configure System Logging for Security Devices

Control Plane and Data Plane Logs

Junos OS generates separate log messages to record events that occur on the system’s control and data planes.

• The control plane logs, also called system logs, include events that occur on the routing platform. The system sends control plane events to the eventd process on the Routing Engine, which then handles the events by using Junos OS policies, by generating system log messages, or both. You can choose to send control plane logs to a file, user terminal, routing platform console, or remote machine. To generate control plane logs, use the syslog statement at the [system] hierarchy level.

• The data plane logs, also called security logs, primarily include security events that are handled inside the data plane. Security logs can be in text or binary format, and they can be saved locally (event mode) or sent to an external server (stream mode). Binary format is required for stream mode and recommended to conserve log space in event mode.

  Note the following:

  • Security logs can be saved locally (on box) or externally (off box), but not both.

  • SRX1400, SRX1500, SRX3400, SRX3600, SRX4100, SRX4200, SRX4600, SRX5400, SRX5600, and SRX5800 devices default to stream mode. To specify binary format and an external server, see Configuring Off-Box Binary Security Log Files.

    Note:

    Logs might be dropped if you configure event mode logging on these devices.

    Starting with Junos OS Release 15.1X49-D100, the default mode for SRX1500 device is stream mode. Prior to Junos OS Release 15.1X49-D100, the default mode for SRX1500 device was event mode.

  • Apart from Junos OS Release 18.4R1, 18,4R2, 19.1 and 19.2R1, in all the other releases starting in Junos OS Release 18.3R3, the default logging mode for SRX300, SRX320, SRX340, SRX345, SRX550, and SRX550M devices is stream mode. Data plane events are written to system log files in a similar manner to control plane events. To specify binary format for the security logs, see Configuring Off-Box Binary Security Log Files.

  Starting in Junos OS Release 20.2R1, we support escape in stream log forwarding and on-box reporting to avoid parsing errors. Stream mode supports escape in sd-syslog and binary formats when logs are not sent to eventd process. For the logs send to eventd process, we recommend not to enable an escape option as the eventd process has enabled the escape for the structure log. Event mode supports escape only in binary format. By default the escape option is disabled. You must enable the escape option using the set security log escape command.

Redundant System Log Server

Security system logging traffic intended for remote servers is sent through the network interface ports, which support two simultaneous system log destinations. Each system logging destination must be configured separately. When two system log destination addresses are configured, identical logs are sent to both destinations. While two destinations can be configured on any device that supports the feature, adding a second destination is primarily useful as a redundant backup for standalone and active/backup configured chassis cluster deployments.

The following redundant server information is available:

• Facility: cron

• Description: cron scheduling process

• Severity Level (from highest to lowest severity): debug

• Description: Software debugging messages

Overview

On-box traffic logging to solid-state drives (SSDs) supports eight external log servers or files.

An all-in-one XML file is added that contains all the traffic logs information. The XML file also generates all the logging header files and traffic log related documents.

A new process (daemon) called local log management daemon (llmd) is supported in Services Processing Cards 0 (SPCs0) to handle on-box traffic logging. Traffic produced by flowd in SPCs is listed in traffics logs. The llmd saves these logs to the local SSD. Traffic logs are saved in the four different formats. See Table 1 to know about the log formats.

On-box reporting mechanism is an enhancement to the existing logging functionality. The existing logging functionality is modified to collect system traffic logs, analyzes the logs, and generate reports of these logs in the form of tables using the CLI. On-box reporting feature is intended to provide a simple and easy to use interface for viewing security logs. The on-box reports are easy to use J-Web pages of various security events in the form of tables and graphs. The reports allow the IT security management to identify security information at a glance, and quickly decide the actions to be taken. Thorough analysis of logs is performed (based on session types) for features such as screen, IDP, Content Security and IPSec.

You can define filters for the log data that is reported on based on the following criteria:

• top <number>—This option allow you to generate reports for top security events as specified in the command. for example: top 5 IPS attacks or top 6 URLs detected through Content Security.

• in-detail <number>—This option allow you to generate detail log content.

• in-interval <time-period>—This option allows you to generate the events logged between certain time intervals.

• summary—This option allows you to generate the summary of the events. In this way, you can fine-tune the report to your needs, and displays only the data that you want to use.

The maximum in-interval number which shows the count in intervals is 30. If large duration is specified, then the counters are assembled to ensure the maximum in-interval is less than 30.

Both in-detail and summary have the “all” option, since different table have different attribute (like session table does not have the attribute “reason” but Content Security has), the “all” option does not have any filter except start-time and stop-time. If there is any other filter other than start time and stop time then an error is displayed.

For example: root@host> show security log report in-detail all reason reason1

The application firewall logs for application and user visibility will list applications and nested applications. When the logs of these features list nested applications then nested applications are listed in J-Web. When the logs list nested applications as not-applicable or unknown then only the applications are listed in J-Web.

Use the following CLI commands for application and user visibility for all the applications and nested applications listing:

• For top nested-application by count—show security log report top session-close top-number <number> group-by application order-by count with user

• For top nested-application by volume—show security log report top session-close top-number <number> group-by application order-by volume with user

• For top user by count with nested application—show security log report top session-close top-number <number> group-by user order-by count with application

The on-box reporting feature is enabled by default when you load the factory-default configurations on the SRX Series Firewall with Junos OS Release 15.1X49-D100 or later.

If you are upgrading your SRX Series Firewall from a Junos OS Release prior to Junos OS 15.1X49-D100, then the SRX Series Firewall inherits the existing configuration and the on-box reporting feature is disabled by default. You need to configure the set security log report command and the set security log mode stream command to enable the on-box reporting feature on the device that are upgraded.

Starting in Junos OS Release 19.3R1 , the factory-default configuration does not include on-box reporting configuration to increase the solid-state drive (SSD) lifetime. You can enable the on-box reporting feature by configuring the set security log report CLI command at [edit security log] hierarchy.

See J-Web User Guide for SRX Series Devices to perform this task on J-Web user interface.

Starting in Junos OS Release 21.3R1, the on-box reporting logs are stored on the memory file system (MFS) if there is no external SSD. The maximum number of logs that you can save on MFS is lesser than what you can save on an external SSD. This prevents memory exhaustion and failure. Logs saved in MFS are not retained after device reboot or power failure. See Table 2 to know the number of logs recorded in on-box reporting and off-box reporting.

After the log message is recorded, the log is stored within a log file which is then stored in the database table of the RE for further analysis (on SRX300, SRX320, SRX340, SRX345, and SRX550M devices) or on the SSD card for further analysis (on SRX1500, SRX4100, and SRX4200 devices).

The benefits of this feature are:

• Reports are stored locally on the SRX Series Firewall and there is no requirement for separate devices or tools for logs and reports storage.

• The on-box reports are easy-to-use J-Web pages of various security events in the form of tables and graphs.

• Provides a simple and easy-to-use interface for viewing security logs.

• The reports generated enables the IT security management team to identify security information at a glance and quickly decide the actions to be taken.

The on-box reporting feature supports:

• Generating reports based on the requirements. For example: count or volume of the session, types of logs for activities such as IDP, Content Security, IPsec VPN.

• Capturing real-time events within a specified time range.

• Capturing all the network activities in a logical, organized, and easy-to-understand format based on various CLI specified conditions.

On-Box Reporting Features

The on-box reporting feature supports:

• Sqlite3 support as a library—sqlite3 was not supported prior to Junos OS release 15.1X49-D100. Starting with Junos OS Release 15.1X49-D100, an SQL log database (SQLite Version 3) is used by the daemons running on the RE as well as other potential modules to store logs on SRX Series Firewalls.

  In Junos OS Release 19.4R1, we've upgraded the on-box logging database to improve query performance.

• Running llmd in both Junos OS and Linux OS—The forwarding daemon (flowd) decodes database index from binary logs and sends both index and log to the local log management daemon (llmd).

  On SRX300, SRX320, SRX340, SRX345, and SRX550M devices, llmd runs in Junos OS. On SRX1500, SRX4100, and SRX4200 devices, llmd runs in Linux. So, for supporting llmd to run in both Junos OS and Linux OS, the llmd code directory is moved from the Linux side to the Junos OS side.

• Storing of logs into specified table of the sqlite3 database by llmd— A new syslog daemon is introduced to collect local logs on SRX Series Firewalls and saving them into the database.

  Starting in Junos OS Release 19.3R1, Junos OS stores logs in multiple tables instead of a single table in a database file. Each table contains the timestamp of the oldest and latest logs. When you initiate a query based on the start and end time, llmd finds latest table to generate reports.

  For example, if there are 5 million logs in one table of a database file generated in the last 10 hours, and if you want to take a report, you should spend more than half an hour. From Junos OS Release 19.3R1, one table is separated into multiple tables, and each table has 0.5 million of logs. To generate the same report, one table information is sufficient.

  We recommend you to query with a shorter time for better performance.

  • Database table definition—For session logs, the data types are source-address, destination-address, application, user, and so on. For logs related to security features, the data types are attack-name, URL, profile protocol, and so on. Therefore, different tables are designed to store different types of logs to help improve the performance and save disk space. SRX Series Firewall creates a database table for each log type, when log data is recorded.

    Each type of database table has its maximum record number that is device specific. When the table record number reaches the limitation, new logs replace the oldest logs. Junos OS stores log in an SRX Series Firewall in which active traffic is passed.

    Starting in Junos OS Release 19.3R1, you can create multiple tables in a database file to store logs. You can define the capacity to store logs in a table.

    If the limit of log number exceeds the table capacity, Junos OS stores the logs in the second table. For example, if the limit of logs in table 1 exceeds the table capacity, Junos OS stores logs in table 2.

    If the limit of the log number exceeds the last table of file 1, Junos OS stores the logs in table 1 of file 2. For example, table n is the last table of file 1. When the logs exceed the table capacity, Junos OS stores the logs in table 1 of file 2.

    To make immediate effect after you change the table number, use clear security log report operational command.

  • Database table rotation—Each type of database table has its maximum record number that is device specific. When the table record number reaches the limitation, new logs replace the oldest logs.

    Following Table 3 describes the database file size capacity:

    Table 3: Database File Size Capacity

    Devices	Session	Screen	IDP	Content Security	IPsec-VPN	SKY
    SRX300, SRX320, SRX340, SRX345, and SRX550M	1.8G	0.18G	0.18G	0.18G	0.06G	0.18G
    SRX1500	12G	2.25G	2.25G	2.25G	0.75G	2.25G
    SRX4100 and SRX4200	15G	2.25G	2.25G	2.25G	0.75G	2.25G
    SRX4600	22.5G	6G	6G	6G	0.75G	2.25G
    vSRX Virtual Firewall	1.8G	0.18G	0.18G	0.18G	0.06G	0.18G

• Calculating and displaying the reports that are triggered by CLI—The reports from the database are received from the CLI as the interface. Using the CLI, you can calculate and display the reporting details.

Table Selection

When you want to generate a report from multiple tables, llmd sorts tables based on timestamp and selects tables as per the requested start-time and stop-time.

For example, there are three tables that is table 1 (1 to 3), table 2 (3 to 5) and table 3 (6 to 8). 1 to 3, 3 to 6, and 6 to 8 denotes time stamp of latest and oldest logs. If you request a report from 4 to 6, Junos OS generates report from table 2 and table 3.

Table Lifetime

You can decide table lifetime by configuring set security log report table-lifetime command. Junos OS removes the table after the table identify time exceeds the table lifetime. For example, if you configure table lifetime as 2, and the current date is 26-July-2019, then it means on 24-July-2019 00:00:00 logs are removed.

If you change the date and time manually on a device, the table lifetime changes. For example, if a table identify time is 19-July-2019, and you configure table lifetime as 10, Junos OS should remove the table on 29-July-2019. If you change the device date as 18-July-2019, then the table real lifetime becomes 30-July-2019.

Table Dense Mode

In Junos OS Release 19.4R1, we've upgraded the default storage and search mechanism in the on-box logging database to manage logs. You can now customize log storage and search mechanism results. For example, if you expect fewer traffic logs, you can use the default configuration with a start time and a stop time.

However, if you expect a large number of traffic logs and greater time intervals for which the logs will be generated, then enable dense mode. To enable dense mode, use the set security log report table-mode dense configuration command.

Chassis Cluster Scenario

For on-box reporting in a chassis cluster, the logs are stored in the local disk on which the device is processing active traffic. These logs are not synchronized to the chassis cluster peer.

Each node is responsible to store logs when each node is processing active traffic. In case of active/passive mode, only the active node processes the traffic and logs are also stored only in the active node. In case of failover, the new active node is processes the traffic and stores logs in its local disk. In case of active/active mode, each node processes its own traffic and logs are stored in the respective nodes.