Understanding On-Box Logging and Reporting

 

This topic describes the on-box logging and reporting CLI functionality and the design aspects of on-box reporting for the SRX devices.

Overview

On-box traffic logging to solid-state drives (SSDs) supports eight external log servers or files.

An all-in-one XML file is added that contains all the traffic logs information. The XML file also generates all the logging header files and traffic log related documents.

A new process (daemon) called local log management daemon (llmd) is supported in Services Processing Cards 0 (SPCs0) to handle on-box traffic logging. Traffic produced by flowd in SPCs is listed in traffics logs. The llmd saves these logs to the local SSD. Traffic logs are saved in the following four different formats:

  • syslog

  • sd-syslog

  • welf

  • binary

On-box reporting mechanism is an enhancement to the existing logging functionality. The existing logging functionality is modified to collect system traffic logs, analyzes the logs, and generate reports of these logs in the form of tables using the CLI. On-box reporting feature is intended to provide a simple and easy to use interface for viewing security logs. The on-box reports are easy to use J-Web pages of various security events in the form of tables and graphs. The reports allow the IT security management to identify security information at a glance, and quickly decide the actions to be taken.

Note

The on-box reporting feature is enabled by default when you load the factory-default configurations on the SRX Series device with Junos OS Release 15.1X49-D100 or later.

If you are upgrading your SRX Series device from a Junos OS Release prior to Junos OS 15.1X49-D100, then the SRX device inherits the existing configuration and the on-box reporting feature is disabled by default. You need to configure the set security log report command and the set security log mode stream command to enable the on-box reporting feature on the device that are upgraded.

Note

You must configure security policy for the session using the set security policies from-zone zone-name to-zone zone-name policy policy-name then log session-close command to list all the applications and nested applications in Application Tracking on J-Web using the on-box reporting feature. See for more log (Security Policies) for details.

After the log message is recorded, the log is stored within a log file which is then stored in the database table of the RE for further analysis (on SRX300, SRX320, SRX340, SRX345, and SRX550M devices) or on the SSD card for further analysis (on SRX1500, SRX4100, and SRX4200 devices).

Note

This feature supports receiving top most reports based on count or volume of the session or the type of log, captures events occurring in each second within a specified time range, captures log content for a specified CLI condition. Various CLI conditions like “summary”, top”, “in-detail”, and “in-interval” are used to generate reports. You can generate only one report at one time using the CLI. All the CLI conditions cannot be used at the same time. You can generate only one report at one time using the CLI.

The benefits of this feature are:

  • Reports are stored locally on the SRX Series device and there is no requirement for separate devices or tools for logs and reports storage.

  • The on-box reports are easy-to-use J-Web pages of various security events in the form of tables and graphs.

  • Provides a simple and easy-to-use interface for viewing security logs.

  • The reports generated enables the IT security management team to identify security information at a glance and quickly decide the actions to be taken.

The on-box reporting feature supports:

  • Generating reports based on the requirements. For example: count or volume of the session, types of logs for activities such as IDP, UTM, IPsec VPN.

  • Capturing real-time events within a specified time range.

  • Capturing all the network activities in a logical, organized, and easy-to-understand format based on various CLI specified conditions.

Understanding On-box logging and Reporting

In the on-box reporting mechanism, CLI is used to fetch the reporting data from the device. The SRX series device collects and saves all the required logs. These recorded logs are then used for further analysis to calculate and generate reports in the form of tables using the CLI. The data generated using CLI in the form of reports can be further retrieved in the form of tables and graphs in J-Web. The reports generated are easy-to-understand tables and graphs in J-Web. Thorough analysis of logs is performed (based on session types) for features such as screen, IDP, UTM and IPSec.

You can define filters for the log data that is reported on based on the following criteria:

Note

The top, in-detail, and in-interval conditions cannot be used at the same time.

  • top <number>—This option allow you to generate reports for top security events as specified in the command. for example: top 5 IPS attacks or top 6 URLs detected through UTM.

  • in-detail <number>—This option allow you to generate detail log content.

  • in-interval <time-period>—This option allows you to generate the events logged between certain time intervals.

  • summary—This option allows you to generate the summary of the events. In this way, you can fine-tune the report to your needs, and displays only the data that you want to use.

The maximum in-interval number which shows the count in intervals is 30. If large duration is specified, then the counters are assembled to ensure the maximum in-interval is less than 30.

Both in-detail and summary have the “all” option, since different table have different attribute (like session table does not have the attribute “reason” but UTM has), the “all” option does not have any filter except start-time and stop-time. If there is any other filter other than start time and stop time then an error is displayed.

For example: root@kujang> show security log report in-detail all reason reason1

The application firewall logs for application and user visibility will list applications and nested applications. When the logs of these features list nested applications then nested applications are listed in J-Web. When the logs list nested applications as not-applicable or unknown then only the applications are listed in J-Web.

Use the following CLI commands for application and user visibility for all the applications and nested applications listing:

  • For top nested-application by count—show security log report top session-close top-number <number> group-by application order-by count with user

  • For top nested-application by volume—show security log report top session-close top-number <number> group-by application order-by volume with user

  • For top user by count with nested application—show security log report top session-close top-number <number> group-by user order-by count with application

On-Box Reporting Features

The on-box reporting feature supports:

  • Sqlite3 support as a library—sqlite3 was not supported prior to Junos OS release 15.1X49-D100. Starting with Junos OS Release 15.1X49-D100, an SQL log database (SQLite Version 3) is used by the daemons running on the RE as well as other potential modules to store logs on SRX Series devices.

  • Running llmd in both Junos OS and Linux OS—The forwarding daemon (flowd) decodes database index from binary logs and sends both index and log to the local log management daemon (llmd).

    On SRX300, SRX320, SRX340, SRX345, and SRX550M devices, llmd runs in Junos OS. On SRX1500, SRX4100, and SRX4200 devices, llmd runs in Linux. So, for supporting llmd to run in both Junos OS and Linux OS, the llmd code directory is moved from the Linux side to the Junos OS side.

  • Storing of logs into specified table of the sqlite3 database by llmd— A new syslog daemon is introduced to collect local logs on SRX Series devices and saving them into the database.

    • Database table definition—For session logs, the data types are source-address, destination-address, application, user, and so on. For logs related to security features, the data types are attack-name, URL, profile protocol, and so on. Therefore, different tables are designed to store different types of logs to help improve the performance and save disk space. SRX device creates a database table for each log type, when log data is recorded.

    • Database table rotation—Each type of database table has its maximum record number that is device specific. When the table record number reaches the limitation, new logs replace the oldest logs.

  • Calculating and displaying the reports that are triggered by CLI—The reports from the database are received from the CLI as the interface. Using the CLI, you can calculate and display the reporting details.

Chassis Cluster Scenario

For on-box reporting in a chassis cluster, the logs are stored in the local disk on which the device is processing active traffic. These logs are not synchronized to the chassis cluster peer.

Each node is responsible to store logs when each node is processing active traffic. In case of active/passive mode, only the active node processes the traffic and logs are also stored only in the active node. In case of failover, the new active node is processes the traffic and stores logs in its local disk. In case of active/active mode, each node processes its own traffic and logs are stored in the respective nodes.