Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Security Director Log Collector Overview

 

The Junos Space Security Director Logging and Reporting module enables log collection across multiple SRX Series devices and enables log visualization.

In Junos Space Security Director 15.2R1, you can set up Log Collectors in a VM environment. From Junos Space Security Director 15.2R2, you can set up Log Collectors in a VM and JA2500 environment. For easy scaling, begin with a single Log Collector and incrementally add dedicated Log Collectors, as your needs expand. You must configure a Log Indexer if you are using more than one Log Collector. In case of VM environment, a single OVA image is used to deploy a Log Collector and Log Indexer. The image presents a configuration script after you log in. During setup, you can configure the node as either a Log Collector or a Log Indexer. At deployment, the user must select appropriate memory and CPU configuration values, as appropriate for the role of the VM.

Table 1: Log Collector Setup Environment

Release

Option

15.2R1

VM

15.2R2 and later releases

VM, JA2500

From Security Director Release 16.1R1, you can set up Log Collector on a VM or a JA2500 appliance. You can configure Log Collector as an All-in-One node or integrated node for small-scale deployments. For larger deployments, begin with a single Log Receiver node and Log Storage node, and incrementally add Log Storage nodes as your needs expand. You can have a maximum of one Log Receiver node and three Log Storage nodes.

You need to set up the Log Collector VM and deploy the Log Collector as an All-in-One node, Log Storage Node, or Log Receiver Node.

The naming conventions for different node types in various releases are described in Table 2.

Table 2: Supported Log Collector Node Types

Node Type in Release 15.2R1

Node Type in Release 15.2R2

Node Type in Release 16.1R1 and later releases

All-in-One node

All-in-One node

All-in-One node

Log Collector node, Log Receiver node

NA

Log Receiver node

Log Data node, Log Indexer node

NA

Log Storage node

Primary-node, Cluster Manager node

NA

NA

Client-node, Log Query node

NA

NA

NA

NA

Integrated node

Note

You can configure eth0 or eth1 for receiving logs from devices in different Log Collector deployment modes.

Note

In Security Director Release 15.2R2, you can deploy Log Collector as an all-in-one node only, with eps rate of 3k.

Note

Starting in Junos Space Security Director 16.2R1, you can use JSA as a Log Collector node. See JSA Log Collector Overview and Adding Log Collector to Security Director.

Note

High Availability is not supported on Security Director Log Collector. However, JSA as Log Collector supports High Availability.

Note

Security Director Logging and Reporting is not supported on JA1500 appliance.

Log Director

Log Director is an application on Junos Space Network Management Platform that gets installed as part of Security Director installation. It is used for system log data collection for SRX and vSRX Series devices running Junos OS. Log Director consists of two components:

  • Junos Space application

  • VM or JA2500 deployment of Log Collector node(s)

Log Collector Deployment Modes

Table 3 and Table 4 describe different modes in which Log Collector can be deployed.

Table 3: Log Collector Deployment Modes for Security Director Release 15.2R1

Node Type

Description

All-in-One Node (Combined deployment)

Both Receiver and Indexer nodes run on the same VM. It supports eps of up to 2,000 with spinning disks and 4,000 with SSD drives. It is suitable for demos and small-scale deployments.

Log Receiver Node (Distributed deployment)

This node receives system logs from SRX Series devices. SRX Series devices must be configured with the Log Receiver node IP to send system logs. Upon configuration, this node parses and forwards logs to Log Indexer node. You must provide the IP address of the Log Indexer node while configuring this node.

Log Indexer Node (Distributed deployment)

This node analyzes, indexes, and stores the system logs. It receives the system logs from Log Receiver node and serves all the queries from Security Director. The Log Indexer node roles are split into the following three major roles when the scale of deployment is more than 10K eps:

  • Log Storage node – Dedicated node for storing the indexed system logs.

  • Primary node – Dedicated cluster manager node that monitors and maintains the integrity of Log Indexer cluster.

  • Query node – Dedicated query node that receives system logs from Log Receiver node(s) and distributes them across the available log storage nodes. Also, this node also acts as the single query point for the Security Director application and responds to all the system log queries.

Note

In Security Director Release 15.2R2, you can deploy Log Collector as an all-in-one node only, with eps rate of 3k. Distributed Log Collector deployment is not supported.

Table 4: Log Collector Deployment Modes for Security Director Release 16.1 and Later

Node Type

Description

All-in-One Node (Combined deployment)

Both the Log Receiver and Log Storage nodes run on the same VM or JA2500 appliance. It supports up to 3,000 eps with spinning disks and 4,000 eps with SSD drives. All-in-One node is suitable for demos and small-scale deployments.

Log Receiver Node (Distributed deployment)

The Log Receiver node receives system logs from SRX Series devices and vSRX Series devices and forwards them to a Log Storage node. You can configure up to three Log Storage nodes. You must configure the IP address of the Log Receiver Node on SRX and vSRX Series devices and the IP address of the Log Storage nodes on the Log Receiver node.

Log Storage Node (Distributed deployment)

This node analyzes, indexes, and stores the system logs. It receives the system logs from Log Receiver node.

Integrated

It is similar to an All-in-One node. It is installed on a Junos Space node (JA2500 appliance or virtual appliance) and it works as both the Log Receiver node and Log Storage node.

Log Collector Storage Requirements

The total storage required for retaining X number of days at a given events per second (eps) rate is:

eps * 0.155 * X = Total storage (in GB)

For example, the storage requirement for 7 days at 500 eps is 500 * 0.155 * 7 = 542 GB, with a +20% margin. The storage space is allocated and equally distributed to the Log Storage nodes.

Note

The logs get rolled over under the following scenarios:

  • Time-based rollover—Logs that are older than 45 days are automatically rolled over, even if the disk space is available.

  • Disk size-based rollover—Older logs get rolled over when the disk size reaches 400 GB.

Deploying Log Collector as an All-in-One Node

An All-in-One node acts both as the Log Receiver and Log Storage node. For a VM environment, a single OVA image is used to deploy the All-in-One, Log Receiver, and Log Storage nodes. The image presents a configuration script after you log in and you must select All-in-One to configure the node. For JA2500 deployments, a single ISO image is used to install the All-in-One, Log Receiver, and Log Storage nodes. During setup, you can configure the node as an All-in-One node.

Figure 1 shows an example of an All-in-One node deployment.

Figure 1: All-in-One Node Deployment
All-in-One Node Deployment

Deploying Multiple Log Collectors

If you have a scenario where you require more log reception capacity or events per second, you can add multiple logging nodes. Multiple logging nodes provide higher rates of logging and better query performance. You can add a maximum of one Log Receiver node and three Log Storage nodes.

For a VM environment, a single OVA image is used to deploy a Log Receiver node and a Log Storage node. The image presents a configuration script after you log in. During setup, you can configure the node as either a Log Receiver or Log Storage node. At deployment, the user must select the memory and CPU configuration values, as appropriate for the VM or JA2500 appliance.

For JA2500 deployments, a single ISO image is used to install the Log Receiver and Log Storage nodes. During setup, you can configure the node as either a Log Receiver or a Log Storage node.

Figure 2 shows the deployment example using multiple nodes for up to 10K eps.

Figure 2: Using Multiple Nodes for Up to 10K eps
Using Multiple Nodes for Up
to 10K eps

Figure 3 shows the deployment example using multiple nodes for greater than 10K eps.

Figure 3: Using Multiple Nodes for Greater Than 10K eps
Using Multiple Nodes for Greater
Than 10K eps

Deploying Log Collector as an Integrated Node

It is installed on a Space node (JA2500 appliance or virtual appliance) and it works as both the Log Receiver node and Log Storage node. You must use the Integrated Log Collector installer for Space application package to install integrated Log Collector on JA2500 appliance or virtual appliance.

Note

Integrated Log Collector is not a feasible solution in Junos Space high-availability (HA) mode. We recommended you to use All-in-one virtual machine or JSA as a Log Collector for Junos Space HA mode.

Figure 4: Integrated Node Deployment
Integrated Node Deployment