Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Navigation
Guide That Contains This Content
[+] Expand All
[-] Collapse All

    Security Director Log Collector Overview

    The Junos Space Security Director Logging and Reporting module enables log collection across multiple SRX Series devices and enables log visualization.

    In Junos Space Security Director 15.2R1, you can set up Log Collectors in a VM environment. From Junos Space Security Director 15.2R2, you can set up Log Collectors in a VM and JA2500 environment. For easy scaling, begin with a single Log Collector and incrementally add dedicated Log Collectors, as your needs expand. You must configure a Log Indexer if you are using more than one Log Collector. In case of VM environment, a single OVA image is used to deploy a Log Collector and Log Indexer. The image presents a configuration script after you log in. During setup, you can configure the node as either a Log Collector or a Log Indexer. At deployment, the user must select appropriate memory and CPU configuration values, as appropriate for the role of the VM.

    Table 1: Log Collector Setup Environment

    Release

    Option

    15.2R1

    VM

    15.2R2 and later releases

    VM, JA2500

    From Security Director Release 16.1R1, you can set up Log Collector on a VM or a JA2500 appliance. You can configure Log Collector as an All-in-One node or integrated node for small-scale deployments. For larger deployments, begin with a single Log Receiver node and Log Storage node, and incrementally add Log Storage nodes as your needs expand. You can have a maximum of one Log Receiver node and three Log Storage nodes.

    You need to set up the Log Collector VM and deploy the Log Collector as an All-in-One node, Log Storage Node, or Log Receiver Node.

    The naming conventions for different node types in various releases are described in Table 2.

    Table 2: Supported Log Collector Node Types

    Node Type in Release 15.2R1

    Node Type in Release 15.2R2

    Node Type in Release 16.1R1 and later releases

    All-in-One node

    All-in-One node

    All-in-One node

    Log Collector node, Log Receiver node

    NA

    Log Receiver node

    Log Data node, Log Indexer node

    NA

    Log Storage node

    Primary-node, Cluster Manager node

    NA

    NA

    Client-node, Log Query node

    NA

    NA

    NA

    NA

    Integrated node

    Note: You can configure eth0 or eth1 for receiving logs from devices in different Log Collector deployment modes.

    Note: In Security Director Release 15.2R2, you can deploy Log Collector as an all-in-one node only, with eps rate of 3k.

    Note: Starting in Junos Space Security Director 16.2R1, you can use JSA as a Log Collector node. See JSA Log Collector Overview and Adding Log Collector to Security Director.

    Note: High Availability is not supported on Security Director Log Collector. However, JSA as Log Collector supports High Availability.

    Note: Security Director Logging and Reporting is not supported on JA1500 appliance.

    Log Director

    Log Director is an application on Junos Space Network Management Platform that gets installed as part of Security Director installation. It is used for system log data collection for SRX and vSRX Series devices running Junos OS. Log Director consists of two components:

    • Junos Space application

    • VM or JA2500 deployment of Log Collector node(s)

    Log Collector Deployment Modes

    Table 3 and Table 4 describe different modes in which Log Collector can be deployed.

    Table 3: Log Collector Deployment Modes for Security Director Release 15.2R1

    Node Type

    Description

    All-in-One Node (Combined deployment)

    Both Receiver and Indexer nodes run on the same VM. It supports eps of up to 2,000 with spinning disks and 4,000 with SSD drives. It is suitable for demos and small-scale deployments.

    Log Receiver Node (Distributed deployment)

    This node receives system logs from SRX Series devices. SRX Series devices must be configured with the Log Receiver node IP to send system logs. Upon configuration, this node parses and forwards logs to Log Indexer node. You must provide the IP address of the Log Indexer node while configuring this node.

    Log Indexer Node (Distributed deployment)

    This node analyzes, indexes, and stores the system logs. It receives the system logs from Log Receiver node and serves all the queries from Security Director. The Log Indexer node roles are split into the following three major roles when the scale of deployment is more than 10K eps:

    • Log Storage node – Dedicated node for storing the indexed system logs.

    • Primary node – Dedicated cluster manager node that monitors and maintains the integrity of Log Indexer cluster.

    • Query node – Dedicated query node that receives system logs from Log Receiver node(s) and distributes them across the available log storage nodes. Also, this node also acts as the single query point for the Security Director application and responds to all the system log queries.

    Note: In Security Director Release 15.2R2, you can deploy Log Collector as an all-in-one node only, with eps rate of 3k. Distributed Log Collector deployment is not supported.

    Table 4: Log Collector Deployment Modes for Security Director Release 16.1 and Later

    Node Type

    Description

    All-in-One Node (Combined deployment)

    Both the Log Receiver and Log Storage nodes run on the same VM or JA2500 appliance. It supports up to 3,000 eps with spinning disks and 4,000 eps with SSD drives. All-in-One node is suitable for demos and small-scale deployments.

    Log Receiver Node (Distributed deployment)

    The Log Receiver node receives system logs from SRX Series devices and vSRX Series devices and forwards them to a Log Storage node. You can configure up to three Log Storage nodes. You must configure the IP address of the Log Receiver Node on SRX and vSRX Series devices and the IP address of the Log Storage nodes on the Log Receiver node.

    Log Storage Node (Distributed deployment)

    This node analyzes, indexes, and stores the system logs. It receives the system logs from Log Receiver node.

    Integrated

    It is similar to an All-in-One node. It is installed on a Junos Space node (JA2500 appliance or virtual appliance) and it works as both the Log Receiver node and Log Storage node.

    Log Collector Storage Requirements

    The total storage required for retaining X number of days at a given events per second (eps) rate is:

    eps * 0.155 * X = Total storage (in GB)

    For example, the storage requirement for 7 days at 500 eps is 500 * 0.155 * 7 = 542 GB, with a +20% margin. The storage space is allocated and equally distributed to the Log Storage nodes.

    Note: The logs get rolled over under the following scenarios:

    • Time-based rollover—Logs that are older than 45 days are automatically rolled over, even if the disk space is available.

    • Disk size-based rollover—Older logs get rolled over when the disk size reaches 400 GB.

    Deploying Log Collector as an All-in-One Node

    An All-in-One node acts both as the Log Receiver and Log Storage node. For a VM environment, a single OVA image is used to deploy the All-in-One, Log Receiver, and Log Storage nodes. The image presents a configuration script after you log in and you must select All-in-One to configure the node. For JA2500 deployments, a single ISO image is used to install the All-in-One, Log Receiver, and Log Storage nodes. During setup, you can configure the node as an All-in-One node.

    Figure 1 shows an example of an All-in-One node deployment.

    Figure 1: All-in-One Node Deployment

    All-in-One Node Deployment

    Deploying Multiple Log Collectors

    If you have a scenario where you require more log reception capacity or events per second, you can add multiple logging nodes. Multiple logging nodes provide higher rates of logging and better query performance. You can add a maximum of one Log Receiver node and three Log Storage nodes.

    For a VM environment, a single OVA image is used to deploy a Log Receiver node and a Log Storage node. The image presents a configuration script after you log in. During setup, you can configure the node as either a Log Receiver or Log Storage node. At deployment, the user must select the memory and CPU configuration values, as appropriate for the VM or JA2500 appliance.

    For JA2500 deployments, a single ISO image is used to install the Log Receiver and Log Storage nodes. During setup, you can configure the node as either a Log Receiver or a Log Storage node.

    Figure 2 shows the deployment example using multiple nodes for up to 10K eps.

    Figure 2: Using Multiple Nodes for Up to 10K eps

    Using Multiple Nodes for Up
to 10K eps

    Figure 3 shows the deployment example using multiple nodes for greater than 10K eps.

    Figure 3: Using Multiple Nodes for Greater Than 10K eps

    Using Multiple Nodes for Greater
Than 10K eps

    Deploying Log Collector as an Integrated Node

    It is installed on a Space node (JA2500 appliance or virtual appliance) and it works as both the Log Receiver node and Log Storage node. You must use the Integrated Log Collector installer for Space application package to install integrated Log Collector on JA2500 appliance or virtual appliance.

    Note: Integrated Log Collector is not a feasible solution in Junos Space high-availability (HA) mode. We recommended you to use All-in-one virtual machine or JSA as a Log Collector for Junos Space HA mode.

    Figure 4: Integrated Node Deployment

    Integrated Node Deployment

    Modified: 2021-05-13