Data Nodes

A data node is an appliance that you can add to your event and flow processors to increase storage capacity and improve search performance. You can add an unlimited number of data nodes to your JSA deployment, and they can be added at any time. Each data node can be connected to only one processor, but a processor can support multiple data nodes.

For more information about planning your deployment, see the Juniper Secure Analytics Architecture and Deployment Guide.

Data Rebalancing After a Data Node is Added

When you add a data node, JSA rebalances the data to improve search and overall system performance.

Data rebalancing includes decompressing older data, and moving data that was on the original storage device to evenly distribute it across all connected devices.

For example, your deployment has an event processor that receives 20,000 events per second (EPS). When you add data nodes, JSA automatically distributes the events across the event processor and all data nodes that are available to it. If you add three data nodes, the event processor stores 5,000 EPS and sends 5,000 EPS to each of the attached data nodes. The event processor is still processing all of the events, but the data nodes provide more storage, indexing, and search capabilities to improve the overall performance.

Data That is Collected in JSA 2014.6 and Earlier

By default, data that is collected with JSA 2014.6 and earlier versions is not compressed. Data compression occurs only when JSA detects that the available storage on a device is less than the available storage threshold. The disk maintenance process compresses the data into gzip format, and applies the data deletion policies until the volume of data is back within the threshold amount.

When you add a data node, JSA rebalances the data, and when enough storage space is available, data that is in gzip format is decompressed. The search performance on the older data immediately improves, and performance continues to improve because most searches use more recent data that is uncompressed.

Because the pre-2014.7 data is now decompressed, the disk volumes might quickly exceed the threshold for free storage space. When the available disk space falls beneath the threshold settings, the disk maintenance process compresses any data that is eligible for compression, according to the retention policies, until the free storage threshold is reached.

Data That is Collected in JSA 2014.7 and Later

In JSA 2014.7, all new data is written to disk in compressed format. The disk maintenance processes do not compress new data.

The data deletion policy is not affected by the data format. When JSA exceeds the available storage threshold, the disk maintenance processes delete data in both the old and new format according to the data retention settings.

Viewing the Progress Of Data Rebalancing

When you add a data node, JSA automatically redistributes the data to balance it across the storage volumes in your deployment.

Search performance improvements are realized only after the data rebalancing is complete. You can view the progress of the data rebalancing, and also see data such as the percentage of disk space that is used.

1. On the navigation menu (), click Admin to open the admin tab.
2. In the System Configuration section, click System and License Management.
3. In the Display list, select Systems.
4. In the host table, select the managed host that you want to view more information about.

   • To view information about the cluster of managed hosts, select the top-level host.

   • To view information about a specific data node, select the data node.

5. On the Actions menu, click View and manage system.
6. Click the Security Data Distribution tab to view the progress of data rebalancing and the capacity of the Data Node appliance.Note

   You can also view information about the progress of data node rebalancing in the deployment status bar on the Admin tab.

Saving All Event Data to a Data Node Appliance

To improve the performance of an event processor, configure JSA to save all event data on a Data Node appliance. With this configuration, the event processor only processes events; it doesn't store event data locally.

An event processor that is configured to only process events still saves event data locally when no active Data Node appliances are available. When a Data Node appliance becomes available, JSA transfers as much data as possible from the event processor to the Data Node.

1. On the navigation menu (), click Admin to open the admin tab.
2. In the System Configuration section, click System and License Management.
3. In the Display list, select Systems.
4. Select the Event Processor from the host table, and on the Deployment Actions menu, click Edit Host.
5. Click the Component Management settings icon ().
6. Under Event Processor, in the Event Processor Mode field, select Processing-Only.
7. Click Save, and then click Save again.
8. On the Admin tab, click Deploy Changes.

Archiving Data Node Content

Configure a Data Node appliance to use Archive mode when you want the Data Node to provide online access to historical data without impacting storage for incoming data.

In Archive mode, the appliance does not receive new data, but existing data is saved.

1. On the navigation menu (), click Admin to open the admin tab.
2. In the System Configuration section, click System and License Management.
3. In the Display list, select Systems.
4. Select the Data Node appliance in the host table, and on the Deployment Actions menu, click Edit Host.
5. Click the Component Management settings icon ().
6. In the Data Node Mode field, select Archive, and then click Save.
7. On the Admin tab, click Deploy Changes.

To resume storing data on the Data Node appliance, set the mode back to Active.