Expanding Deployments to Add More Capacity
Your business might create or expand a deployment beyond an JSA All-in-One appliance because of the lack of processing or data storage capacity, or when you have specific data collection requirements.
The topology and composition of your JSA deployment are influenced by the capability and capacity of that deployment to collect, process, and store all the data that you want to analyze in your network.
To get rough estimates of the events per second (EPS) or flows per minute (FPM) that you need to process in your deployment, use the size of your logs that are collected from firewalls, proxy servers, and Windows boxes.
Reasons to Add Event or Flow Processors to an All-in-One Deployment
You might need to add flow or event collectors to your deployment under these conditions:
Your data collection requirements exceed the collection capability of the All-in-One appliance.
You must collect events and flows in a different location than where your All-in-One appliance is installed.
You are monitoring larger, or higher-rate packet-based flow sources that are faster than the 50 Mbps connection on the All-in-One.
An All-in-One appliance can collect up to 15,000 events per second (EPS) and 300,000 flows per minute (FPM). If your collection requirements are greater, you might want to add event collectors and flow processors to your deployment.
An All-in-One appliance processes the events and flows that are collected. By adding event collectors and flow processors, you can use the processing that the All-in-One appliance usually does for searches and other security tasks.
Packet-based flow sources require a flow processor that is connected either to a Flow Processor, or to an All-in-One appliance in deployments where there is no Flow Processor appliance. You can collect external flow sources, such as NetFlow, or IPFIX, directly on a Flow Processor or All-in-One appliance.
Adding Remote Collectors to a Deployment
Add JSA event collectors or JSA flow processors to expand a deployment when you need to collect more events locally and collect events and flows from a remote location.
For example, you are a manufacturing company that has a JSA All-in-One deployment and you add e-commerce and a remote sales office. You now must monitor for security threats and are also now subject to PCI audits.
You hire more employees and the Internet usage changes from mostly downloading to two-way traffic between your employees and the Internet. Here are details about your company.
The current events per second (EPS) license is 1000 EPS.
You want to collect events and flows at the sales office and events from the e-commerce platform.
Event collection from the e-commerce platform requires up to 2000 events-per-second (EPS).
Event collection from the remote sales office requires up to 2000 events-per-second (EPS).
The flows per minute (FPM) license is sufficient to collect flows at the remote office.
You take the following actions:
You add the e-commerce platform at your head office, and then you open a remote sales office.
You install an Event Collector and a flow processor at the remote sales office that sends data over the Internet to the All-in-One appliance at your head office.
You upgrade your EPS license from 1000 EPS to 5000 EPS to meet the requirements for the extra events that are collected at the remote office.
The following diagram shows an example deployment of when an Event Collector and a flow processor are added at a remote office.
In this deployment, the following processes occur:
At your remote office, the Event Collector collects data from log sources and the flow processor collects data from routers and switches. The collectors coalesce and normalize the data.
The collectors compress and send data to the All-in-One appliance over the wide area network.
The All-in-One appliance processes, and stores the data.
Your company monitors network activity by using the JSA web application for searches, analysis, reporting, and for managing alerts and offenses.
The All-in-one collects and processes events from the local network.
Adding Processing Capacity to an All-in-One Deployment
Add Event Processors and Flow Processors to your JSA deployment to increase processing capacity and increase storage. Adding processors frees up resources on your JSA Console by moving the processing and storage load to dedicated servers.
When you add Event Processors or Flow Processors to an All-in-One appliance the All-in-One acts as a JSA Console. The processing power on the All-in-One appliance is dedicated to managing and searching the data that is sent by the processors, and data is now stored on the Event Processors and other storage devices, rather than on the Console.
You typically add Event Processors and Flow Processors to your JSA deployment for the following reasons:
As your deployment grows, the workload exceeds the processing capacity of the All-in-One appliance.
Your security operations center employs more analysts who do more concurrent searches.
The types of monitored data, and the retention period for that data increases, which increases processing and storage requirements.
As your security analyst team grows, you require better search performance.
Running multiple concurrent JSA searches and adding more types of log sources that you monitor, affects the processing performance of your All-in-One appliance. As you increase the number of searches and the amount of monitored data, add Event Processors and Flow Processors to improve the performance of your JSA deployment.
When you scale your JSA deployment beyond the 15,000 EPS and 300,000 FPM on the most powerful All-in-One appliance, you must add processor appliances to process that data.
Example: Adding a JSA Event Processor to your deployment
You can add a JSA Event Processor 1624, which collects and processes up to 40,000 EPS. You increase your capacity by another 40,000 EPS every time you add a JSA Event Processor 1624 to your deployment. Add a JSA Flow Processor 1724, which collects and processes up to 1,200,000 FPM.
The JSA Event Processor 1624 is a collector and a processor. If you have a distributed network, it’s a good practice to add Event Collectors to distribute the load and to free system resources on the Event Processor.
In the following diagram, processing capacity is added when an Event Processor and a Flow Processor are added to a JSA appliance (All-in-One), and the following changes take place:
Event and flow processing is moved off the All-in-One appliance to the event and flow processors.
Event processing capacity increases to 40,000 EPS, which includes the 15,000 EPS that was on the All-in-One.
Flow processing capacity increases to 1,200,000 FPM, which includes the 300,000 FPM that was on the All-in-One.
Data that is sent by the event and flow processor is processed and stored on the event and flow processors.
Search performance is faster when you install Event Processors and Flow Processors on the same network as your JSA Console.
Adding processors and collectors expands the processing capacity of your JSA deployment. You can also increase the storage capacity of your deployment. Your company's data retention needs can increase due to more traffic or to changes to retention policies. Adding Data Nodes to your deployment expands your data storage capacity, and improves search performance.
When to add Collectors to Processors
Add Event Collectors and Flow Processors to Event Processors for the same reasons that you add collectors to an All-in-One appliance:
Your data collection requirements exceed the collection capability of your processor.
You must collect events and flows at a different location than where your processor is installed.
You are monitoring packet-based flow sources.
Event Collectors can buffer events, but Flow Processors can't buffer flows.
Because search performance is improved when processors are installed on the same network as the console, adding collectors in remote locations, and then sending that data to the processor, speeds up your JSA searches.