Amazon VPC Flow Logs

Amazon VPC Flow Logs

The JSA integration for Amazon VPC (Virtual Private Cloud) Flow Logs collects VPC flow logs from an Amazon S3 bucket by using an SQS queue.

To integrate Amazon VPC Flow Logs with JSA, complete the following steps:

1. If automatic updates are not enabled, download and install the most recent version of the Amazon VPC Flow Logs DSM RPM from the https://support.juniper.net/support/downloads/ onto your JSA console.

   • Protocol Common RPM

   • AWS S3 REST API PROTOCOL RPM

   Note:

   If you are installing the RPM to enable additional AWS-related VPC flow fields in the QRadar Network Activity Flow Details window, then the following services must be restarted before they are visible. You don't have to restart the services for the protocol to function.

   • hostcontext

     To restart hostcontext, see QRadar: Hostcontext service and the impact of a service restart.

   • tomcat

     On the Console, click the Admin tab, and then click Advanced > Restart Web Server.

2. Configure your Amazon VPC Flow Logs to publish the flow logs to an S3 bucket.

3. Create the SQS queue that is used to receive ObjectCreated notifications from the S3 bucket that you used in step 2.

4. Create security credentials for your AWS user account.

5. Add an Amazon VPC Flow Logs log source on the JSA Console.

   Note:

   A Flow Processor must be available and licensed to receive the flow logs. Unlike other log sources, AWS VPC Flow Log events are not sent to Log Activity tab. They are sent to Network Activity tab.

   The following table describes the parameters that require specific values to collect events from Amazon VPC Flow Logs:

   Table 1: Amazon VPC Flow Logs log source parameters

   Parameter	Value
   Log Source type	A custom log source type
   Protocol Configuration	Amazon AWS S3 REST API
   Target Event Collector	The Event Collector or Event Processor that receives and parses the events from this log source. Note: This integration collects events about Amazon VPC Flow Logs. It does not collect flows. You cannot use a Flow Collector or Flow Processor as the target event collector.
   Log Source Identifier	Type a unique name for the log source. The Log Source Identifier can be any valid value and does not need to reference a specific server. The Log Source Identifier can be the same value as the Log Source Name. If you configured more than one Amazon VPC flow Logs log source, you might want to name in an identifiable way. For example, you can identify the first log source as vpcflowlogs1 and the second log source as vpcflowlogs2.
   Authentication Method	Access Key ID / Secret Key Standard authentication that can be used from anywhere. For more information, see Configuring Security Credentials for your AWS User Account. EC2 Instance IAM Role If your managed host is running on an AWS EC2 instance, choosing this option uses the IAM Role from the instance metadata assigned to the instance for authentication. No keys are needed. This method works only for managed hosts that are running within an AWS EC2 container.
   Assume IAM Role	Enable this option by authenticating with an Access Key or EC2 instance IAM Role. Then, you can temporarily assume an IAM Role for access. This option is available only when you use the SQS Event Notifications collection method. For more information about creating IAM users and assigning roles, see Creating an Identity and Access Management (IAM) user in the AWS Management Console.
   Event Format	AWS VPC Flow Logs
   S3 Collection Method	SQS Event Notifications
   VPC Flow Destination Hostname	The hostname or IP address of the Flow Processor where you want to send the VPC logs. Note: For JSA to accept IPFIX flow traffic, you must configure a NetFlow/IPFIX flow source that uses UDP. Most deployments can use a default_Netflow flow source and set the VPC Flow Destination Hostname to the hostname of that managed host. If the managed host configured with the NetFlow/IPFIX flow source is the same as the Target Event Collector that was chosen earlier in the configuration, you can set the VPC Flow Destination Hostname to localhost.
   VPC Flow Destination Port	The port for the Flow Processor where you want to send the VPC logs. Note: This port must be the same as the monitoring port that is specified in the NetFlow flow source. The port for the default_Netflow flow source is 2055
   SQS Queue URL	The full URL that begins with https://, for the SQS Queue that is set up to receive notifications for ObjectCreated events from S3.
   Region Name	The region that is associated with the SQS queue and S3 bucket. Example: us-east-1, eu-west-1, ap-northeast-3
   Show Advanced Options	The default is No. Select Yes if you want to customize the event data.
   File Pattern	This option is available when you set Show Advanced Options to Yes. Type a regex for the file pattern that matches the files that you want to pull; for example, .*? \.json\.gz
   Local Directory	This option is available when you set Show Advanced Options to Yes. The local directory on the Target Event Collector. The directory must exist before the AWS S3 REST API PROTOCOL attempts to retrieve events.
   S3 Endpoint URL	This option is available when you set Show Advanced Options to Yes. The endpoint URL that is used to query the AWS REST API. If your endpoint URL is different from the default, type your endpoint URL. The default is http://s3.amazonaws.com.
   Use Proxy	If JSA accesses the Amazon Web Service by using a proxy, enable Use Proxy. If the proxy requires authentication, configure the Proxy Server, Proxy Port, Proxy Username, and Proxy Password fields. If the proxy does not require authentication, configure the Proxy Server and Proxy Port fields.
   Recurrence	How often the Amazon AWS S3 REST API Protocol connects to the Amazon cloud API, checks for new files, and if they exist, retrieves them. Every access to an AWS S3 bucket incurs a cost to the account that owns the bucket. Therefore, a smaller recurrence value increases the cost. Type a time interval to determine how frequently the remote directory is scanned for new event log files. The minimum value is 1 minute. The time interval can include values in hours (H), minutes (M), or days (D). For example, 2H = 2 hours, 15 M = 15 minutes.
   EPS Throttle	The maximum number of events per second that are sent to the flow pipeline. The default is 5000. Ensure that the EPS Throttle value is higher than the incoming rate or data processing might fall behind.

6. To send VPC flow logs to the JSA Cloud Visibility app for visualization, complete the following steps:

   1. On the Console, click the Admin tab, and then click System Configuration > System Settings.

   2. Click the Flow Processor Settings menu, and in the IPFix additional field encoding field, choose either the TLV or TLV and Payload format.

   3. Click Save.

   4. From the menu bar on the Admin tab, click Deploy Full Configuration and confirm your changes.

      Warning:

      When you deploy the full configuration, JSA services are restarted. During this time, events and flows are not collected, and offenses are not generated.

   5. Refresh your browser.