Troubleshooting Amazon AWS Log Source Integrations
You configured a log source in JSA to collect Amazon AWS logs, but the log source status is Warn and events are not generated as expected.
Symptom:
Error that is shown in /var/log/qradar.error:
[ecs-ec] [Amazon AWS S3 REST API Protocol Provider Thread: class com.q1labs.semsources.sources.amazonawsrest.AmazonAWSRESTProvider2 9154] com.q1labs.semsources.sources.amazonawsrest.utils.web.SimpleRESTFileLister: [ERROR] [NOT:0000003000] [x.x.x.x/- -] [-/- -]IOException encountered when trying to list files from remote Amazon S3 bucket. [ecs-ec] [Amazon AWS S3 REST API Protocol Provider Thread: class com.q1labs.semsources.sources.amazonawsrest.AmazonAWSRESTProvider2 9154] javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: Server certificate not recognized [ecs-ec] [Amazon AWS S3 REST API Protocol Provider Thread: class com.q1labs.semsources.sources.amazonawsrest.AmazonAWSRESTProvider2 9154] at com.ibm.jsse2.j.a(j.java:15) [ecs-ec] [Amazon AWS S3 REST API Protocol Provider Thread: class com.q1labs.semsources.sources.amazonawsrest.AmazonAWSRESTProvider2 9154] at com.ibm.jsse2.qc.a(qc.java:728)
Cause:
This error was probably caused by exporting the Amazon SSL certificate from the incorrect URL or by not using the Automatically Acquire Server Certificate(s) option when you configured the log source.
Environment:
All JSA versions.
Diagnosing the problem:
Verify that the certificate that is on the whitelist does not intersect with the server certificate that is provided by the connection. The server certificate that is sent by Amazon covers the *.s3.amazonaws.com domain. You must export the certificate for the following URL:
https://<bucketname>.s3.amazonaws.com
The stack trace in JSA indicates the issue with the Amazon AWS S3 REST API Protocol. In the following example, JSA is rejecting an unrecognized certificate. The most common cause is that the certificate is not in the correct format or is not placed in the correct directory on the correct JSA appliance.
[ecs-ec] [Amazon AWS S3 REST API Protocol Provider Thread: class com.q1labs.semsources.sources.amazonawsrest.AmazonAWSRESTProvider29154] com.q1labs.frameworks.crypto.Q1X509TrustManager: [WARN] [NOT:0000004000][x.x.x.x/- -] [-/- -] Rejecting SSL/TLS connection because server presented unrecognized certificate. The chain sent by the server is [ecs-ec] [Amazon AWS S3 REST API Protocol Provider Thread: class com.q1labs.semsources.sources.amazonawsrest.AmazonAWSRESTProvider29154] com.q1labs.frameworks.crypto.Q1X509TrustManager: [WARN] [NOT:0000004000][x.x.x.x/- -] [-/- -] Subject = CN=*.s3.amazonaws.com, O=Amazon.com Inc., L=Seattle, ST=Washington, C=US [ecs-ec] [Amazon AWS S3 REST API Protocol Provider Thread: class com.q1labs.semsources.sources.amazonawsrest.AmazonAWSRESTProvider29154] com.q1labs.frameworks.crypto.Q1X509TrustManager: [WARN] [NOT:0000004000][x.x.x.x/- -] [-/- -] Subject = CN=q1.us.ibm.com, OU=IBM, O=IBM, L=John, ST=Doe, C=IN, EMAILADDRESS=jdoe@us.ibm.com [ecs-ec] [Amazon AWS S3 REST API Protocol Provider Thread: class com.q1labs.semsources.sources.amazonawsrest.AmazonAWSRESTProvider29154] com.q1labs.frameworks.crypto.Q1X509TrustManager: [WARN] [NOT:0000004000][x.x.x.x/- -] [-/- -]The current certificate white list is: [ecs-ec] [Amazon AWS S3 REST API Protocol Provider Thread: class com.q1labs.semsources.sources.amazonawsrest.AmazonAWSRESTProvider29154] com.q1labs.frameworks.crypto.Q1X509TrustManager: [WARN] [NOT:0000004000][x.x.x.x/- -] [-/- -] Subject = EMAILADDRESS=q1sales@us.ibm.com, O=IBM Corp, L=Waltham, ST=Massachusetts, C=US [ecs-ec] [Amazon AWS S3 REST API Protocol Provider Thread: class com.q1labs.semsources.sources.amazonawsrest.AmazonAWSRESTProvider29154] com.q1labs.frameworks.crypto.Q1X509TrustManager: [WARN] [NOT:0000004000][x.x.x.x/- -] [-/- -] Subject = O=SyslogTLS_Server, CN=* [ecs-ec] [Amazon AWS S3 REST API Protocol Provider Thread: class com.q1labs.semsources.sources.amazonawsrest.AmazonAWSRESTProvider29154] com.q1labs.frameworks.crypto.Q1X509TrustManager: [WARN] [NOT:0000004000][x.x.x.x/- -] [-/- -] Subject = CN=s3-console-us-standard.console.aws.amazon.com, O="Amazon.com, Inc.", L=Seattle, ST=Washington, C=US [ecs-ec] [Amazon AWS S3 REST API Protocol Provider Thread: class com.q1labs.semsources.sources.amazonawsrest.AmazonAWSRESTProvider29154] com.q1labs.frameworks.crypto.Q1X509TrustManager: [WARN] [NOT:0000004000][x.x.x.x/- -] [-/- -] To establish trust in this server certificate, place a copy in /opt/qradar/conf/trusted_certificates
Resolving the problem:
If you downloaded the certificate automatically when you created the log source, verify the following steps:
You configured the correct Amazon S3 endpoint URL and the correct bucket name.
You selected the Yes option for Automatically Acquire server Certificate(s).
You saved the log source.
The log source automatically downloads the .DER certificate file to the /opt/qradar/conf/ trusted_certificates directory. To verify that the correct certificate is downloaded and working, complete the following steps:
From the Navigation menu, click Enable/Disable to disable the log source.
Enable the Amazon AWS CloudTrail log source.
If you manually downloaded the certificate , you must move the .DER certificate file to the correct JSA appliance. The correct JSA appliance is assigned in the Target Event Collector field in the Amazon AWS CouldTrail log source.
The certificate must have a .DER extension. The .DER extension is
case-sensitive and must be in uppercase. If the certificate is exported
in lowercase, then the log source might experience event collection
issues.
Access your AWS CloudTrail S3 bucket at
https://<bucketname>.s3.amazonaws.comUse Firefox to export the SSL certificate from AWS as a DER certificate file.
Copy the DER certificate file to the
/opt/qradar/conf/trusted_certificatesdirectory on the JSA appliance that manages the Amazon AWS CloudTrail log source.Note The JSA appliance that manages the log source is identified by the Target Event Collect field in the Amazon AWS CloudTrail log source. The JSA appliance has a copy of the DER certificate file in the
/opt/qradar/conf/trusted_certificatesfolder.Log in to JSA as an administrator.
Click the Admin tab.
Click the Log Sources icon.
Select the Amazon AWS CloudTrail log source.
From the navigation menu, click Enable/Disable to disable, then re-enable the Amazon AWS CloudTrail log source.
Note Forcing the log source from disabled to enabled connects the protocol to the Amazon AWS bucket as defined in the log source. A certificate check takes place as part of the first communication.
If you continue to have issues, verify that the Amazon AWS bucket name in the Log Source Identifier field is correct. Ensure that the Remote Directory path is correct in the log source configuration.