Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
 

Gateway Log Source

Use a gateway log source to configure a protocol to use many Device Support Modules (DSMs) instead of relying on a single DSM type. With a gateway log source, event aggregator protocols can dynamically handle various event types.

Before you configure your gateway log source, you must understand the difference between protocols, DSMs, and log sources.

Protocol

Protocols provide the capability of collecting a set of data files by using various connection options. These connections pull the data back, or passively receive data, into the event pipeline in JSA. Then, the corresponding DSM parses and normalizes the data.

DSM

A DSM is a code module that parses received events from multiple log sources and converts them to a standard taxonomy format that can be displayed as output. Each type of log source has a corresponding DSM.

Log source

A log source is a data source that creates an event log. For more information, see Introduction to Log Source Management.

Gateway log sources support the following protocols:

  • Amazon AWS S3 REST

  • Amazon AWS Web Services

  • Google Cloud Pub Sub

  • Kafka

  • Microsoft Azure Event Hubs

  • TCPMutilineSyslog

  • TLS Syslog

  • UDPMutilineSyslog

Tip:

To provide the best fit for the generic data, use the Universal DSM when you configure your gateway log source.

A gateway log source does not use a DSM. It delegates the DSM parsing to stand-alone Syslog log sources that have an appropriate identifier and DSM. These log sources are a collector log source (the gateway) and a parser log source. Parser log sources match data that comes in from the gateway and do not actively collect the events themselves.

Before you create your gateway log source, you must know what types of data you expect to collect from the data gateway. Data gateways can collect many data types and JSA does not support all data types by default. To parse the data correctly, a DSM must exist that can handle the events that you are collecting. Even if JSA supports the event’s source, if the gateway returns it in an unexpected format, the DSM might not parse it. For example, if the data gateway returns an event in a JSON format, but the DSM expects a LEEF format, you might need a custom DSM to parse the data.

A gateway log source works in the same way as other log sources by using its selected protocol to reach out and collect events. The difference between a gateway log source and other log sources occurs when the collected events are ready to be posted. A normal log source attempts to force the events to be parsed by the selected DSM. A gateway log source sends the events as a Syslog payload with a default identifier set to either 0.0.0.0 or to the connected Services IP address.

When an event is posted to the event pipeline as a Syslog payload, the events are handled by log source auto detection. If an existing dummy log source with the provided identifier exists, the event is handled by that log source, regardless of whether the event parses with that DSM. If no existing dummy log source exists, the event is parsed by DSMs that support auto detection. If the event correctly parses with a DSM, then it updates the identifier to “IP or Host @ DSM” and creates a log source.

Log sources that are automatically created do not have their identifier set by the protocol. These log sources identifiers are in the “IP or Host @ DSM Type” format. To match to an automatically created dummy log source, the Syslog payload must have an identifier that is the same IP or Host, and the selected DSM must be able to parse it. The default identifier is sent as [IP or Host], not “IP or Host @ DSM Type”. For the identifier to be updated with the DSM Type, it must parse with that DSM Type. If you use the default settings, events that cannot be parsed are sent directly to sim-generic.

Tip:

Manually created log sources that have an identifier that matches the identifier of the Syslog payload are used even if the DSM of the log source fails to parse the event.

To configure a gateway log source, enable the Use As A Gateway Log Source option for the selected protocol. If you enable this option, the events are sent to the event pipeline and are autodetected. To get the maximum value from this feature, use the Log Source Identifier Pattern.