DSM Editor Overview
Instead of manually creating a log source extension to fix parsing issues or extend support for new log source types, use the DSM Editor. The DSM Editor provides different views of your data. You use the DSM Editor to extract fields, define custom properties, categorize events, and define new QID definition.
The DSM Editor provides the following views:
The Workspace shows you raw event data. Use sample event payloads to test the behavior of the log source type, and then the Workspace area shows you the data that you capture in real time.
All sample events are sent from the workspace to the DSM simulator, where properties are parsed and QID maps are looked up. The results are displayed in the Log Activity Preview section. Click the pencil icon to open in edit mode.
In the edit mode, you paste up to 100,000 characters of event data into the workspace or edit data directly. When you edit properties on the Properties tab, matches in the payload are highlighted in the workspace. Custom properties and overridden system properties are also highlighted in the Workspace.
Log Activity Preview
The Log Activity Preview simulates how the payloads in the workspace appear in the Log Activity viewer. Every standard property that is supported is displayed. The fields that are marked with an asterisk (*), for example, Event name, Severity, Low-level category, and QID, are populated from the QID map. Fields that are populated from the QID map cannot be parsed verbatim from the raw events data in the workspace, so they cannot be defined or edited. However, you can adjust their values by selecting the corresponding event ID and category combination from the Event Mappings tab.
Click the wrench icon to select which columns to show or to hide in the Log Activity Preview window, and to reorder the columns.
The properties tab contains the combined set of system and custom properties that constitute a DSM configuration. Configuring a system property differs from configuring a custom property. You can override a property, by selecting the Override system behaviour check box and entering the regex or JSON expression.
If you override the Event Category property, you must also override the Event ID property.
Matches in the payload are highlighted in the event data in the workspace. The highlighting color is two-toned, depending on what you capture. For example, the orange highlighting represents the capture group value while the bright yellow highlighting represents the rest of the regex that you specified. The feedback in the workspace shows whether you have the correct regex. If an expression is in focus, the highlighting in the workspace reflects only what that expression can match. If the overall property is in focus, then the highlighting turns green and shows what the aggregate set of expressions can match, taking into account the order of precedence.
In the format strings field, capture groups are represented by using $<number> notation. For example, $1 represents the first capture group from the regex, $2 is the second capture group, and so on.
You can add multiple expressions to the same property, and you can assign precedence by dragging and dropping the expressions to the top of the list.
A warning tool tip beside any of the properties indicates that no expression was added.
Event Mappings Tab
The Event Mappings tab displays all the event ID and category combinations that exist in the system. If a new event mapping is created, it is added to the list of event ID and category combination that is displayed in the Event Mappings tab. In general, the Event Mappings tab displays all event ID and category combinations and the QID records that they are mapped to.
You can configure Auto Property Discovery for structured data that are in JSON format. By default, log source types have Auto Property Discovery turned off.You can configure Auto Property Discovery for structured data that are in JSON format. By default, log source types have Auto Property Discovery turned off.
When you enable Auto Property Discovery on the Configuration tab, the property discovery engine automatically generates new properties to capture all fields that are present in the events that are received by a log source type. You can configure the number of consecutive events to be inspected for new properties in the Discovery Completion Threshold field. Newly discovered properties appear in the Properties tab, and are made available for use in the rules and search indexes. However, if no new properties are discovered before the threshold, the discovery process is considered complete and Auto Property Discovery for that log source type is disabled. You can manually enable the Auto Property Discovery on the Configuration tab at any time.
To continuously inspect events for a log source type, you must make sure that you set the Discovery Completion Threshold value to 0.