Custom Property Definitions in the DSM Editor
You can define a custom property and reuse the same property in a separate DSM. Use these properties in searches, rules, and to allow specific user-defined behavior for parsing values into those fields.
Where relevant, each custom property has a set of configuration options that includes selectivity and data parsing. Each custom property definition within a DSM configuration is an ordered group of expressions that consists of regular expressions, a capture group, an optional selectivity configuration, and an enabled or disabled toggle button. You can't modify the Name, Field type, Description, optimize fields, or any advanced options for a custom property on the Properties tab in the DSM Editor.
A custom property is shared across all DSMs, while specific implementations for reading values from payloads are at the DSM level.
Selectivity is specified when you configure an expression to run only when certain conditions are met.
The Capture Group field of a custom property cannot be assigned a value greater than the number of capture groups in the regex.
In the DSM Editor, you can restrict running a custom property to certain criteria for better performance.
The following are the types of restrictions:
By high-level category and low-level category -- A property is evaluated only when the high-level and low-level categories match a specific combination. For example, a property is evaluated only when the event is known to have a high-level category of Authentication and a low-level category of Admin Logout.
By specific QID -- A property is evaluated only when the event that is seen maps to a specific QID. For example, when the event maps to a QID of Login Failed, the property is evaluated.
You can define expressions for custom properties in the DSM Editor. Expressions are the mechanism that defines the behavior of a property. The main component of an expression is a valid regex. The data that makes up an expression depends on the property type.
For a custom property, you can choose only one capture group from the regex.
Creating a Custom Property
In the DSM Editor, you can define a custom property for one or more log sources, whose events do not fit into the JSA normalized event model. For example, a system property might fail to capture data from some applications, operation systems, databases, and other systems.
You can create custom property for data that does not fit into JSA system properties. Use the custom properties in searches and test against them in rules.
Table 1: Custom Property Parameters
A descriptive name for the custom property that you create.
The default is Text.
Note: When you select Number or Date from the Field Type list, extra fields are displayed.
Enable this Property for use in Rules and Search Indexing
When enabled, during the parsing stage of the event pipeline, JSA attempts to extract the property from events immediately as they enter the system. Other components downstream in the pipeline such as rules, forwarding profiles and indexing can use the extracted values. Property information is persisted along with the rest of the event record and doesn't need to be extracted again when it is retrieved as part of a search or report. This option enhances performance when the property is retrieved, but can have a negative impact on performance during the event parsing process, and impacts storage.
When not enabled, JSA extracts the property from the events only when they are retrieved or viewed.
Note: For Custom Properties to be used in rule tests, forwarding profiles, or for search indexing, this check box must be selected because rule evaluation, event forwarding, and indexing occur before events are written to disk, so the values must be extracted at the parsing stage.
Use number format from a Locale
This field displays when you select Number from the Field Type list. If you select the Use number format from a Locale check box, you must select an Extracted Number Format from the list.
Extracted Date/Time Format
This field displays when you select Date from the Field Type list. You must provide a datetime pattern that matches how the datetime appears in the original event.
For example, 'MMM dd YYYY HH:mm:ss' is a valid datetime pattern for a time stamp like 'Apr 17 2017 11:29:00'.
This field displays when you select Date from the Field Type list. You must select the locale of the event.
For example, if the locale is English, it will recognizes 'Apr' as a short form of the month 'April'. But if the event is presented in French and the month token is 'Avr' (for Avril), then set the locale to a French one, or the code does not recognize it as a valid date.
- To add a custom property, click the Add (+) icon on the Properties tab in the DSM Editor.
- To create a new custom property definition, use the following
Select Create New on the Choose a Custom Property Definition to Express page.
On the Create a new Custom Property Definition page, enter values for the name, field type and description fields.
When you select Number or Date from the Field Type list, extra fields display.
If you want to extract the property from events as they enter the system, select the Enable this property for use in Rules and Search indexing check box.
- To use an existing custom property, use the following
On the Choose a Custom Property Definition to Express page, search for an existing custom property from the Filter Definitions field.
Click Select to add the custom property.
- To configure a custom property, use the following steps:
Locate and select the custom property on the Properties tab. Custom properties have the word Custom next to them, to differentiate them from system properties.
Select an expression type (Regex or JSON) from the Expression Type list.
Define a valid expression for the custom property based on the expression type that is selected in step b.
For Regex, the expression must be a valid java-compatible regular expression. Case-insensitive matching is supported only by using the (?i) token at the beginning of the expression. The (?i) token is saved in the log source extension .xml file. To use other expressions, such as (?s), manually edit the log source extension .xml file.
For JSON, the expression must be a path in the format of /"<name of top-level field>" with additional /"<name of sub-field>" to capture subfields if any.
For LEEF and CEF, to capture the value of a key/value pair, set the expression to the key. To capture the value of a header field, set the expression to the corresponding reserved word for that header field.
If the expression type is Regex, select a capture group.
Optional: To limit an expression to run against a specific category, click Edit to add selectivity to the custom property, and select a High Level Category and a Low Level Category.
Optional: To limit an expression to run against a specific event or QID, click Choose Event to search for a specific QID.
From the Expression window, click Ok.
- To add multiple expressions and reorder them, follow these
Click the Add (+) icon at the top of the expressions list.
Drag expressions in the order that you want them to run.