Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 

Defining Custom Properties by Using Custom Property Expressions

 

Define a custom property for an event payload by using a Regex, JSON, LEEF, or CEF expression. Because JSON parsing begins when a valid JSON object is detected, the entire event does not need to be in JSON format. Similarly, LEEF and CEF parsing begins only when a valid LEEF/CEF message is detected within the event. Regex parsing runs through the entire payload. You can extract properties from an event that is presented in JSON, LEEF, or CEF format by writing a JSON, LEEF, or CEF expression that matches the property.

You can use different expressions to capture various custom properties for the same event. You can also use a combination of JSON, Regex, LEEF, or CEF expressions to capture the same custom property if that property can be captured from multiple event formats.

  1. Log in to JSA and click the Admin tab.
  2. From the Data Sources section, click Custom Event Properties, and then click Add.
  3. In the Property Type Selection section, select Extraction Based.
  4. In the Test Field, enter the event payload that you want to use to test your custom property.
  5. In the Property Definition section, complete the following steps:
    1. If you're adding an expression to an existing property, select Existing Property and select a property from the list.

    2. If you're defining a new property, select New Property and enter the name of the property.

    3. To use the property for rules, reports and searches, select the Parse in advance for rules, reports, and searches check box.

      Note

      You must select this check box to use the property for rules and indexes. Selecting the check box increases the efficiency of reports and searches, but you don't need to select it to use the property for reports and searches. When you select the check box, properties are parsed when the event is initially received and before it is stored. As a result, the loads are put on the event collection service.

    4. Select a Field Type for the property.

    5. Optional: Enter a description for the property.

  6. In the Property Expression Definition section, complete the following steps:
    1. Keep the Enabled check box selected; otherwise, clear the check box to disable the property.

    2. From the Log Source Type list, select a log source type for the property.

    3. If the expression is only evaluated against events for a specific log source, select the log source from the Log Source list. If you want it to be evaluated against all log sources, don't select.

    4. If the expression is only evaluated against events with a specific event name or QID, click the Event Name and browse for a QID to associate the expression with.

    5. If the expression is evaluated against any event with a specific low-level category, select Category, and select the High Level Category and then Low Level Category for the event.

      Note

      If the expression is evaluated for all events of the selected log source type and log source, ensure that you set the Low Level Category and High Level Category to Any.

    6. From the Extraction using field, select the extraction method to use for the property.

    7. If the extraction method is Regex, enter the regex and the capture group number.

    8. If the extraction method is JsonKeypath, enter the JSON expression.

      Note

      A valid JSON expression is in the form:

      /"<name of top-level field>"

      For an event in a nested JSON format, a valid JSON expression is in the form:

      /"<name of top-level field>"/"<name of sub-level field>"..../"<name of sub-level field_n>"

      The following two examples show how to extract data from a JSON record.

      • Simple case of an event for a flat JSON record:

        {"action": "login", "user": "Firstname Lastname"}

        To extract the 'user' field, type /"user" in the JsonKeypath field.

      • Complex case of an event for a JSON record with nested objects:

        { "action": "login", "user": { "first_name": "Firstname", "last_name": "Lastname" } }

        To extract just the 'last_name' value from the 'user' subobject, type this expression:

        /"user"/"last_name"

    9. If the extraction method is LEEF Key, enter the LEEF expression.

      Note

      Valid LEEF expressions are in the form of either a single key reference, or a special LEEF header field reference.

      The following examples show how to extract data from a LEEF V1.0 record and a LEEF V2.0 record that contain the same keys.

      • Simple case of an event formatted in LEEF V1.0:

        LEEF:1.0|ABC Company|SystemDefender|1.13|console_login|devTimeFormat=yyyy-MM-dd’T’HH:mm:ss.SSSZ devTime=2017-10-18T11:26:03.060+0200 usrName=flastname name=Firstname Lastname authType=interactivePassword src=192.168.0.1

      • Simple case of an event formatted in LEEF V2.0 with the caret (^) separator character:

        LEEF:2.0|ABC Company|SystemDefender|1.13|console_login|^|devTimeFormat=yyyy-MMdd’T’HH:mm:ss.SSSZ^ devTime=2017-10-18T11:26:03.060+0200^usrName=flastname^name=Firstname Lastname ^authType=interactivePassword^src=192.168.0.1

      To extract the 'usrName' property, type usrName in the LEEF Key field.

      The possible keys that can be extracted in these examples are:

      • - devTimeFormat

      • - devTime

      • - usrName

      • - name

      • - authType

      • - src

      To extract a header key property, type the key in the following format in the LEEF Key field:

      $eventid$

      The LEEF header values can be extracted by using the following expressions:

      • - $leefversion$

      • - $vendor$

      • - $product$

      • - $version$

      • - $eventid$

    10. If the extraction method is CEF Key, enter the CEF expression.

      Note

      Valid CEF expressions are in the form of either a single key reference, or a special CEF header field reference.

      The following example shows how to extract data from a CEF record.

      • Simple case of an event formatted in CEF:

        CEF:0|ABC Company|SystemDefender|1.13|console_login|Console Login|1|start=Oct 18 2017 11:26:03 duser=flastname cs1=Firstname Lastname cs1Label=Person Name cs2=interactivePassword cs2Label=authType src=192.168.0.1

        To extract the 'cs1' property, type cs1 in the CEF Key field.

        The possible keys that can be extracted in the example are:

        • - start

        • - duser

        • - cs1

        • - cs1Label

        • - cs2

        • - cs2Label

        • - src

        To extract a header key property, type the key in the following format in the CEF Key field:

        $id$

        The CEF header values can be extracted by using the following expressions:

        • - $cefversion$

        • - $vendor$

        • - $product$

        • - $version$

        • - $id$

        • - $name$

        • - $severity$

    11. If you chose the Numeric Field Type in the Property Definition section, select a number format in the Extracted Number Format field in the Format section to define any digit group separators for the locale of the custom property.

    12. If you chose the Date/Time Field Type in the Property Definition section, enter a format in the Extracted Date/Time Format and Locale fields in the Format section to define the date and time for the locale of the custom property.

    13. Click Test to test the property expression definition.

  7. Click Save.