Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Paragon Insights Tagging

 

You can use the Paragon Insights (formerly HealthBot) graphical user interface (GUI) to create tagging profiles. You can configure a tagging profile to insert fields, values, and keys into a Paragon Insights rule. You can also set conditions that are checked against values stored in the times series database (TSDB) or Redis database.

Overview

Tagging allows you to insert fields, values, and keys into a Paragon Insights rule when certain conditions are met.

Paragon Insights supports the following types of tagging:

  • Static Tagging

    In static tagging, the tagging profile is applied to values stored in the time series data base (TSDB). These values do not vary a lot with time. In static tagging, you can avoid using When statements, and you can add Then statements individually to a row of the TSDB. You can add tags to all rows since no conditions are present.

  • Dynamic Tagging

    Paragon Insights Release 4.0.0 supports dynamic tagging where conditions used in Paragon Insights tagging are checked against values that are stored in Redis database. This database acts like a cache memory that stores dynamic data. Dynamic data is real-time data that is stored in Redis database.

Tagging Profile Terminology

The following list describes the tagging profile terminologies:

Policy

A policy is the top-level element in a tagging profile. You can add multiple policies within a single tagging profile. Multiple policies that exist within a tagging profile can have their own rules and terms.

Usage Notes:

  • Defining multiple policies within a single profile allows you to define terms for each rule in one profile rather than having to create one profile for each rule.

Rules

A rule is any defined Paragon Insights rule. The rule element type in a tagging profile is a list element. You can apply a specific policy profile to the rule(s) ([rule1, rule2]) included within the tagging profile.

Usage Notes:

You can describe the topic-name/rule-name requirement for the rules element in the following ways:

  • To name specific rules within a tagging profile, use the form: topic-name/rule-name.

    For example, protocol.bgp/check-bgp-advertised-routes. Navigate to Configuration>Rules to view configured rules.

  • Use an asterisk (*) with no other value or brackets to match all rules.

  • Use python-based fnmatch patterns to select all rules within a specific topic. For example, line-cards/*.

    For more information, see fnmatch — Unix filename pattern matching.

Terms

The term section of the tagging profile is where the match conditions are set and examined, and actions based on those matches are set and carried out. Set the conditions for a match in a when statement. Set the actions to be carried out upon completing a match in one or more then statements.

Usage Notes:

  • Each term can contain a when statement but it is not mandatory.

  • Each term must contain at least one then statement.

  • Multiple terms can be set within a single policy.

  • Terms are processed sequentially from top to bottom until a match is found. If a match is found, processing stops after completing the statements found in the then section. Other terms, if present, are not processed unless the next flag is enabled within the matched term. If the matched term has the next flag enabled, then subsequent terms are processed in order.

When Statements

When statements define the match conditions that you specify. When statements ultimately resolve to be true or false. You can define a term without a when statement. This equates to a default term wherein the match is assumed true and the subsequent then statement is carried out. Conversely, multiple conditions can be checked within one when statement.

If one or more of the conditions set forth in a when statement are not met, the statement is false and the term has failed to match; processing moves to the next term, if present.

Usage Notes:

When statements perform boolean operations on the received data to determine if it matches the criteria you set. The supported operations are:

  • Numeric Operations:

    • equal-to

    • not-equal-to

    • greater-than

    • greater-than-or-equal-to

    • less-than

    • less-than-or-equal-to

  • String Operations:

    • matches-with

    • does-not-match-with

  • Time Operations:

    • matches-with-scheduler

    Note

    The matches-with-scheduler option requires that a discreet scheduler be configured in the Administration > Ingest Settings > Scheduler page. The name of the scheduler can then be used in the matches-with-scheduler when statement

  • Go Language Expressions:

    • eval <simple-go-expression>

    For example: eval a + b <= c.

Then Statements

Then statements implement the tagging instructions that you provide. This is done only after there is a complete match of the conditions set forth in a when statement contained in the same term. Each term defined must have at least one then statement. Each then statement must have one or more than one action(s) defined; the actions available in then statements are:

add-fieldAdds a normal field to the rule(s) listed in the rule section.

Multiple fields can be added within a then statement. The add-field action requires that you also define the kind of field you are adding with the field-type parameter:

  • string

  • integer

  • float

Note

If you do not define a field type, the new field gets added with the default field-type of string.

add-keyAdds a key field with string data type to the rule(s). Added key fields are indexed and can be searched for just like any other key field.

Usage Notes:

  • You can set the next flag to true within a then statement. When this flag is set to true, the next term in the policy gets evaluated if all of the conditions of the current term match.

Example Configuration: Elements of a Tagging Profile

Paragon Insights configuration elements are displayed as pseudo-config. This configuration resembles the hierarchical method used by Junos OS.

Elements of a Tagging Profile shows how tagging profile elements are named and how they are related to each other.

Elements of a Tagging Profile

How do Tagging Profiles Work?

You can use tagging profiles to set the conditions, define new fields and keys, and insert values into those fields. Tagging profiles are applied as part of ingest settings to allow the tags to be added to the incoming data before Paragon Insights processes the data. Since one or more rules are defined within each profile, the rules are added to a playbook and applied to a device group when the tagging profile is applied to a device.

Table 1 shows an example application identification scenario based on source-port, destination-port, and protocol of traffic seen in a NetFlow stream.

Table 1: Fields in NetFlow Stream

source-port

destination-port

protocol

derived-application

2541

Any

6 (TCP)

NetChat

Any

2541

6 (TCP)

1755

Any

17 (UDP)

MS-streaming

Any

830

6 (TCP)

netconf-ssh

7802

Any

17 (UDP)

vns-tp

In Table 1, you use three existing fields in a NetFlow stream to guess the application traffic in the stream. You then create a new field called derived-application and populate it based on the values seen in the traffic.

You can apply tagging profiles at the device group level. See Example pseudo-configuration shown below.

  • When a device in a device group has a tagging profile applied to it, and the device group has another tagging profile applied to the whole group of devices, the tagging profile of the device group is merged with the existing tagging profile of the device.

    For example, D-A-Net is a device that is part of a device group called Group-D1. D-A-Net has a tagging profile applied to it. There is another tagging profile applied on the device group, Group-D1, as well. In such a scenario, the tagging profile applied to the device group is merged with the tagging profile of the device, D-A-Net.

  • When the tagging profile applied to the device group and the tagging profile applied to the device in the group renders the same output, the tagging profile of the device is preserved.

Example pseudo-configuration shown below

In this example, device r0 has tagging profile, profile1, assigned at the device level and tagging profile, profile2, assigned by its membership in the device- group (core).

Device r1 has tagging profile, profile2, assigned by its membership in device group, core.

In this scenario, profile1 and profile2 are merged on device r0. However, if profile1 and profile2 both define the same fields but the fields contain different values, the value from profile1 takes precedence because it is assigned directly to the device.

Device r1 only gets tagging profile profile2.

Caveats

  • Fields and keys added using tagging profiles cannot be used within periodic aggregation fields. This is because periodic aggregation must take place before any UDFCode function (reference, vector, UDF, ML) is applied.

  • Tagging profiles can consist of only fields in add-key or add-field. Vectors cannot be added to a rule by a tagging profile.

  • Vector comparison operations cannot be used within tagging profile terms. Only field Boolean operations are permitted.

  • For tagging profile conditional operations within a when statement, the used field must be of type sensor, constant, or reference.

    This is applicable only in static tagging.

  • If the field used within tagging profile Boolean operation is of type reference, then this reference field must not depend on any user-defined-function or formula defined within the same rule.

Types of Tagging

Paragon Insights supports static tagging and dynamic tagging.

Static Tagging

In static tagging, the tagging profile is applied to values stored in the time series data base (TSDB). These values do not vary a lot with time. In static tagging, you can avoid using When statements, and you can add Then statements to a tagging profile.

Sample Static Tagging Configuration

In this sample static tagging configuration, the lack of a when statement means that any device that this tagging profile is applied to will have the field tenant-id assigned with the value tenant1. The fields and values defined in this profile are assigned to all rules that are applied to a device or device-group because of the * in the rules parameter.

You can also create a static tagging profile from the Paragon Insights graphical user interface (GUI). Navigate to Configuration > Sensor > Settings > Tagging Profile page to create a tagging profile.

Application Identification

Table 1 shows an example application identification scenario based on source-port, destination-port, and protocol of traffic seen in a NetFlow stream.

To create the derived-application field as given in Table 1 from the received data (data under source-port, destination port, and protocol), you must use a tagging profile definition that looks like this:

Dynamic Tagging

Paragon Insights supports dynamic tagging. In dynamic tagging, you can set conditions in a tagging profile, that in turn are checked against values that are stored in Redis database. When these conditions are met, they are applied to incoming data before Paragon Insights processes the data.

Benefits of Dynamic Tagging

  • Values stored in Redis database are current and dynamic.

  • Redis database can be used as a cache memory to store real-time data.

Understanding Redis Database and Dynamic Tagging Configurations

Understanding Redis Database and Dynamic Tagging Configurations

  • Key structure is <Device-group-name>::<device-id>::<key-name> __network::<network-group-name>::<key-name>, where :: is the key separator.

    Example key structures:

    • Device Group

      Core:r1::/components/

    • Network Group

      network::net_check::topic/rule

  • Values are stored in JSON string format <json dump as string> in Redis. However, values are provided in string, integer, and float formats.

    Example value formats:

    • Core:r1::/components/= value1

    • Core:r1::/components/='{“key1”: value1, “key2”: value2}’

    • Core:r1::/components/='{“key1”: {“key2”: value1, “key3”: value2}’

    • Core:r1::/components/='{“key1”: {“key2”: ‘[list of values]’, “key3”: value1}’

    • Sample tagging-profile configurations using when statement.

    • Use a . operator between interfaces.

      In the following example, key3 interface is nested within key2 interface in the right operand.

    • Sample tagging-profile configurations using then statement.

    • Use a . operator between interfaces.

      In the following example, key3 interface is nested within key2 interface in the right operand.

  • Using exist operator in configurations.

    • Using exist as key.

      Redis Data Structure

      tagging-profile Using when Statement

    • Using exist as value in list.

      Redis Data Structure

      tagging-profile Using when Statement

  • Using $ in then statements.

    When you use $<field-name> within a Redis key, $<field-name> is replaced with a value from the already processed database value.

    As an example, consider that ge-1/0/2 is present within Redis key.

    Redis Data Structure

    Example tagging -profile

    In this scenario, the tagging-profile checks if $interface-name is present in the Redis database, and if key1 value for the given interface name is greater than 30. If the statement is true, the tagging-profile fetches key2 value from name field. In this example tagging profile, the name value is interface-meta-data.

  • To enable dynamic tagging, set in-memory value to true.

    By default in-memory value is set to false.

Add a Tagging Profile

You can use the Paragon Insights graphical user interface (GUI) to add static tagging and dynamic tagging profiles.

Adding a Static Tagging Profile

To add a static tagging profile:

  1. Navigate to Settings > Ingest.

    The Ingest Settings page is displayed.

  2. Click the Tagging Profile tab and then click (+) icon to add a tagging profile.

    The Add Tagging Profile page is displayed.

  3. Enter the following information in the Add Tagging Profile page:
    1. Enter a name for the tagging profile in the Name text box.

      The maximum length is 64 characters.

      Regex pattern: “[a-zA-Z][a-zA-Z0-9_-]*

    2. Click Add Policy under Policies to define a policy for this tagging profile.

      You can define one or more policies.

      The New Policy section is displayed.

      1. Enter a name for the new policy in the Policy Name text box.

        The maximum length is 64 characters.

        Regex pattern: “[a-zA-Z][a-zA-Z0-9_-]*

      2. Enter a rule that you want to apply to this tagging profile. The rule can contain an fnmatch expression.

        You can apply one or more than one rule to a profile. A rule is any defined Paragon Insights rule.

    3. Click Add Term under Terms to define a list of conditions.

      The Term: New Term drop-down list is added.

      Click the Term: New Term drop-down list to configure the following text boxes.

      1. Enter a name for the match condition in the Term Name text box.
      2. Configure When and Then statements.

        You can define tagging instructions in a Then statement. After the conditions that you set in a When statement are met, the Then statement is implemented. However, When statements are not mandatory in static tagging. A static tagging profile can be configured with just a Then statement.

        To configure a Then statement:

        1. Click Add Keys to add a key to the rules listed.

          The Name and Value text boxes are displayed.

          • Enter a name for the key in the Name text box.

            The maximum length is 64 characters.

            Regex pattern: “[a-zA-Z][a-zA-Z0-9_-]*

            This name is added as the key field for all rules configured within the tagging profile rules section.

          • Enter a value that you want to associate to the key, in the Valuetext box.

        2. Click Add Field to add a field to the rules listed.

          The Name and Value text boxes, and the Type drop-down list are displayed.

          • Enter the name in the Name text box.

          • Enter a value in the Value text box.

          • Select the field type from the Type drop-down list.

            String type is selected by default.

        3. Set the Evaluate next term flag to True to evaluate conditions in the next term. Evaluate next term only if the first condition is satisfied.

          By default, the Evaluate next term flag is set to False.

  4. Click Save to only save the configuration.

    Click Save & Deploy to save and deploy the configuration immediately.

Adding a Dynamic Tagging Profile

To configure a dynamic tagging profile with Redis:

  1. Navigate to Settings > Ingest.

    The Ingest Settings page is displayed.

  2. Click the Tagging Profile tab and then click (+) icon to add a tagging profile.

    The Add Tagging Profile page is displayed.

  3. Enter the following information in the Add Tagging Profile page.
    1. Enter a name for the tagging profile in the Name text box.

      The maximum length is 64 characters.

      Regex pattern: “[a-zA-Z][a-zA-Z0-9_-]*

    2. Click Add Policy under Policies to define a policy for this tagging profile.

      You can define one or more policies.

      The New Policy section is displayed.

      1. Enter a name for the new policy in the Policy Name text box.

        The maximum length is 64 characters.

        Regex pattern: “[a-zA-Z][a-zA-Z0-9_-]*

      2. Enter a rule that you want to apply to this tagging profile. The rule can contain an fnmatch expression.

        You can apply one or more rules to a profile. A rule is any defined Paragon Insights rule.

    3. Click Add Term under Terms to define a list of conditions.

      The Term: New Term drop-down is added.

      Click the Term: New Term drop-down list to configure the following fields:

      1. Enter a name for the match condition in the Term Name text box.
      2. Configure When and Then statements:

        You set conditions for a match in a when statement. To configure When statement,

        1. Click Add When.

          The New When Condition drop-down list is displayed.

        2. Click New When Condition to view the Operator drop-down list.
        3. Select a boolean operation that you want to apply to incoming data from the Operator drop-down list.

          The Left Operand and Right Operand text boxes are displayed.

          Note

          The New When Condition drop-down list is automatically renamed to the operator condition that you selected.

          • Enter the value of the left operand of assignment that you selected, in the Left Operand text box.

            You can use $ as prefix to populate database values. For example, $memory. However, using $ as prefix is not mandatory.

          • Enter the value of the right operand of assignment that you selected, in the Right Operand text box.

            This value is populated from the Redis database.

          • Set the Evaluate in Memory flag to True to populate data from the Redis database.

            By default, the Evaluate in Memory flag is set to False. When the flag is set to false, data is populated from the TSDB.

            Figure 1: Flags: Evaluate in Memory, Evaluate Next Term
            Flags: Evaluate in Memory,
Evaluate Next Term
          • Set the Evaluate next term flag to True to evaluate conditions in the next term. After the first condition is satisfied, the conditions in the next term are evaluated.

            By default, the Evaluate next term flag is set to False.

        You can define tagging instructions in a Then statement. After the conditions that you set in a When statement are met, the Then statement is implemented. However, When statements are not mandatory. A tagging profile can be configured with just a Then statement.

        To configure a Then statement:

        1. Click Add Keys to add a key to the rules listed.

          The Name and Value text boxes are displayed.

          • Enter a name for the key in the Name text box.

            The maximum length is 64 characters.

            Regex pattern: “[a-zA-Z][a-zA-Z0-9_-]*

            This name will be added as key field for all rules configured within the tagging profile rules section.

          • Enter a value that you want to associate with the key, in the Value text box.

        2. Click Add Field to add a text box to the rules listed.

          The Name and Value text boxes, and the Type drop-down list are displayed.

          • Enter the name in the Name text box.

          • Enter a value in the Value text box.

          • Select the field type from the Type drop-down list.

            String type is selected by default.

        3. Set the Evaluate in Memory flag to True to populate data from the Redis database.

          By default, the Evaluate in Memory flag is set to False.

        4. Set the Evaluate next term flag to True to evaluate conditions in the next term. The next term is evaluated only if the first condition is satisfied.

          By default, the Evaluate next term flag is set to False.

  4. Click Save to only save the configuration.

    Click Save & Deploy to save and deploy the configuration immediately.

Apply a Tagging Profile

You can configure a tagging profile to insert fields, values, and keys into a Paragon Insights rule. You can also set conditions that are checked against values stored in the times series database (TSDB) or Redis database.

After you have created a tagging profile from the Paragon Insights graphical user interface (GUI), you can apply a tagging profile to:

  • a new device

  • to an existing device

  • to a new device group

  • to an existing device group

Follow these steps to apply a tagging profile.

To apply a tagging profile to a new device:

  1. Navigate to Configuration > Device.

    The Device Configuration page is displayed.

  2. Click (+) icon to add a new device.

    The Add Device(s) page is displayed.

  3. After you have entered the necessary information to add a device, click the Tagging Profiles section.
  4. Select the tagging profile you want to apply to the device, from the Tagging Profiles drop-down list.
  5. Click Save to only save the configuration.

    Click Save & Deploy to save and immediately deploy the new configuration.

To apply a tagging profile to an existing device:

  1. Navigate to Configuration > Device.

    The Device Configuration page is displayed.

  2. Select the check box next to the name of the device and click Edit device.

    The Edit “device” page is displayed.

  3. Click the Tagging Profiles section to view the Tagging Profiles drop-down list.
  4. Select the tagging profile you want to apply to the device, from the Tagging Profiles drop-down list.
  5. Click Save to only save the configuration.

    Click Save & Deploy to save and immediately deploy the new configuration.

To apply a tagging profile to a new device group:

  1. Navigate to Configuration > Device Group.

    The Device Group Configuration page is displayed.

  2. Click (+) icon to add a new device group.

    The Add Device Group page is displayed.

  3. After you have entered the necessary information to add a device group, click the Tagging Profiles section.
  4. Select the tagging profile you want to apply to the device, from the Tagging Profiles drop-down list.
  5. Click Save to only save the configuration.

    Click Save & Deploy to save and immediately deploy the new configuration.

To apply a tagging profile to an existing device group:

  1. Navigate to Configuration > Device Group.

    The Device Group Configuration page is displayed.

  2. Select the check box next to the name of the device group and click the Edit device group icon.

    The Edit “device” page is displayed.

  3. Click the Tagging Profiles section to view the Tagging Profiles drop-down list.
  4. Select the tagging profile you want to apply to the device group, from the Tagging Profiles drop-down list.
  5. Click Save to only save the configuration.

    Click Save & Deploy to save and immediately deploy the new configuration.

Note
  • When a device in a device group has a tagging profile applied to it, and the device group has another tagging profile applied to the whole group of devices, the tagging profile of the device group is merged with the existing tagging profile of the device.

    For example, D-A-Net is a device that is part of a device group called Group-D1. D-A-Net has a tagging profile applied to it. There is another tagging profile applied on the device group, Group-D1, as well. In such a scenario, the tagging profile applied to the device group is merged with the tagging profile of the device, D-A-Net.

  • When the tagging profile applied to the device group and the tagging profile applied to the device in the group renders the same output, the tagging profile of the device is preserved.

Release History Table
Release
Description
Paragon Insights Release 4.0.0 supports dynamic tagging where conditions used in Paragon Insights tagging are checked against values that are stored in Redis database.