Paragon Insights Tagging
You can use the Paragon Insights (formerly HealthBot) graphical user interface (GUI) to create tagging profiles. You can configure a tagging profile to insert fields, values, and keys into a Paragon Insights rule. You can also set conditions that are checked against values stored in the times series database (TSDB) or Redis database.
Overview
Tagging allows you to insert fields, values, and keys into a Paragon Insights rule when certain conditions are met.
Paragon Insights supports the following types of tagging:
Static Tagging
In static tagging, the tagging profile is applied to values stored in the time series data base (TSDB). These values do not vary a lot with time. In static tagging, you can avoid using When statements, and you can add Then statements individually to a row of the TSDB. You can add tags to all rows since no conditions are present.
Dynamic Tagging
Paragon Insights Release 4.0.0 supports dynamic tagging where conditions used in Paragon Insights tagging are checked against values that are stored in Redis database. This database acts like a cache memory that stores dynamic data. Dynamic data is real-time data that is stored in Redis database.
Tagging Profile Terminology
The following list describes the tagging profile terminologies:
- Policy
- Rules
- Terms
- When Statements
- Then Statements
- Example Configuration: Elements of a Tagging Profile
Policy
A policy is the top-level element in a tagging profile. You can add multiple policies within a single tagging profile. Multiple policies that exist within a tagging profile can have their own rules and terms.
Usage Notes:
Defining multiple policies within a single profile allows you to define terms for each rule in one profile rather than having to create one profile for each rule.
Rules
A rule is any defined Paragon Insights rule. The rule element type in a tagging profile is a list element. You can apply a specific policy profile to the rule(s) ([rule1, rule2]) included within the tagging profile.
Usage Notes:
You can describe the topic-name/rule-name requirement for the rules element in the following ways:
To name specific rules within a tagging profile, use the form: topic-name/rule-name.
For example, protocol.bgp/check-bgp-advertised-routes. Navigate to Configuration>Rules to view configured rules.
Use an asterisk (*) with no other value or brackets to match all rules.
Use python-based fnmatch patterns to select all rules within a specific topic. For example,
line-cards/*
.For more information, see fnmatch — Unix filename pattern matching.
Terms
The term section of the tagging profile is where the match conditions are set and examined, and actions based on those matches are set and carried out. Set the conditions for a match in a when statement. Set the actions to be carried out upon completing a match in one or more then statements.
Usage Notes:
Each term can contain a when statement but it is not mandatory.
Each term must contain at least one then statement.
Multiple terms can be set within a single policy.
Terms are processed sequentially from top to bottom until a match is found. If a match is found, processing stops after completing the statements found in the then section. Other terms, if present, are not processed unless the next flag is enabled within the matched term. If the matched term has the next flag enabled, then subsequent terms are processed in order.
When Statements
When statements define the match conditions that you specify. When statements ultimately resolve to be true or false. You can define a term without a when statement. This equates to a default term wherein the match is assumed true and the subsequent then statement is carried out. Conversely, multiple conditions can be checked within one when statement.
If one or more of the conditions set forth in a when statement are not met, the statement is false and the term has failed to match; processing moves to the next term, if present.
Usage Notes:
When statements perform boolean operations on the received data to determine if it matches the criteria you set. The supported operations are:
Numeric Operations:
equal-to
not-equal-to
greater-than
greater-than-or-equal-to
less-than
less-than-or-equal-to
String Operations:
matches-with
does-not-match-with
Time Operations:
matches-with-scheduler
Note:The
matches-with-scheduler
option requires that a discreet scheduler be configured in the Administration > Ingest Settings > Scheduler page. The name of the scheduler can then be used in thematches-with-scheduler
when statementGo Language Expressions:
eval <simple-go-expression>
For example:
eval a + b <= c
.
Then Statements
Then statements implement the tagging instructions that you provide. This is done only after there is a complete match of the conditions set forth in a when statement contained in the same term. Each term defined must have at least one then statement. Each then statement must have one or more than one action(s) defined; the actions available in then statements are:
add-field | Adds a normal field to the rule(s) listed in the rule section. Multiple fields can be added within a then statement. The add-field action requires that you also define the kind of field you are adding with the field-type parameter:
Note:
If you do not define a field type, the new field gets added with the default field-type of string. |
add-key | Adds a key field with string data type to the rule(s). Added key fields are indexed and can be searched for just like any other key field. |
Usage Notes:
You can set the next flag to true within a then statement. When this flag is set to true, the next term in the policy gets evaluated if all of the conditions of the current term match.
Example Configuration: Elements of a Tagging Profile
Paragon Insights configuration elements are displayed as pseudo-config. This configuration resembles the hierarchical method used by Junos OS.
Elements of a Tagging Profile shows how tagging profile elements are named and how they are related to each other.
Elements of a Tagging Profile
healthbot { ingest-settings { data-enrichment { tagging-profile <tagging-profile-name> { policy <policy-name> { rules [ List of Rules ]; term <term-name1> { when { <condition1> <condition2> } then { add-field <field-name1> { value <field-value1>; type <field-type>; } add-field <field-name2> { value <field-value2>; type <field-type>; } add-key <key-field-name> { value <key-field-value>; } } } term <term-name2> { then { add-field <field-name> { value <field-value>; type <field-type>; } } } } } } } }
How do Tagging Profiles Work?
You can use tagging profiles to set the conditions, define new fields and keys, and insert values into those fields. Tagging profiles are applied as part of ingest settings to allow the tags to be added to the incoming data before Paragon Insights processes the data. Since one or more rules are defined within each profile, the rules are added to a playbook and applied to a device group when the tagging profile is applied to a device.
Table 1 shows an example application identification scenario based on source-port, destination-port, and protocol of traffic seen in a NetFlow stream.
source-port |
destination-port |
protocol |
derived-application |
---|---|---|---|
2541 |
Any |
6 (TCP) |
NetChat |
Any |
2541 |
6 (TCP) |
|
1755 |
Any |
17 (UDP) |
MS-streaming |
Any |
830 |
6 (TCP) |
netconf-ssh |
7802 |
Any |
17 (UDP) |
vns-tp |
In Table 1, you use three existing fields in a NetFlow stream to guess the application traffic in the stream. You then create a new field called derived-application and populate it based on the values seen in the traffic.
You can apply tagging profiles at the device group level. See Example pseudo-configuration.
When a device in a device group has a tagging profile applied to it, and the device group has another tagging profile applied to the whole group of devices, the tagging profile of the device group is merged with the existing tagging profile of the device.
For example, D-A-Net is a device that is part of a device group called Group-D1. D-A-Net has a tagging profile applied to it. There is another tagging profile applied on the device group, Group-D1, as well. In such a scenario, the tagging profile applied to the device group is merged with the tagging profile of the device, D-A-Net.
When the tagging profile applied to the device group and the tagging profile applied to the device in the group renders the same output, the tagging profile of the device is preserved.
Example pseudo-configuration shown below
device r0 { host r0; tagging-profile [ profile1 ] } device r1 { host r1 } device-group core { devices [ r0 r1 ]; tagging-profile [ profile2 ] }
In this example, device r0 has tagging profile, profile1, assigned at the device level and tagging profile, profile2, assigned by its membership in the device- group (core).
Device r1 has tagging profile, profile2, assigned by its membership in device group, core.
In this scenario, profile1 and profile2 are merged on device r0. However, if profile1 and profile2 both define the same fields but the fields contain different values, the value from profile1 takes precedence because it is assigned directly to the device.
Device r1 only gets tagging profile profile2.
Caveats
Fields and keys added using tagging profiles cannot be used within periodic aggregation fields. This is because periodic aggregation must take place before any UDFCode function (reference, vector, UDF, ML) is applied.
Tagging profiles can consist of only fields in add-key or add-field. Vectors cannot be added to a rule by a tagging profile.
Vector comparison operations cannot be used within tagging profile terms. Only field Boolean operations are permitted.
For tagging profile conditional operations within a when statement, the used field must be of type sensor, constant, or reference.
This is applicable only in static tagging.
If the field used within tagging profile Boolean operation is of type reference, then this reference field must not depend on any user-defined-function or formula defined within the same rule.
Types of Tagging
Paragon Insights supports static tagging and dynamic tagging.
Static Tagging
In static tagging, the tagging profile is applied to values stored in the time series data base (TSDB). These values do not vary a lot with time. In static tagging, you can avoid using When statements, and you can add Then statements to a tagging profile.
Sample Static Tagging Configuration
healthbot { ingest-settings { data-enrichment { tagging-profile profile { policy policy1 { rules *; term term1 { then { add-key "tenant-id" { value tenant1; } } } } } } } }
In this sample static tagging configuration, the lack of a when statement means that any device that this tagging
profile is applied to will have the field tenant-id assigned with the value tenant1. The fields
and values defined in this profile are assigned to all rules that
are applied to a device or device-group because of the *
in the rules parameter.
You can also create a static tagging profile from the Paragon Insights graphical user interface (GUI). Navigate to Configuration > Sensor > Settings > Tagging Profile page to create a tagging profile.
Application Identification
Table 1 shows an example application identification scenario based on source-port, destination-port, and protocol of traffic seen in a NetFlow stream.
To create the derived-application field as given in Table 1 from the received data (data under source-port, destination port, and protocol), you must use a tagging profile definition that looks like this:
healthbot { ingest-settings { data-enrichment { tagging-profile profile1 { policy policy1 { rules *; term term1 { when { matches-with "$source-port" "$netchat-source-port"; matches-with "$protocol" "6 (TCP)"; } then { add-key "application" { value netchat; } } } term term2 { when { matches-with "$protocol" "6 (TCP)"; matches-with "$destination-port" "$netchat-dest-port"; } then { add-key "application" { value netchat; } } } term term3 { when { matches-with "$source-port" "$ms-streaming-source-port"; matches-with "$protocol" "17 (UDP)"; } then { add-key "application" { value ms-streaming; } } } term term4 { when { matches-with "$source-port" "$netconf-ssh-source-port"; matches-with "$protocol" "6 (TCP)"; } then { add-key "application" { value netconf-ssh; } } } term term5 { when { matches-with "$source-port" "$vns-tp-source-port"; matches-with "$protocol" "17 (UDP)"; } then { add-key "application" { value vns-tp; } } } } } } } }
Dynamic Tagging
Paragon Insights supports dynamic tagging. In dynamic tagging, you can set conditions in a tagging profile, that in turn are checked against values that are stored in Redis database. When these conditions are met, they are applied to incoming data before Paragon Insights processes the data.
Benefits of Dynamic Tagging
Values stored in Redis database are current and dynamic.
Redis database can be used as a cache memory to store real-time data.
Understanding Redis Database and Dynamic Tagging Configurations
Understanding Redis Database and Dynamic Tagging Configurations
Key structure is
<Device-group-name>:<device-id>::key-name__network:<network-group-name>::key-name
, where::
is the key separator.Example key structures:
Device Group
Syntax:
<Device-group-name>:<device-id>::key-name
Example:
Device GroupCore:r1::/components/
Network Group
Syntax:
__network:<network-group-name>::key-name
Example:
network::net_check::topic/rule
Values are stored in JSON string format
<json dump as string>
in Redis. However, values are provided in string, integer, and float formats.Example value formats:
Core:r1::/components/= value1
Core:r1::/components/='{“key1”: value1, “key2”: value2}’
Core:r1::/components/='{“key1”: {“key2”: value1, “key3”: value2}’
Core:r1::/components/='{“key1”: {“key2”: ‘[list of values]’, “key3”: value1}’
Sample
tagging-profile
configurations usingwhen
statement."when" : { "matches-with" : [ { "left-operand" : "$field1", "right-operand" : “/interfaces/.key1”, “in-memory”: true } ] }
Use a
.
operator between interfaces.In the following example,
key3
interface is nested withinkey2
interface in the right operand."when" : { "matches-with" : [ { "left-operand" : "$field1", "right-operand" : “/interfaces/.key2.key3”, } ] }
Sample
tagging-profile
configurations usingthen
statement."then" : { "add-field" : [ { "name" : "field1", "value" : "redis-key", "type" : "integer", "in-memory": true } ] }
Use a
.
operator between interfaces.In the following example,
key3
interface is nested withinkey2
interface in the right operand."then" : { "add-field" : [ { "name" : "field2", "value" : "redis-key1.redis-key2.redis-key3", "type" : "integer", "in-memory": true } ] }
Using
exist
operator in configurations.Using
exist
as key.Redis Data Structure
“Core:r1::/interfaces/” = ‘{“ge-1/0/2”: {“key1”: value1, “key2”: value2}’
tagging-profile Using when Statement
“when”: { “exists”: { “field”: “$interface-name”, “path”: “/interfaces/”, “in-memory”: true } “then”: { do-something.. }
Using
exist
as value in list.Redis Data Structure
“Core:r1::/interfaces/” = ‘{“key1”: {“key2”: [‘ge-1/0/2’, ‘ge-1/0/3’], “key3”: value1}}
tagging-profile Using when Statement
“when”: { “exists”: { “field”: “$interface-name”, “path”: “/interfaces/.key1.key2” , “in-memory”: true } }, “then”: { “add-field”: [ “name”: “field1”, “value”: “/interfaces/.key1.key3”, “in-memory”: true ] }
Using $ in then statements.
When you use
$<field-name>
within a Redis key,$<field-name>
is replaced with a value from the already processed database value.As an example, consider that ge-1/0/2 is present within Redis key.
Redis Data Structure
“Core:r1::/interfaces/” = ‘{“ge-1/0/2”: {“key1”: value1, “key2”: value2}, “ge-1/0/3”: {“key1”: value1, “key2”: value2}’
Example tagging -profile
“when”: { “exists”: [ { “field”: “$interface-name”, “path”: “/interfaces/”, “in-memory”: true } ], “greater-than”: [ { “left-operand”: “30”, “right-operand”: “/interfaces/.$interface-name.key1” , “in-memory”: true } ] }, “then”: { “add-field”: [ “name”: “interface-meta-data”, “value”: “/interfaces/.$interface-name.key2”, “in-memory”: true ] }
In this scenario, the
tagging-profile
checks if$interface-name
is present in the Redis database, and ifkey1
value for the given interface name is greater than 30. If the statement is true, thetagging-profile
fetcheskey2
value fromname
field. In this example tagging profile, the name value isinterface-meta-data
.To enable dynamic tagging, set
in-memory
value to true.By default
in-memory
value is set to false.“when”: { “exists”: { “field”: “$interface-name”, “path”: “/interfaces/.key1.key2” , “in-memory”: true } } “then”: { “add-field”: [ “name”: “interface-meta-data”, “value”: “/interfaces/.$interface-name.key2”, “in-memory”: true ] }
Add a Tagging Profile
You can use the Paragon Insights graphical user interface (GUI) to add static tagging and dynamic tagging profiles.
Adding a Static Tagging Profile
To add a static tagging profile:
Adding a Dynamic Tagging Profile
To configure a dynamic tagging profile with Redis:
Navigate to Settings > Ingest.
The Ingest Settings page is displayed.
Click the Tagging Profile tab and then click the plus (+) icon to add a tagging profile.
The Create Tagging Profile page is displayed.
Enter the following information in the Create Tagging Profile page.
Enter a name for the tagging profile in the Profile Name text box.
The maximum length is 64 characters.
Regex pattern: “
[a-zA-Z][a-zA-Z0-9_-]*
”Click the plus (+) icon under Policies to define a policy for this tagging profile.
You can define one or more policies.
The Policies section is displayed.
Enter a name for the new policy in the Policy Name text box.
The maximum length is 64 characters.
Regex pattern: “
[a-zA-Z][a-zA-Z0-9_-]*
”Enter a rule that you want to apply to this tagging profile. The rule can contain an fnmatch expression.
You can apply one or more rules to a profile. A rule is any defined Paragon Insights rule.
Click the plus (+) icon under Terms to define a list of conditions.
Enter a name for the match condition in the Term Name text box.
Configure When and Then statements:
You set conditions for a match in a when statement. To configure When statement,
Click the Edit (pencil) icon.
The When Condition page is displayed.
Click + Add another when to view the Operator drop-down list.
Select a boolean operation that you want to apply to incoming data from the Operator drop-down list.
The Left Operand and Right Operand text boxes are displayed.
Note:+ Add another when is automatically renamed to the operator condition that you selected.
Enter the value of the left operand of assignment that you selected, in the Left Operand text box.
You can use
$
as prefix to populate database values. For example,$memory
. However, using$
as prefix is not mandatory.Enter the value of the right operand of assignment that you selected, in the Right Operand text box.
This value is populated from the Redis database.
Set the Evaluate in Memory flag to True to populate data from the Redis database.
By default, the Evaluate in Memory flag is set to False. When the flag is set to false, data is populated from the TSDB.
Click OK.
Set the Evaluate next term flag to True to evaluate conditions in the next term. After the first condition is satisfied, the conditions in the next term are evaluated.
By default, the Evaluate next term flag is set to False.
You can define tagging instructions in a
Then
statement. After the conditions that you set in aWhen
statement are met, theThen
statement is implemented. Starting in Paragon Insights Release 4.2.0,When
statements are mandatory in tagging profile.To configure a Then statement:
Click the plus (+) icon to add a key to the rules listed.
The Key Name and Value text boxes are displayed.
Enter a name for the key in the Key Name text box.
The maximum length is 64 characters.
Regex pattern: “
[a-zA-Z][a-zA-Z0-9_-]*
”This name will be added as key field for all rules configured within the tagging profile rules section.
Enter a value that you want to associate with the key, in the Value text box.
Click the plus (+) to add a text box to the rules listed.
The Field Name and Value text boxes, and the Type drop-down list are displayed.
Enter the name in the Field Name text box.
Enter a value in the Value text box.
Select the field type from the Type drop-down list.
String type is selected by default.
Starting in Paragon Insights Release 4.2.0, you can also select unsigned integer as a name field data type. An unsigned integer is a data type that can contain values from 0 through 4,294,967,295.
Set the Evaluate in Memory flag to True to populate data from the Redis database.
By default, the Evaluate in Memory flag is set to False.
Click OK.
Set the Evaluate next term flag to True to evaluate conditions in the next term. The next term is evaluated only if the first condition is satisfied.
By default, the Evaluate next term flag is set to False.
Click Save to only save the configuration.
Click Save & Deploy to save and deploy the configuration immediately.
Apply a Tagging Profile
You can configure a tagging profile to insert fields, values, and keys into a Paragon Insights rule. You can also set conditions that are checked against values stored in the times series database (TSDB) or Redis database.
After you have created a tagging profile from the Paragon Insights graphical user interface (GUI), you can apply a tagging profile to:
a new device
to an existing device
to a new device group
to an existing device group
Follow these steps to apply a tagging profile.
To apply a tagging profile to a new device:
To apply a tagging profile to an existing device:
Navigate to Configuration > Device.
The Device Configuration page is displayed.
Select the check box next to the name of the device and click Edit device.
The Edit “device” page is displayed.
Click the Tagging Profiles section to view the Tagging Profiles drop-down list.
Select the tagging profile you want to apply to the device, from the Tagging Profiles drop-down list.
Click Save to only save the configuration.
Click Save & Deploy to save and immediately deploy the new configuration.
To apply a tagging profile to a new device group:
Navigate to Configuration > Device Group.
The Device Group Configuration page is displayed.
Click the add (+) icon to add a new device group.
The Add Device Group page is displayed.
After you have entered the necessary information to add a device group, click the Tagging Profiles section.
Select the tagging profile you want to apply to the device, from the Tagging Profiles drop-down list.
Click Save to only save the configuration.
Click Save & Deploy to save and immediately deploy the new configuration.
To apply a tagging profile to an existing device group:
Navigate to Configuration > Device Group.
The Device Group Configuration page is displayed.
Select the check box next to the name of the device group and click the Edit device group icon.
The Edit “device” page is displayed.
Click the Tagging Profiles section to view the Tagging Profiles drop-down list.
Select the tagging profile you want to apply to the device group, from the Tagging Profiles drop-down list.
Click Save to only save the configuration.
Click Save & Deploy to save and immediately deploy the new configuration.
When a device in a device group has a tagging profile applied to it, and the device group has another tagging profile applied to the whole group of devices, the tagging profile of the device group is merged with the existing tagging profile of the device.
For example, D-A-Net is a device that is part of a device group called Group-D1. D-A-Net has a tagging profile applied to it. There is another tagging profile applied on the device group, Group-D1, as well. In such a scenario, the tagging profile applied to the device group is merged with the tagging profile of the device, D-A-Net.
When the tagging profile applied to the device group and the tagging profile applied to the device in the group renders the same output, the tagging profile of the device is preserved.
Delete a Tagging Profile
To delete a tagging profile:
Change History Table
Feature support is determined by the platform and release you are using. Use Feature Explorer to determine if a feature is supported on your platform.