Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Paragon Insights Push-Model Ingest Methods

 

Paragon Insights currently supports the following push-model sensors:

Native GPB

Native sensors use a Juniper-proprietary data model using Google protocol buffers (GPB). The device pushes telemetry data (when configured) over UDP.

The device pushes data from the Packet Forwarding Engine, that is, directly from a line card. This means telemetry data is sent over the forwarding plane, so the collector must have in-band connectivity to the device.

To use native format, you configure the device with settings that include where to send the telemetry data. When you configure Paragon Insights to start collecting the data, the stream is already flowing towards the server.

For more information on native sensors, see Understanding the Junos Telemetry Interface Export Format of Collected Data.

Native GPB - Device Configuration

The requirements for using the Native GPB sensor type on a Junos OS device are shown below.

  • Junos OS Version: 15.1 or later

  • Required configuration—configure a sensor profile for each relevant related rule in Paragon Insights:

    To configure streaming server port in Paragon Insights GUI:

    1. Go to Settings > Ingest.
    2. Select Native GPB tab on the Ingest Settings page.
    3. Enter the port number. The port number must be the same as the remote-port configured in the streaming server profile.

      You can use the toggle button to enable or disable the Port field.

    4. Click Save & Deploy to enable the sensor to collect data in your network.

    See Configuring a Junos Telemetry Interface Sensor for more information.

NetFlow

Starting with Release 3.0.0, Paragon Insights (formerly HealthBot) supports NetFlow natively as another ingest method, using a data model that aligns with other Paragon Insights ingest mechanisms to provide all the same feature richness. With this release, Paragon Insights supports NetFlow v9 and NetFlow v10 (IPFIX).

How it Works

NetFlow is a network protocol for collecting IP traffic statistics, which can then be exported to a tool for analysis. NetFlow is available in different versions, the latest being NetFlow v9 and NetFlow v10. The NetFlow v9 data export format is described in RFC 3954; NetFlow v10 is officially known as IPFIX and standardized in RFC 7011.

Junos devices support flow monitoring and aggregation using these protocols; the Junos OS samples the traffic, builds a flow table, and sends the details of the flow table over a configured UDP port to a collector, in this case Paragon Insights. Paragon Insights receives the incoming Netflow data, auto-detects it as v9 or v10, and process it further.

As shown above, the network device pushes data from the Packet Forwarding Engine, that is, directly from a line card. This means flow data is sent over the forwarding plane, so the collector must have in-band connectivity to the device. To use the flow sensor option, you configure the device with settings that include where to send the flow data. When you configure Paragon Insights to start collecting the data, the flow data is already flowing towards the server.

NetFlow Templates

Where other ingest methods have established sensor formats and identification details - for example, native GPB references paths, SNMP references MIBs, etc. - flow has no equivalent mechanism. Instead, Paragon Insights uses templates. These flow templates provide a mechanism to identify and decode incoming flow data before sending it for further processing.

Paragon Insights provides predefined flow templates for NetFlow v9 and v10 (IPFIX), or you can define your own. The predefined templates match those which the Junos OS currently supports. For example, the Junos OS template, ipv4-template, aligns with the Paragon Insights template hb-ipfix-ipv4-template. To view the fields used in the Junos OS templates, see Understanding Inline Active Flow Monitoring.

Note

In the current ingest implementation for NetFlow, the following field types are not supported:

  • Fields for enterprise specific elements

  • Variable length fields

NetFlow Ingest Processing

The raw flow data that Paragon Insights receives is in binary format and unreadable. In order to make this data usable, Paragon Insights processes the incoming flow data as follows:

  • Paragon Insights listens for incoming flow data on a configured port

  • Since NetFlow messages don’t include a field that identifies the sending device, Paragon Insights uses the configured source IP address to derive a device ID

  • Templates identify and decode incoming flow data to determine which fields it contains

    The resulting decoded and normalized data is now in a readable and usable format. Here is an example of flow data decoded using the hb-ipfix-ipv4-template template:

    hb-ipfix-ipv4-template, destinationIPv4Address=192.168.48.200,

    destinationTransportPort=443,

    icmpTypeCodeIPv4=0,ingressInterface=1113, ipClassOfService=0,protocolIdentifier=6,

    sourceIPv4Address=172.16.235.191,sourceTransportPort=51032, bgpDestinationAsNumber=4200000000i,bgpSourceAsNumber=65000i,

    destinationIPv4PrefixLength=24i, dot1qCustomerVlanId=0i,dot1qVlanId=0i,egressInterface=1041i,

    flowEndMilliseconds=1483484591502u, flowEndReason=2i,flowStartMilliseconds=1483484577531u,

    ipNextHopIPv4Address="192.168.41.49", maximumTTL=120i,minimumTTL=120i,octetDeltaCount=188i,

    packetDeltaCount=4i,sourceIPv4PrefixLength=19i,

    tcpControlBits=16i,vlanId=0i 1483484642000000000

    The data shows information from the NetFlow messages using naming according to IPFIX Information Elements. For example, destinationIPv4Address maps to element ID 12 in the elements table.

  • Paragon Insights then performs further tagging, normalization, and aggregation as defined in the corresponding rule by the user.

  • Finally, the time-series database, TSDB receives the data. This is where things like trigger evaluation happen.

Warning

For NetFlow ingest, ensure that there is no source NAT in the network path(s) between the device and Paragon Insights. If the network path contains source NAT, then the received device information is not accurate.

NetFlow - Device Configuration

To use NetFlow as an ingest method in Paragon Insights, you must add configuration to the device you wish to monitor, to enable it to export flow data into Paragon Insights.

This example includes the Netflow v10 IPv4 template; adjust as needed for your environment. If not already done, complete device configuration to send NetFlow data to Paragon Insights as shown below.

## IPFIX template configuration

## Apply IPFIX template to enable traffic sampling

## 10.102.70.200 = Paragon Insights server

## port 2055; use this value in Paragon Insights GUI (device group config)

## inline-jflow = Enable inline flow monitoring for traffic from the designated address

## 198.51.100.1 = in-band interface doing the exporting; use this value in Paragon Insights GUI (device config)

## Associate sampling instance with the FPC

## Specify which interface traffic to sample

Example: Add a Device In Paragon Insights, Configure Paragon Insights for NetFlow, and Monitor

The following example walks through how to:

Add the Device In Paragon Insights

Now add the device in Paragon Insights, specifying the IP address(es) that will send the flow data.

  1. In the Paragon Insights GUI, click Configuration > Device in the left-nav bar, and click the add device button (+ Device).
  2. Click the + (Add Device) button
  3. In the Add Device(s) window that appears, fill in the appropriate fields.

    Be sure to fill in the Flow IPs field with the IP address(es) from which NetFlow data will arrive.

  4. Click Save & Deploy.

For more information about adding a device, see Adding a Device in Manage Devices, Device Groups, and Network Groups.

Usage Notes:

  • Incoming NetFlow messages don’t include a device ID; Paragon Insights uses the message’s source IP address to derive a device ID

  • When configuring this step, use the in-band interface IP address you configured in the sampling instance configuration on the device.

Add Device Group

With the device added, you now need to create a device group and define the flow ingest port for the device group.

  1. Click Configuration > Device Group
  2. Click the + (Add Device Group) button
  3. In the Add Device Group window that appears, fill in the fields as appropriate. In the Flow Ports field, enter the port(s) on which NetFlow data will arrive.Note

    If your Paragon Insights installation is a multi-node installation using Kubernetes, you must also specify which Paragon Insights nodes will receive the NetFlow traffic by filling in the Flow Deploy Nodes field in the Add/Edit Device Group window.

  4. Click Save & Deploy

    Usage Notes:

    • Paragon Insights will listen for NetFlow messages on this port for devices in this group.

    • The configured NetFlow ingest ports cannot be the same across device groups. You must configure a different port (or ports) for each group.

Define NetFlow Ingest Settings - Review Predefined Templates

NetFlow templates provide a mechanism to identify and decode incoming flow data before sending it for further processing within Paragon Insights.

  1. Click Settings > Ingest in the left-nav bar.

    The Ingest Settings page appears.

  2. Click the NetFlow tab to view the NetFlow Settings page.
  3. On the NetFlow settings page, review the available templates for use in a rule.

Usage Notes:

  • Notice that there are default flow templates for IPv4, IPv6, MPLS, MPLS-IPv4, MPLS-IPv6, and VPLS, for each of NetFlow v9 and v10.

  • The NetFlow templates include recognition patterns, called include fields and exclude fields, which help to recognize, identify, and categorize the incoming messages.

  • Since NetFlow messages don’t distinguish between keys and values (all fields are simply incoming data), the templates specify which fields should be treated as keys for raw data.

Define NetFlow Ingest Settings - (Optional) Create Your Own NetFlow Template

If the existing templates do not meet your needs, you can create your own template. You can also use custom templates to support other vendors’ devices.

  1. Click the plus (+) icon on the NetFlow settings page. .
  2. In the Add Template window that appears, fill in the following fields (you can leave the other settings as is):
    • Template Name - give the template a name

    • NetFlow version - select v9 or v10

    • Priority - Available values are 1 through 10

    • Include Fields - add one or more fields that you want included in the template you wish to use

    • Exclude Fields - add one or more fields that you do not want included in the template you wish to use

    • Key Fields - specify which fields in the incoming messages should be treated as keys

  3. Click Save & Deploy

    You should now see the template added to the NetFlow settings page.

  4. (Optional) Repeat the steps above to create more templates.

Usage Notes:

  • Priority - when a playbook includes multiple rules using the flow sensor, the priority value identifies which sensor and template gets priority over the other(s).

  • Include/Exclude fields - include fields to help identify the template to use, or at least a ‘short list’ of templates to use; exclude fields then narrow down to the single desired template.

    • Example 1 - consider the hb-ipfix-ipv4-template template: it includes two IPv4 fields to narrow down to hb-ipfix-ipv4-template and hb-ipfix-mpls-ipv4-template, and excludes an MPLS field to eliminate hb-ipfix-mpls-ipv4-template, leaving only hb-ipfix-ipv4-template.

    • Example 2 - consider the hb-ipfix-mpls-ipv4-template template: it includes the same two IPv4 fields to narrow down to hb-ipfix-ipv4-template and hb-ipfix-mpls-ipv4-template. It also includes an MPLS field, which immediately eliminates the former template and leaving the latter as the template to use.

Define NetFlow Ingest Settings - Clone an Existing NetFlow Template

Starting in Paragon Insights Release 4.0.0, you can clone an existing NetFlow template.

To clone an existing NetFlow template:

  1. Click Settings > Ingest from the left-nav bar.

    The Ingest Settings page is displayed.

  2. Click the NetFlow tab to view the Netflow Settings page.
  3. To clone a particular template for Paragon Insights Releases 4.1.0 and 4.0.0, click the Clone icon.

    To clone a particular template for Paragon Insights Release 4.2.0 and later, select the option button next to the name of the template and click Clone.

    The Clone Template: <name of template> page is displayed.

    From the Clone Template: <name of template> page, you can

    • Edit the Template Name, Description, and Priority sections.

    • Choose between Netflow v9 or Netflow v10 versions.

    • Add or exclude fields from Include Fields, Exclude Fields, and Key Fields.

  4. After you have made the necessary edits, click Save to save the modifications and to clone the template.

    Alternatively, click Save & Deploy to save modifications, clone the template, and deploy the template.

Configure a Rule Using the Flow Sensor

With the NetFlow ingest settings complete, you can now create a rule using flow as the sensor.

This example rule includes three elements:

  • A flow sensor that uses the NetFlow v10 IPv4 template

  • Six fields capturing data of interest

  • A trigger that indicates when traffic flow is higher or lower than expected

Note

See the usage notes at the end of this section for more detail on what has been configured.

  1. Click Configuration > Rules in the left-nav bar.
  2. On the Rules page, click the + Add Rule button.

    The Rules page refreshes to show a nearly empty rule on the right part of the page.

  3. In the top row of the rule window, leave the topic set as external and set the rule name that appears after the slash (/). In this example, it is periodic-aggregation-flow-rule.
  4. Add a description and synopsis if you wish.
  5. Click the + Add Sensor button and enter the following parameters in the Sensors tab:
  6. Now move to the Fields tab, click the + Add Field button and enter the following parameters to configure the first field, source-ipv4-address:
  7. Click the + Add Field button again and enter the following parameters to configure the second field, destination-ipv4-address:
  8. Click the + Add Field button again and enter the following parameters to configure the third field, sensor-traffic-count:
  9. Click the + Add Field button again and enter the following parameters to configure the fourth field, total-traffic-count:
  10. Click the + Add Field button again and enter the following parameters to configure the fifth field, traffic-count-maximum:
  11. Click the + Add Field button once more and enter the following parameters to configure the sixth field, traffic-count-minimum:
  12. As the last step for the fields configuration, set the field aggregation time-range value to 10s:
  13. Now move to the Variables tab, click the + ADD VARIABLE button and create the traffic-count-max and traffic-count-min variables that are the constants for the traffic-count-maximum and traffic-count-minimum fields, respectively.
    Note

    Only the definition for the traffic-count-max is shown graphically. Choose an appropriate Default Value when configuring both traffic-count-max and traffic-count-min variables. The value shown above is for testing purposes only and may not be appropriate for your network.

  14. Now move to the Triggers tab, click the + Add trigger button and enter the following parameters to configure a trigger called traffic-measurement-trigger:
  15. At the upper right of the window, click the Save & Deploy button.

Usage Notes:

  • Sensor Tab:

    • The sensor name ipv4-flow-sensor is user-defined

    • The sensor type is flow

    • The sensor uses the predefined template hb-ipfix-ipv4-template

  • Variables Tab:

    • The variables traffic-count-max and traffic-count-min are statically configured integers. In this case the values represent Bytes per second

    • These values are referenced in fields traffic-count-maximum and traffic-count-minimum and provide a reference point to compare against the total-traffic-count field

  • Fields Tab:

    • Six fields are defined; some fields are used in the trigger settings while one field is referenced within another field

    • The field names are user-defined fields (UDF)

    • Fields source-ipv4-address, destination-ipv4-address, and sensor-traffic-count are extracting information from the flow sensor input

    • Path values for these fields identify specific values from the NetFlow messages, using naming according to IPFIX Information Elements

    • Fields source-ipv4-address and destination-ipv4-address have the Add to rule key setting enabled, indicating that this field should be shown as a searchable key for this rule on the device health pages

    • Field total-traffic-count - sums the IPv4 packet count from the sensor-traffic-count field every 10 seconds

    • The fields traffic-count-maximum and traffic-count-minimum are simply fixed values; the values are derived from the variables defined above

    • Field aggregation time-range - typically set to a value higher (longer) than individual field time range settings with the aim of reducing the frequency of information being sent to the database

  • Triggers Tab:

    • The trigger name traffic-measurement-trigger is user-defined.

    • frequency 90s - HearthBot compares traffic counts every 90 seconds

    • In the term traffic-abnormal-gr:

      • When $total-traffic-count (the periodic count of incoming IPv4 traffic) is greater than $traffic-count-maximum (2500 Bps), show red and the message: “Total traffic count is above normal. Current total traffic count is : $total-traffic-count”.

    • In the term traffic-abnormal-ls:

      • When $total-traffic-count (the periodic count of incoming IPv4 traffic) is less than $traffic-count-minimum (500 Bps), show yellow and the message: “Total traffic count is below normal. Current total traffic count is : $total-traffic-count”.

    • In the term default-term:

    • Otherwise, show green and the message: “Total traffic count is normal. Current total traffic count is : $total-traffic-count”.

Add the Rule to a Playbook

With the rule created, you can now add it to a playbook. For this example, create a new playbook to hold the new rule.

  1. Click Configuration > Playbooks in the left-nav bar.
  2. On the Playbooks page, click the create + Create Playbook button.
  3. On the page that appears, enter the following parameters:
  4. Click Save & Deploy

Apply the Playbook to a Device Group

  1. On the Playbooks page, click the Apply (Airplane) icon for the playbook you configured above.
  2. On the Run Playbook page that appears
    • Enter a name for the playbook instance.

    • Select the desired device group from the Apply Group pull-down menu.

    • Click Run Instance.

  3. On the Playbooks page, confirm that the playbook instance is running. Note that the playbook may take some time to activate.

Monitor the Devices

With the playbook applied, you can begin to monitor the devices.

  1. Click Monitor > Device Group Health in the left-nav bar and select the device group to which you applied the playbook from the Device Group pull-down menu.
  2. Select one of more of the devices to monitor.
  3. In the Tile View, the external tile contains the parameters from the rule you configured earlier.

Differences Between NetFlow and sFlow

While the names sound similar, NetFlow and sFlow are two very different protocols. Table 1 shows the differences between NetFlow and sFlow.

Table 1: NetFlow and sFlow Differences

NetFlow/IPFIX

sFlow

Overview

A flow is a sequence of packets that share the same properties and travels between a sending host and a receiving host. Flow analysis looks at the metadata in the flow of packets. NetFlow is based on flow analysis techniques.

sFlow takes actual network packet samples from a device and forwards them to a collector for analysis. sFlow is a packet sampling and analysis technology, not a flow analysis technology.

More Detail

Flow analysis deals with identifying top talkers, top protocols, bandwidth usage, etc. within the network. It uses traffic metadata to provide this information.

Packet analysis deals with gaining in-depth information regarding a specific network conversation. This is possible because the packet header and payload are included in the sFlow data.

Data

NetFlow/IPFIX allows a user to define what data is collected from the devices as fields in a template. The data can be based on packet headers, traffic characteristics, or values from within the devices themselves.

NetFlow can provide information from layer 2 through layer 4 of the OSI model.

sFlow specifies a single way to report data: provide the entire packet header and payload data from a sampled interface.

sFlow can provide information from layer 2 through layer 7 of the OSI model.

Time

NetFlow/IPFIX headers contain the export time of the flow. Flow data can accumulate and be sent at any configured frequency. Template definitions allow for over 30 different methods to represent the time of an observed flow.

The sFlow standard requires that sampled packets are forwarded to the collector immediately upon capture.

State

NetFlow/IPFIX is a stateful protocol. The device receiving NetFlow data (Paragon Insights) must confirm the receipt of the data before more is sent.

sFlow is a stateless protocol. The sender simply forwards captured packets to the receiver (Paragon Insights).

Processing

NetFlow/IPFIX is a stateful protocol. The sending and receiving devices must both have the correct NetFlow/IPFIX template to encode and decode the flow. New templates can be configured by users and shared between sender and receiver (Paragon Insights)

The sending device uses the sFlow standard to encode the messages (packets) and the collector (Paragon Insights) uses the same standard to decode them.

sFlow

Starting with Paragon Insights Release 3.2.0, Paragon Insights supports sFlow (v5) natively as another flow-based ingest method, using a data model that aligns with other Paragon Insights ingest mechanisms to provide all the same feature richness.

sFlow Protocol

sFlow is a statistical sampling-based technology for high-speed switched or routed networks. You can configure sFlow to continuously monitor traffic at wire speed on all interfaces simultaneously if you want.

sFlow provides or helps with:

  • detailed and quantitative traffic measurements at gigabit speeds

  • insight into forwarding decisions

  • troubleshooting for network issues

  • congestion control

  • security and audit trail analysis

  • route profiling

Everything that sFlow does above, it does without impact to forwarding or network performance. For more information on sFlow, see: RFC 3176, InMon Corporation's sFlow: A Method for Monitoring Traffic in Switched and Routed Networks.

As a statistical sampling protocol, Juniper’s sFlow agent samples the traffic and counters on network interfaces, creates sFlow datagrams and forwards them to external sFlow collectors. Paragon Insights is one such collector.

sFlow Ingest Processing

The raw sFlow data that Paragon Insights receives is in binary format and unreadable. In order to make this data usable, Paragon Insights processes the incoming flow data as follows:

  • Paragon Insights listens for incoming flow data on a configured port

  • Based on Paragon Insights rule configuration, data is decoded and grouped into data sets by record type (raw packet header, Ethernet frame, etc.)

  • Maps the sFlow source IP (extracted from sFlow packets) to a Paragon Insights Device ID

    Note

    If no match can be made, the packet is dropped with no further decoding performed.

  • Normalizes the fields for further use within the system

  • Paragon Insights then performs further tagging, normalization, and aggregation as defined in the corresponding rule by the user.

  • Finally, the time-series database, TSDB receives the data. This is where things like trigger evaluation happen.

sFlow - Configuration in Paragon Insights

As mentioned above, processing of sFlow packets depends on Paragon Insights rule configuration. It also requires that you enable sFlow in the device group or device definition. This section describes sFlow enablement, and rule and sensor configuration options for sFlow.

First, to enable sFlow, you must enter at least one IP address in the device definition under Flow Source IPs, and enter at least one port number in the device group definition under sFlow Ports. Figure 1 below is a composite image that shows the device group definition overlaid with the device definition. The appropriate sections of each window are highlighted in red.

Figure 1: Enable sFlow Composite Image
Enable sFlow Composite Image

The devices in the group send their sFlow packets to Paragon Insights over the configured UDP port from the configured IP address(es). The port number(s) used in these definitions must be unique across the entire Paragon Insights installation.

Note
  • The Flow Source IPs address(es) must match an IP address that can be mapped from the Hostname/IP Address/Range field in the device definition. If devices send sFlow packets, but Paragon Insights cannot match the source IP to a defined device IP, then the packets are dropped without decoding.

  • Paragon Insights cannot differentiate sFlow from NetFlow by looking at the packets. If you are using both NetFlow and sFlow, the port numbers must also be unique between the two flow types.

Due to the nature of sFlow and the potentially huge amount of data that can come from even a single device, we recommend the following best-practices for managing sFlow ingest:

Best Practice
  • Use unique ports from the range: UDP/49152 to UDP/65535 for sFlow.

  • Use periodic aggregation to reduce the number of write procedures in the TSDB.

  • Do not enable the raw table data storage option in sFlow unless sufficient high-speed storage is available for Paragon Insights TSDB.

Configure sFlow Ingest

As with other ingest methods, navigate to Settings > Ingest and choose the sFlow tab on the left of the Ingest window.

Figure 2: sFlow Ingest Settings
sFlow Ingest Settings

As you can see in Figure 2 above, the Sflow Settings are broken down into 4 sections:

  • SampleThere are two pre-defined sample categories and each is represented in the sFlow header as an integer sample-type value. Table 2 below shows the sample types and their numeric value.

    Table 2: sFlow Sample Types

    Sample Type

    Integer Value in sFlow Header

    counter-sample

    2

    expanded-counter-sample

    4

    flow-sample

    1

    expanded-flow-sample

    3

    Note

    The difference between the expanded sensor types and the non-expanded sample types is the size of the data fields. The field names and types are the same, but the field sizes are larger in the expanded sample types.

    Packet definitions for these sample types can be found here: sFlow Samples  

    Table 3 shows the other fields contained in an sFlow sample header (by sample type) along with the field type.

    Table 3: sFlow Packet Header Fields

    field type/size in bits

    counter-sample

    flow-sample

    integer/32

    sampleSequenceNumber

    sampleSequenceNumber

    integer/8

    sourceIDType

    • 0 = SNMP interface index

    • 1 = VLAN ID (smonVlanDataSource)

    • 2 = Physical entity (entPhysicalEntry)

    sourceIDType

    • 0 = SNMP interface index

    • 1 = VLAN ID (smonVlanDataSource)

    • 2 = Physical entity (entPhysicalEntry)

    integer/24

    sourceIDValue

    sourceIDValue

    integer/32

    n (the number of sampled records contained in the Counter sample)

    sampleSamplingRate

    integer/32

    -

    samplePool (number of packets that could have been sampled)

    integer/32

    -

    sampleDroppedPackets (number of packets dropped due to lack of resources)

    integer/8

    -

    sampleInputInterfaceFormat (input interface type)

    integer/32

    -

    sampleInputInterfaceValue (input interface (SNMP interface index)

    integer/1

    sampleOutputInterfaceFormat (output interface type)

    integer/33

    -

    sampleOutputInterfaceValue (SNMP interface index)

    integer/32

    -

    n (the number of flow records)

    data

    counter records

    flow records

  • Flow RecordThe Flow Record tab provides the tools needed to define the different types of flow that might be seen in an sFlow capture. Paragon Insights ships with 16 types of pre-defined flow records, each of which have a format number and a sensor path for use in defining sFlow rules, shown in Table 4 below. There are several fields in each type of flow record. These can be seen by selecting the desired record type from the list and clicking the edit (pencil) button.

    Table 4: Flow Record Types

    Record Type

    Format Number

    Sensor Path Value

    raw packet headers

    1

    /sflow-v5/flow-sample/raw-packet-header

    Ethernet frame data

    2

    /sflow-v5/flow-sample/ethernet-frame-data

    IPv4 data

    3

    /sflow-v5/flow-sample/ipv4-data

    IPv6 data

    4

    /sflow-v5/flow-sample/ipv6-data

    extended switch data

    1001

    /sflow-v5/flow-sample/extended-switch-data

    extended router data

    1002

    /sflow-v5/flow-sample/extended-router-data

    extended gateway data

    1003

    /sflow-v5/flow-sample/extended-gateway-data

    extended user data

    1004

    /sflow-v5/flow-sample/extended-user-data

    extended URL data

    1005

    /sflow-v5/flow-sample/extended-url-data

    extended MPLS data

    1006

    /sflow-v5/flow-sample/extended-mpls-data

    extended NAT data

    1007

    sflow-v5/flow-sample/extended-nat-data

    extended MPLS tunnel

    1008

    /sflow-v5/flow-sample/extended-mpls-tunnel

    extended MPLS VC

    1009

    /sflow-v5/flow-sample/extended-mpls-vc

    extended MPLS FEC

    1010

    /sflow-v5/flow-sample/extended-mpls-fec

    extended LVP FEC

    1011

    /sflow-v5/flow-sample/extended-mpls-lvp-fec

    extended VLAN tunnel

    1012

    /sflow-v5/flow-sample/extended-vlan-tunnel

    When you configure rules for sFlow, you can choose from any of these record types. You can create new flow records by clicking the Add (+) button on the Sflow Settings page.

  • Counter RecordThe Counter Record tab provides the definition for the two pre-defined counter record types. There are two types of counter records, ethernet-interface-counters and generic-interface-counters. Generic interface counters are format number 1 and Ethernet interface counters are format number 2. The sensor path for generic interface counters is /sflow-v5/counter-sample/generic-interface-counter. The sensor path for Ethernet interface counters is /sflow-v5/counter-sample/ethernet-interface-counter.

    The fields available within the counter records are the possible errors and the countable statistics such as:

    • frame errors

    • collisions

    • deferred transmissions

    • transmit errors

    • administration status

    • operational status

    • input packets

    • output packets

    • input errors

    • output errors

    • and others

    You can use either the generic interface counter or Ethernet interface counter in rules that you define. The counter sensors can be defined to pick even single fields from either of the available counters. You can create additional counter record types by clicking the Add (+) on the Sflow Settings page (Counter Record tab).

  • ProtocolThe Protocol tab provides a means to define which protocol the sFlow captures contain and allow for the decoding of many network protocols. The fields that are contained in each protocol entry are the same fields as would be seen in a frame or packet of that type. For example, an Ethernet frame would have a destination MAC address, a source MAC address, and an ethernet-next-header-type field. The fields defined in any protocol you want to decode must appear in the protocol definition in the same order as they would appear in the packet or frame.

    The number column that appears is the IANA protocol number assigned to that protocol. For example, the tcp protocol is protocol number 6.

Note

On the Sample, Flow Record, and Counter Record tabs, there is an Enterprise column. This column is for the use of vendor-specific or custom decoding details. For example, a Foundry ACL-based flow sample has the enterprise value 1991, Format 1, includes additional fields specifically for that Foundry flow.In most instances, the Enterprise value is 0.

Extend sFlow Decoding Capabilities

You can extend the protocol decoding capabilities of Paragon Insights by creating additional decoding schemas under the Sample, Flow Record, Counter Record, and Protocol tabs on the sFlow ingest settings page.

Once you have updated the ingest settings, the new rules can make use of them as follows:

  • Use a packet capture utility such as Wireshark to identify the order in which the new protocols appear within the sFlow packets.
  • Ensure that the sensor path you use in the sensor definition is formatted like this: /sflow-v5/<sample-name>/<record-name>/<l2-protocol>/<l3-protocol>, where sample-name is the new sample, record-name is the new counter or flow record, and the l2-protocol and l3-protocol are the new protocols. Again, these protocols must be named and positioned as they appear in the sFlow captures.

Configure sFlow Rules

As with other rule definitions, sFlow rules are made up of sensors, fields, vectors, and so on. An sFlow sensor has a Sensor Name, a Sensor Type of sFlow, and an sFlow Path as shown in Figure 3.

Figure 3: sFlow Sensor Definition
sFlow Sensor Definition

The sensor path serves a big role in sensor definition. Paragon Insights uses the sensor path to define not only the sFlow flow type, but the sample type, record type, protocol, and other custom path elements if needed.

sFlow - Device Configuration

When you configure a device to send sFlow to a collector, you simply set an source IP address, sample-rate, polling-interval, udp-port, interface to capture from, and provide the IP address of the collector. There is no opportunity to filter or choose what data gets sent from the device side.

The following example shows the output from a switch already configured to send sFlow to a collector at IP address 10.204.32.46

OpenConfig

This format utilizes the OpenConfig data model using gRPC. The network device acts as a gRPC server to terminate gRPC sessions from Paragon Insights, which runs as the client. RPC calls from Paragon Insights trigger the creation of sensors that either stream data periodically or report events, which are then funneled onto the appropriate gRPC channel as protobuf (GPB) messages.

In Paragon Insights releases prior to 3.1.0, OpenConfig could only be used on Junos OS and certain Cisco devices. This is because Paragon Insights has native support for the OpenConfig RPCs used by Juniper and Cisco, each of which uses its own proprietary encoding for OpenConfig data. Starting with release 3.1.0, Paragon Insights supports the gRPC Network Management Interface (gNMI) for OpenConfig communication. This protocol allows Paragon Insights to work with many third-party OpenConfig implementations in a vendor independent way.

With OpenConfig, the device receives sensor subscriptions from Paragon Insights and pushes telemetry data through any available interface, whether in-band or out-of-band.

Note that in the figure above, the out-of-band connection is shown as connecting to the fxp0 interface of a Junos device. If the device was from a third-party vendor, the out-of-band connection would be to the vendor-specific management port. Similarly, the in-band connection(s) would be to the vendor-specific interface designation.

OpenConfig RPC

To use OpenConfig format, you configure the device as a gRPC server. With Paragon Insights acting as the client, you define which sensors you want it to subscribe to, and it makes subscription requests towards the device.

Data streamed through gRPC is formatted in OpenConfig key/value pairs in protocol buffer (GPB) encoded messages. Keys are strings that correspond to the path of the system resources in the OpenConfig schema for the device being monitored; values correspond to integers or strings that identify the operational state of the system resource, such as interface counters. OpenConfig RPC messages can be transferred in bulk, such as providing multiple interface counters in one message, thereby increasing efficiency of the message transfer.

For more information on OpenConfig sensors, see Understanding OpenConfig and gRPC on Junos Telemetry Interface.

gNMI-Encoded OpenConfig RPC

gNMI-encoded OpenConfig works much like OpenConfig RPC in that you must set the network device up as an OpenConfig server to which Paragon Insights makes subscription requests. However, gNMI supports more subscription types than Paragon Insights currently supports. Currently, Paragon Insights only supports gNMI STREAM subscriptions in the SAMPLE mode. STREAM subscriptions are long-lived subscriptions that continue, indefinitely, to transmit updates relating to the set of paths configured within the subscription. SAMPLE mode STREAM subscriptions must include a sample_interval.

The messages returned to the client through gNMI are encoded by the device in protobuf, JSON, or JSON-IETF format and cannot be sent in bulk. This, in part, makes gNMI-encoded messaging less efficient than gRPC-encoded messaging.

Warning
  • For JSON or JSON-IETF, it is assumed that the device returns gNMI updates as only leaf values encoded in JSON, rather than returning an entire hierarchy or sub-hierarchy as a JSON object.

  • Numbers encoded in JSON or JSON-IETF are decoded by Paragon Insights as either float64, int64, or string, according to RFC 7159 and RFC 7951. If your OpenConfig rules contain fields that are of a different type, we recommend that you change the field types accordingly.

Junos OS and Cisco devices can be managed by Paragon Insights using gNMI-encoded OpenConfig. If a device does not support gNMI in general, or the STREAM subscription in SAMPLE mode, or does not support an OpenConfig request, it returns one of the following errors:

  • Unimplemented

  • Unavailable

  • InvalidArgument

In the case of a Junos OS or Cisco device, the error causes the connection to fall back to OpenConfig RPC. In the case of a third-party device, the connection simply fails due to the error.

gNMI-encoded OpenConfig can be enabled at the device or device-group level. If enabled at the device-group level, then all devices added to the group use gNMI by default. If enabled (or not enabled) at the device level, then the device level setting takes precedence over the device-group level setting.

Note

During the initial connection gNMI devices attempt to perform an initial sync with the client. The device sends a continuous stream of data until the device and the collector (Paragon Insights) are in sync. After initial sync, the device begins normal streaming operations based on the configured reporting rate. Because of the processing load this creates, Paragon Insights has this feature disabled by default. It can be enabled at the device-group or device level if needed.

For more information about gNMI, see: gRPC Network Management Interface (gNMI).

OpenConfig - Device Configuration

OpenConfig requires:

  • Junos OS Version: 16.1 or later

    • The OpenConfig sensor requires that the Junos device have the OpenConfig and network agent packages installed. These packages are built into Junos OS Releases 18.2X75, 18.3, and later. For releases between 16.1 and 18.2X75 or 18.2, you must install the packages.

      To verify whether you have these packages, enter the following command:

      user@host> show version | match "Junos:|openconfig|na telemetry"

      See Understanding OpenConfig and gRPC on Junos Telemetry Interface for more information.

    • Network agent is not supported on PPC platforms (MX104, MX80, and so on)

  • Required configuration:

    set system services extension-service request-response grpc clear-text port number

Syslog

In addition to the JTI-related options above, starting with Release 2.1.0, Paragon Insights also supports syslog natively as another data collection method, using a data model that aligns with other Paragon Insights ingest mechanisms to provide all the same feature richness.

A device can push syslog messages (when configured) over UDP to the Paragon Insights server either out-of-band through the Routing Engine (RE) using the router’s management interface, or in-band through the Packet Forwarding Engine, that is, directly from a line card.

To use syslog format, you configure the device with settings that include where to send the syslog messages. See Syslog - Device Configuration below for details. When you configure Paragon Insights to start collecting the data, messages are already flowing towards the server.

For more information on syslog as used by Juniper devices, see Junos OS System Log Overview.

Overview

Syslog ingest requires some setup before you can use it as a sensor in a rule:

  • Pattern - A pattern identifies some syslog event; you create a pattern for each event you want to monitor. You can configure patterns for both structured and unstructured events.

  • Pattern set - With the patterns configured, you then group them into a pattern set, which you then reference when defining the syslog sensor settings within a rule.

To see how patterns and pattern sets are used, see Example: Creating a Rule Using Syslog Ingest.

System-Generated Fields

Some fields are common in syslog messages. Paragon Insights extracts these fields and includes them automatically in the raw table, enabling you to make use of them directly when creating a rule, and avoiding the need to configure patterns.

To illustrate use of these values, consider the following example syslog messages:

Structured - <30>1 2019-11-22T03:17:53.605-08:00 R1 mib2d 28633 SNMP_TRAP_LINK_DOWN [junos@2636.1.1.1.2.29 snmp-interface-index="545" admin-status="up(1)" operational-status="down(2)" interface-name="ge-1/0/0.16"] ifIndex 545, ifAdminStatus up(1), ifOperStatus down(2), ifName ge-1/0/0.16

Equivalent unstructured - <30>Nov 22 03:17:53 R1 mib2d[28633]: SNMP_TRAP_LINK_DOWN: ifIndex 545, ifAdminStatus up(1), ifOperStatus down(2), ifName ge-1/0/0.16

System-generated fields:

  • "__log_priority__" - Priority of syslog message

    • In the examples, <30> denotes the priority

  • “__log_timestamp__" - Timestamp in epcoh in the syslog message

    • In the structured example, 2019-11-22T03:17:53.605-08:00 is converted to epoch with -08:00 indicating the time zone

    • In the unstructured example, the time zone from the configuration will be used to calculate epoch

  • "__log_host__" - Host name in the syslog message

    • In the examples, R1 denotes the host name

  • "__log_application_name__” - Application name in the syslog message

    • In the examples, mib2d is the application name

  • "__log_application_process_id__” - Application process ID in the syslog message

    • In the examples, 28633 is the ID

  • "__log_message_payload__" - Payload in the message

    • Structured example - “SNMP_TRAP_LINK_DOWN [junos@2636.1.1.1.2.29 snmp-interface-index="545" admin-status="up(1)" operational-status="down(2)" interface-name="ge-1/0/0.16"] ifIndex 545, ifAdminStatus up(1), ifOperStatus down(2), ifName ge-1/0/0.16”

    • Unstructured example - “SNMP_TRAP_LINK_DOWN: ifIndex 545, ifAdminStatus up(1), ifOperStatus down(2), ifName ge-1/0/0.16”

  • "Event-id" - Denotes the event ID configured in the pattern

    • In the examples, SNMP_TRAP_LINK_DOWN is the event ID

Note

Be sure not to define any new fields using a name already defined above.

Usage Notes

  • If you add a device in Paragon Insights using its IP address (and the address is not the actual host name of the device), you must also add the device’s host name in the Syslog Hostnames field.

  • Multiple device groups can listen for syslog messages using the same port.

  • A configured time zone is not considered when processing structured messages, as the messages themselves include the time zone.

  • Daylight savings time is not currently supported.

Optional Configuration Elements

Configure Syslog Ports

By default, Paragon Insights listens for syslog messages from all device groups on UDP port 514. You can change the system-level syslog port, as well as configure one or more ports per device group. The more specific device group setting takes precedence over the system level setting.

To change the system-level syslog port:

  1. Click Settings > Ingest in the left-nav bar.
  2. Select the Syslog tab on the Ingest Settings page.
  3. On the Syslog page, edit the port number.
  4. Click Save & Deploy.

To configure a syslog port for a device group:

  1. Go to the Configuration > Device Group page and click on the name of a device group.
  2. Click the Edit (pencil) icon.
  3. In the pop-up window, enter the port(s) in the Syslog Ports field.
  4. Click Save & Deploy.

Configure Time Zone for a Device

When a device exports structured syslog messages, time zone information is included within the message. However, unstructured syslog messages do not include time zone information. By default, Paragon Insights uses GMT as the time zone for a device. In these cases you can assign a time zone to a device or device group within Paragon Insights.

To configure a device’s time zone at the device group level:

  1. Go to the Configuration > Device Group page and click on the name of a device group.
  2. Click the Edit (pencil) icon.
  3. In the pop-up window, enter a value in the Timezone field, for example -05:00.

To configure a device’s time zone at the device level:

  1. Go to the Configuration > Device page and click on the name of a device.
  2. Click the Edit (pencil) icon.
  3. In the pop-up window, enter a value in the Timezone field, for example -05:00.
Note

The more specific device setting takes precedence over the device group setting.

Configure Multiple Source IP Addresses for a Device

In cases where syslog messages arrive from a device using a different source IP address than the one originally configured in the Paragon Insights GUI, you can add additional source IP addresses.

To support additional source IP addresses:

  1. Go to the Configuration > Device page and click on the name of a device.
  2. Click the Edit (pencil) icon.
  3. In the pop-up window, enter the IP address(es) in the Syslog Source IPs field.

Configure Host Name Aliases for a Device

When a device has more than one host name, such as a device with dual REs, syslog messages can arrive at the Paragon Insights server with a host name that is not the device’s main host name. In these cases, you can add host name aliases for that device.

Note

If you add a device in Paragon Insights using its IP address, you must also add the host name that will appear in the syslog messages.

To configure additional hostname aliases:

  1. Go to the Configuration > Device page and click on the name of a device.
  2. Click the Edit (pencil) icon.
  3. In the pop-up window, enter the host name(s) in the Syslog Host Names field.

Syslog - Device Configuration

Syslog requires that you have at least the following configuration elements committed on your device(s) in addition to

  • Junos OS Version: Any release

  • Required configuration:

    set system syslog host 110.1x.x0.1 any any
    set system syslog host 10.1x.x0.1 allow-duplicates
    set system syslog host 10.1x.x0.1 structured-data

    ## 10.1x.x0.1 = Paragon Insights server

Example: Creating a Rule Using Syslog Ingest

To illustrate how to configure and use a syslog sensor, consider a scenario where you want to:

  • Monitor interface operational down status

  • Use two syslog events, one structured and one unstructured

  • Use a rule with a trigger to indicate when the interface goes down

To implement this scenario, you will need to complete the following activities:

The workflow is as follows:

CONFIGURE NETWORK DEVICES

Note

This example assumes you have already added your devices into Paragon Insights and assigned them to a device group.

Add Syslog Configuration to the Network Device

If not already done, configure your network device(s) to send syslog data to Paragon Insights. The example device configuration from the previous section is repeated here:

set system syslog host 10.1x.x0.1 any any
set system syslog host 10.1x.x0.1 allow-duplicates
set system syslog host 10.1x.x0.1 structured-data

## 10.1x.x0.1 = Paragon Insights server

SET UP SYSLOG INGEST

Configure Events Patterns

An Event pattern is a configuration to monitor some syslog event; you create a pattern for each event you want to monitor. This example uses patterns to monitor four syslog events (two structured and two unstructured).

Note

See the usage notes at the end of this section for more detail on what has been configured.

  1. In the Paragon Insights GUI, click Settings > Ingest in the left-nav bar.
  2. Select the Syslog tab on the Ingest Settings page.
  3. On the Syslog page, click the plus (+) icon.
  4. In the pop-up window that appears, enter the following parameters for the first pattern, named snmp-if-link-down:
  5. Click Save and Deploy.
  6. Click the plus (+) icon once more and enter the following parameters for the second pattern, named fpc-offline:
    Note

    The full value entered in the Filter field is fpc%{NUMBER:fpc} Marking ports %{WORD:port-status}

  7. Click Save and Deploy. On the Syslog page you should see the two patterns you just created.

Usage notes for the patterns

For structured syslog:

  • The event ID (SNMP_TRAP_LINK_DOWN) references the event name found within the syslog messages.

  • Fields are optional for structured syslog messages; if you don’t configure fields, the attribute names from the message will be treated as field names.

    • In this example, however we have user-defined fields:

      • The field names (if-name, snmp-index) are user-defined.

      • The field interface-name value is an attribute from the syslog message, for example, ge-0/3/1.0; this field is renamed as if-name

      • The field snmp-interface-index value is an attribute from the syslog message, for example, ifIndex 539; this field is renamed as snmp-index

      • The field snmp-interface-index here is defined as an integer; by default the fields extracted from a syslog message are of type string, however type integer changes this to treat the value as an integer

  • The constant section is optional, in this example, we have user-defined constants.

    • The constant name ifOperStatus is user-defined; in this case it has the integer value of '2'

  • Filter configuration is optional for a structured syslog, though you can do so if desired; if used, the filter-generated fields will override the fields included in the syslog message.

  • The key fields section is optional; by default the hostname and event ID will be the keys used by Paragon Insights; add additional key fields here; in this example, we have key-fields, namely interface-name, where the name and value are extracted from the syslog message’s attribute-value pair

For unstructured syslog:

  • The event ID is user defined, this case PSEUDO_FPC_DOWN

    • For example, neither the unstructured syslog Nov 22 02:27:05 R1 fpc1 Marking ports down nor its structured counterpart <166>1 2019-11-22T02:38:23.132-08:00 R1 - - - - fpc1 Marking ports down includes an event ID

  • A filter must be used to derive fields (unlike proper structured syslog); this example uses fpc%{NUMBER:fpc} Marking ports %{WORD:port-status}, where fpc becomes the field name and NUMBER denotes the syntax used to extract the characters out of that particular portion of the message, for example “2”

    • An example of a syslog message that matches the grok filter is “fpc2 Marking ports down”

  • constant fpc-status - has a string value of ‘online’

Regarding filters:

  • By default in a pattern, field and constant values are a string; to treat it as an integer or float, define the pattern’s field type as integer or float

  • For unstructured patterns, you must configure a filter as the messages are sent essentially as plain text and don’t include field info on their own

  • Filters should always be written to match the portion of message after the event ID; this allows the filter to parse a syslog message irrespective of whether it arrives in unstructured or structured format

    • For example, the filter fpc%{NUMBER:fpc} Marking ports %{WORD:port-status} matches both versions of the following syslog message:

      • Structured: <166>1 2019-11-22T02:38:23.132-08:00 R1 - - - - fpc1 Marking ports down

      • Unstructured: Nov 22 02:27:05 R1 fpc1 Marking ports down

Clone an Existing Syslog Events Pattern

Starting with Paragon Insights Release 4.0.0, you can clone an existing Syslog pattern.

To clone an existing Syslog events pattern:

  1. Click Settings > Ingest in the left-nav bar.

    The Ingest Settings page is displayed.

  2. Click the Syslog tab to view the Syslog page.
  3. Click to expand the Events Pattern section in the Syslog Settings page to view existing syslog events patterns.
  4. To clone a pattern for Paragon Insights Release 4.0.0 and 4.1.0, click the Clone icon.

    To clone a particular template for Paragon Insights Release 4.2.0 and later, select the option button next to the name of the template and click Clone.

    The Clone Pattern: <name of syslog pattern> page is displayed.

    From the Clone Pattern: <name of syslog pattern> page, you can

    • Edit existing fields

    • Add new fields or constants

    • Add or remove key fields

  5. Click Save to save configuration and clone the syslog pattern.

    Alternatively, click Save & Deploy to save configuration, clone syslog pattern, and deploy the pattern.

Add Patterns to a Pattern Set

With the patterns configured, group them into a pattern set.

  1. On the Syslog page,click to expand the Pattern Set section and click the plus (+) icon.
  2. In the pop-up window that appears, enter the following parameters:
  3. Click Save and Deploy. On the Syslog page you should see the pattern set you just created.

Clone an Existing Syslog Pattern Set

Starting with Paragon Insights Release 4.0.0, you can clone an existing Syslog pattern set.

To clone an existing Syslog pattern set:

  1. Click Settings > Ingest in the left-nav bar.

    The Ingest Settings page is displayed.

  2. Click the Syslog tab to view the Syslog page.
  3. Click to expand the Pattern Set section in the Syslog page to view existing syslog pattern sets.
  4. To clone a pattern set for Paragon Insights Release 4.0.0 and 4.1.0, click the Clone icon.

    To clone a particular template for Paragon Insights Release 4.2.0 and later, select the option button next to the name of the template and click Clone.

    The Clone Pattern-set: <name of pattern-set> page is displayed.

    From the Clone Pattern-set: <name of pattern-set> page, you can

    • Edit the name and description fields

    • Add or remove patterns from the Patterns field.

  5. Click Save to save configuration and clone the syslog pattern set.

    Alternatively, click Save & Deploy to save configuration, clone syslog pattern set, and deploy the pattern set.

Configure Header Pattern

Starting in Paragon Insights Release 4.0.0, you can configure the pattern for parsing the header portion of a syslog message. With this release, unstructured syslog messages of non-Juniper devices are supported. In earlier releases, you can only parse the payload portion of either a structured syslog message as specified in RFC 5424 standard, or a Juniper device’s unstructured syslog message.

In general, it is assumed that any unstructured syslog message matches the Juniper syslog message pattern. For example, you do not have to configure a Juniper header pattern as this pattern is inbuilt with Paragon Insights. However, in case of a non-Juniper device’s unstructured syslog message that does not match with the inbuilt pattern, a first match is made with one of the user-configured header patterns. Following a successful match, the fields are extracted. When there is no match, the incoming syslog message is dropped.

To configure a header pattern:

  1. Navigate to Settings > Ingest in the left-nav bar.

    The Ingest Settings page is displayed.

  2. Click Syslog to view the Syslog page.
  3. Click to expand the Header Pattern section.

    The Header Pattern section of the Syslog page is displayed. You can add a new header pattern and edit or delete an existing header pattern from this page.

  4. Click the plus (+) icon to add a new header.

    The Add Header Pattern page is displayed.

  5. Enter the following information in the Add Header Pattern page.
    1. Enter a name for the header pattern in the Name field.
    2. Enter a description for the header pattern in the Description field.

      For example, you can provide a one-line description of why you are creating this header pattern.

    3. Enter the filter or regular expression (regex) for the header patter in the Filter field.Note

      You can use regex101.com to edit, validate, and modify the filter pattern you want to add to the header pattern.

      An example of a filter pattern is (.*):([A-Z][a-z]{2} \d{1,2} \d{1,2}:\d{1,2}:\d{1,2}\.\d*)\s:\s([a-z]*)\[(\d*)\]:\s*(.*)\s*.

    4. log-host, log-timestamp, and log-message-payload of the Fields section are mandatory fields that determine the position of the header.

      In the Fields section,

      1. Click log-host, and enter the following information.
        • Enter a name for the log host in the Name field.

          log-host is the default name.

        • Enter a description for log-host in the Description field.

          The default description is Position of host name.

        • Enter the capture group value with prefix $ in the From field.

          The capture group determines from which position in the header the log-host starts.

      2. Click log-timestamp, and enter the following information.
        • Enter a name for the log timestamp in the Name field.

          log-timestamp is the default name.

        • Enter a description for log-timestamp in the Description field.

          The default description is Position of time stamp.

        • Enter the capture group value with prefix $ in the From field.

          The capture group determines from which position in the header the log-timestamp starts.

        Note

        Ensure that timestamp format follows this sample timestamp format: “Jan _2 15:04:05 2006”. Otherwise parsing of syslog messages will lead to an undefined behaviour.

      3. Click log-message-payload, and enter the following information.
        • Enter a name for the log message payload in the Name field.

          log-message-payload is the default name.

        • Enter a description for log-message-payload in the Description field.

          The default description is Position of payload.

        • Enter the capture group value with prefix $ in the From field.

          The capture group determines from which position in the header the log-message-payload starts.

      4. (Optional) Click the plus (+) icon to add a new field.
        • Enter a name for the new field in the Name field.

        • Enter a description for the new field in the Description field.

        • Enter the capture group value with prefix $ in the From field.

          The capture group determines from which position in the header the new field starts.

        You can add one or more than one fields by clicking the plus (+) icon.

  6. Enter the name(s) of key fields in the Key Fields text box.
  7. Click Save to save configuration, or click Save & Deploy to save and immediately deploy the configuration.

    Alternatively, to cancel the configuration, click Cancel.

Editing an Existing Header Pattern

To edit an already configured header pattern:

  1. Navigate to Settings > Ingest in the left-nav bar.

    The Ingest Settings page is displayed.

  2. Click Syslog to view the Syslog page.
  3. Click to expand the Header Pattern section.

    The Header Pattern section of the Syslog Setting page is displayed.

  4. Select the header pattern you want to edit by selecting the check box next to the name of the header pattern, and click the Edit (pencil) icon.

    The Edit Header Pattern Header Name page is displayed.

  5. After you have edited the required fields, click Save to save configuration.

    You can also click Save & Deploy to save and immediately deploy the edited configuration.

CREATE RULE, APPLY PLAYBOOK

Configure a Rule Using the Syslog Sensor

With the syslog ingest settings complete, you can now create a rule using syslog as the sensor.

This rule includes three elements:

  • A syslog sensor

  • Four fields capturing data of interest

  • A trigger that indicates when the interface goes down

Note

See the usage notes at the end of this section for more detail on what has been configured.

  1. Click Configuration > Rules in the left-nav bar.
  2. On the Rules page, click the + Add Rule button.
  3. On the page that appears, in the top row of the rule window, set the rule name. In this example, it is check-interface-status.
  4. Add a description and synopsis if you wish.
  5. Click the + Add sensor button and enter the following parameters in the Sensors tab:
  6. Now move to the Fields tab, click the + Add field button, and enter the following parameters to configure the first field, named event-id:
  7. Click the + Add field button again and enter the following parameters to configure the second field, named fpc-slot:
  8. Click the + Add fieldbutton again and enter the following parameters to configure the third field, named if-name:
  9. Click the + Add field button once more and enter the following parameters to configure the fourth field, named snmp-index:
  10. Now move to the Triggers tab, click the + Add trigger button, and enter the following parameters to configure a trigger named link-down:
  11. At the upper right of the window, click the + Save & Deploy button.

Usage Notes for the rule

  • Sensor tab

    • The sensor name if-status-sensor is user-defined

    • The sensor type is syslog

    • Pattern set check-interface-status - reference to the pattern set configured earlier

    • If not set, the Maximum hold period defaults to 1s

  • Fields tab

    • Four fields are defined; although the patterns are capturing more than four fields of data, this example defines four fields of interest here; these fields are used in the trigger settings

    • The field names (event-id, fpc-slot, if-name, snmp-index) are user-defined

    • path event-id - default field created by syslog ingest in the raw table; references the field from the pattern configuration

    • path fpc - references the value from the filter used in the unstructured pattern configuration

    • path if-name - references the field from the pattern configuration

      • Data if missing all interfaces - if the if-name value is not included in the syslog message, use the string value “all interfaces”

    • path snmp-index - references the field from the pattern configuration

  • Triggers tab

    • The trigger name link-down is user-defined

    • frequency 2s - Paragon Insights checks for link-down syslog messages every 2 seconds

    • term is-link-down - when $event-id is like SNMP_TRAP_LINK_DOWN, in any syslog message in the last 300 seconds, make red and show the message Link down for $if-name(snmp-id: $snmp-index)

      • $event-id - $ indicates to reference the rule field event-id

      • Link down for $if-name(snmp-id: $snmp-index) - for example, “Link down for ge-2/0/0 of FPC 2”

      • $if-name - references the field value, i.e., the name of the interface in the syslog message

    • term is-fpc-down - when $event-id is like PSEUDO_FPC_DOWN, in any syslog message in the last 300 seconds, make red and show the message Link down for $if-name of FPC$fpc-slot

      • $event-id - $ indicates to reference the rule field event-id

      • $if-name - “all interfaces”

      • Link down for $if-name of FPC$fpc-slot - for example, “Link down for all interfaces of FPC 2”

Add the Rule to a Playbook

With the rule created, you can now add it to a playbook. For this example, create a new playbook to hold the new rule.

  1. Click Configuration > Playbooks in the left-nav bar.
  2. On the Playbooks page, click the + Create Playbook button.
  3. On the page that appears, enter the following parameters:
  4. Click Save & Deploy.

Apply the Playbook to a Device Group

To make use of the playbook, apply it to a device group.

  1. On the Playbooks page, click the Apply (Airplane) icon for the check-interface-status playbook.
  2. On the page that appears:
    • Enter a name

    • Select the desired device group

    • Click Run Instance

  3. On the Playbooks page, confirm that the playbook instance is running. Note that the playbook may take some time to activate.

Monitor the Devices

With the playbook applied, you can begin to monitor the devices.

  1. Click Monitor > Device Group Health in the left-nav bar, and select the device group to which you applied the playbook from the Device Group pull-down menu.
  2. Select one of more of the devices to monitor.
  3. In the Tile View, the tile labeled external contains the parameters from the rule you configured earlier.
Note

For this example, since the rule trigger does not include a ‘green’ term, the status will show red when there is an issue and otherwise will show gray (no data).

SNMP Trap and Inform Notifications

Paragon Insights Release 4.0.0 supports inform and trap notifications that are sent by devices in the network for fault management. Traps and informs are notifications about change of state in network that are sent between the SNMP manager (Paragon Insights) and the SNMP agents (devices), on which Paragon Insights performs trigger evaluations. Paragon Insights processes traps and informs from the configured device only if a playbook containing an SNMP-notification rule is running for the specified device. In all other cases, the trap or inform message is dropped by the SNMP Manager.

The following sections describe relevant terms, configuration of traps and informs through CLI, port configuration, and accessing status of SNMP traps through CLI.

Note

SNMP trap notifications are supported by SNMPv2c and SNMPv3. SNMP inform messages are supported only when you use SNMPv3 protocol.

Glossary

The following terms are used when describing processes or concepts related to SNMP traps and informs.

  • Authoritative agent — In SNMPv3 transactions between two entities (agent and manager), the flow of sending notification is controlled through authentication and privacy that are unique features in SNMPv3.Authentication identifies and verifies the source of an SNMPv3 message. The privacy feature prevents packet analyzers from snooping the content of messages by encrypting them.

    The entity that controls the notification flow is known as authoritative agent. In SNMPv3, the non-authoritative entity must know the <Engine ID> of the authoritative agent for a successful communication.

  • Traps or trap messages — A trap is an unacknowledged notification sent from an SNMP agent to the SNMP manager. In trap messages, SNMP agent is the authoritative agent. The administrator must configure the SNMP v3 <user> (distinct from the local IAM users) and <Context Engine ID> on the device that sends out the trap messages. For traps, the <Context Engine ID> is set to the Engine ID that uniquely identifies the SNMP agent.

  • Informs or inform messages — An inform is also a notification sent from an SNMP agent to the SNMP manager. In inform messages, SNMP manager is the authoritative agent. The configuration is done on the device that needs to send inform messages, with the details of the remote authoritative agent, SNMP manager. The administrator must configure the <user> found in the remote SNMP manager.

  • Engine ID<Engine ID> is a hexadecimal generated for a given agent that uniquely identifies the SNMP agent and needs to be unique across a given administrative domain. It also must be persistent across reboots or upgrades.

  • Security Engine ID — It is a security parameter in the SNMP communication between the agent and the manager. It is usually set to the <Engine ID> of the authoritative agent involved. A trap message has two parts: a header and a trap Protocol Data Unit (PDU). The header contains the <Security Engine ID> and a <username> set in the trap configuration. When an agent sends a trap, these parameters in the trap header are checked against the details stored in the USM table. The trap is further processed only when there is a match.

  • Context Engine ID<Context Engine ID> is part of a trap PDU. It uniquely identifies a device which has sent the original trap message. <Context Engine ID> and <Security Engine ID> are identical is most cases.

  • USM Table — SNMP managers receiving the traps needs to maintain the USM table (User-based Security Model) which has <Security Engine ID> and <username> as the key to verify the source of the trap messages.

  • Virtual IP Address for SNMP Proxy — Paragon Insights is deployed as a Kubernetes application and it uses load balancers that exposes virtual IP addresses to external entities. When messages are routed through the load balancer, the source IP address of the trap message will be set to the IP address of the load balancer.

    The load balancer has an option to retain the source IP address if you allot an exclusive virtual IP address for services that require the source IP address of SNMP agents to be preserved. Since in SNMP notifications, the source IP address is required for parsing the message, an exclusive virtual IP must be allotted for SNMP Proxy. The same virtual IP needs to be configured on the device or device groups as the target address.

Events such as interface flap in your network can create multiple inform or trap notifications. To determine the order of the notifications, Paragon Insights must capture the timestamp of devices sending SNMP notifications in nanosecond. However, SNMP protocol records the device timestamp (in seconds) as the notifications reach Paragon Insights. As SNMP notifications are sent through UDP, there is also no guarantee of the order of SNMP notifications.

To enable you to capture the order of SNMP notifications, the following fields are parsed from the SNMP Header of the notification packets.

  • Message ID: Used by devices to identify the SNMP message and match responses to the message. Message ID can be found in the SNMP Header.

    When you create a rule for SNMP trap or inform notifications, use the path __msg_id__ in a field to collect message ID.

  • Request ID: Used to identify the SNMP protocol data unit (PDU). Request ID can be found in the trap or inform PDU.

    When you create a rule for SNMP trap or inform notifications, use the path __req_id__ in a field to collect request ID.

  • SNMP Version: Identifies the version number of the SNMP notification (2 or 3).

    When you create a rule for SNMP trap or inform notifications, use the path __version__ in a field to collect version number.

  • SNMP PDU Type: Identifies the type of SNMP notification (v2c trap or inform request).

    When you create a rule for SNMP trap or inform notifications, use the path __pdu_type__ in a field to collect the PDU type.

    The SNMP PDU type field data captured in the time-series database shows v2c for all SNMP trap notifications. You can check the version number to determine if the trap is generated using SNMP v2c or SNMP v3.

Configurations

The following sections detail how to:

Find the Engine Id

Depending on if you configure devices to send trap or inform notifications, you need to first find the <Engine ID> of either the SNMP agent. You can refer to the sample commands below to find the engine id in Junos devices.

Note

The CLI command to find <Engine ID> varies from vendor-to-vendor.

To find the <Engine ID> of SNMP agents (devices) that are Junos-based platforms, enter the following command in CLI.

show snmp v3 engine-id

You will receive a HEX output as the device <Engine ID>.

Trap Configurations

You can configure a device to send trap notifications using SNMPv2c and SNMPv3.

The source IP address needs to be unique across all the devices as it uniquely identifies the device. The source IP address can only be configured under device while community name can be configured under both device and device group.

Note

In Paragon Insights, the SNMPv2c and SNMPv3 ingest and trap configurations share the same workflow.

To configure SNMP trap notifications at the device level:

  1. Click the Configuration > Device option in the left navigation bar.
  2. Click the add device button (+).
  3. Enter the necessary values in the text boxes and select the appropriate options for the device.

    The following table describes the attributes in the Add a Device window:

    Table 5: Add Device(s) Page Details

    Attributes

    Description

    Name

    Name of the device. Default is hostname. (Required)

    Hostname / IP Address / Range

    Hostname or IP address of a single device. If you are providing a range of IP addresses, enter the IP address for the device that marks the start and end of the address range. (Required)

    SNMP

    SNMP Port Number

    Port number required for SNMP. The port number is set to the standard value of 161.

    SNMP Version

    Select either v2c or v3 in the drop-down menu.

    SNMP Community

    This field appears if you selected v2c in SNMP Version field.

    Enter an SNMP Community string if you configure SNMPv2c for trap notifications.

    In SNMPv2c, Community string is used to verify the authenticity of the trap message issued by the SNMP agent (devices such as routers, switches, servers, and so on).

    SNMPv3 Username

    This field appears if you selected v3 in SNMP Version field.

    Enter a username for trap notifications.

    Authentication None

    This field appears if you selected v3 in SNMP Version field.

    Enable this option on if you want to set SNMPv3 authentication to None.

    Protocol None

    This field appears if you selected v3 in SNMP Version field.

    Enable this option on if you want to set SNMPv3 protocol to None.

    SNMPv3 Authentication Protocol

    This field appears if you selected v3 in SNMP Version field and disabled Authentication None.

    Select an authentication protocol from the drop-down menu.

    SNMP authentication protocol hashes the SNMP username with the passphrase you enter. The hashed output is sent along with the trap notification message. Paragon Insights again hashes the username with the passphrase you entered for authentication. If the output matches, the trap notification is further processed.

    SNMPv3 Authentication Passphrase

    This field appears if you selected v3 in SNMP Version field and disabled Privacy None.

    Enter a passphrase for SNMPv3 authentication.

    SNMPv3 Privacy Protocol

    Select a privacy protocol from the drop-down menu.

    Privacy algorithm encrypts the trap notification message with the protocol passphrase so that the message cannot be read by an unauthorized application in the network.

    SNMPv3 Privacy Passphrase

    This field appears if you selected v3 in SNMP Version field and disabled Privacy None.

    Enter a passphrase to encrypt the trap notification.

    Context Engine ID

    This field appears if you selected v3 in SNMP Version field.

    The Engine ID must be set to engine-id of the SNMP agent.

    Source IP Address

    Enter the source IP address of the device.

    This field is mandatory for SNMPv2c traps and optional for SNMPv3 traps and informs.

    If you use NAT or an SNMP Proxy, the virtual IP address you configure for the SNMP Proxy must be set as the source IP address.

    To set virtual IP address for SNMP Proxy, go to Settings > Deployment in the left navigation bar. In the Loadbalancer page, enter the virtual IP adres and click Save and Deploy button.

  4. Click Save to commit the configuration or click Save and Deploy to deploy the configuration in Paragon Insights.

To configure SNMP trap notifications at the device-group level:

  1. Click the Configuration > Device Group option in the left navigation bar.
  2. Click the add group button (+).
  3. Enter the necessary values in the text boxes and select the appropriate options for the device group.

    The following table describes the attributes in the Add a Device Group window:

    Table 6: Add Device Group Page Details

    Attributes

    Description

    Name

    Name of the device group. (Required)

    Description

    Description for the device group.

    Devices

    Add devices to the device group from the drop-down list. (Required)

    Starting in Paragon Insights Release 4.0.0, you can add more than 50 devices per device group. However, the actual scale of the number of devices you can add depends on the available system resources.

    For example, consider that you want to create a device group of 120 devices. In releases earlier than release 4.0.0, it is recommended that you create three device groups of 50, 50, and 20 devices respectively. With Paragon Insights Release 4.0.0, you just create one device group.

    SNMP

    SNMP Port Number

    Port number required for SNMP. The port number is set to the standard value of 161.

    SNMP Version

    Select either v2c or v3 in the drop-down menu.

    • If you select v2c, the SNMP Community name field appears. The string used in v2c authentication is set to public by default. It is recommended that users change the community string.

    • If you select v3, you are given an option to set username, authentication and privacy methods, and authentication and privacy secrets.

    Notification Ports

    Enter notification ports separated by comma.

    Paragon Insights listens on these notification ports for traps and inform notification messages from device groups.

    SNMP Community

    This field appears if you selected v2c in SNMP Version field.

    Enter an SNMP Community string if you configure SNMPv2c for traps and ingest.

    In SNMPv2c, Community string is used to verify the authenticity of the trap notification messages issued by the SNMP agent.

    SNMPv3 Username

    This field appears if you selected v3 in SNMP Version field.

    Enter a username for trap notifications.

    The USM configuration configured under device-groups is shared among devices configured under the same device-group.

    Authentication None

    This field appears if you selected v3 in SNMP Version field.

    Enable this option on if you want to set SNMPv3 authentication to None.

    Privacy None

    This field appears if you selected v3 in SNMP Version field.

    Enable this option on if you want to set SNMPv3 privacy protocol to None.

    SNMPv3 Authentication Protocol

    This field appears if you selected v3 in SNMP Version field and disabled Authentication None.

    Select an authentication protocol from the drop-down menu.

    SNMP authentication protocol hashes the SNMP username with the passphrase you enter. The hashed output is sent along with the trap notification message. Paragon Insights again hashes the username with the passphrase you entered for authentication. If the output matches, the trap notification is further processed.

    SNMPv3 Authentication Passphrase

    This field appears if you selected v3 in SNMP Version field and disabled Privacy None.

    Enter a passphrase for SNMPv3 authentication.

    SNMPv3 Privacy Protocol

    Select a privacy protocol from the drop-down menu.

    Privacy algorithm encrypts the trap notification message with the protocol passphrase so that the message cannot be read by an unauthorized application in the network.

    SNMPv3 Privacy Passphrase

    This field appears if you selected v3 in SNMP Version field and disabled Privacy None.

    Enter a passphrase to encrypt the trap notification.

    Logging Configuration

    SNMP Notification

    Paragon Insights Release 4.0.0 supports collecting log data for SNMP notification. You can collect different severity levels of logs for snmp-notification service in a device group.

    Use these fields to configure which log levels to collect:

    Global Log LevelFrom the drop-down list, select the level of the log messages that you want to collect for every running Paragon Insights service for the device group. The level is set to error by default.
    Log Level for specific servicesSelect the log level from the drop-down list for any specific service that you want to configure differently from the Global Log Level setting. The log level that you select for a specific service takes precedence over the Global Log Level setting.
  4. Click Save to commit the configuration or click Save and Deploy to deploy the configuration in Paragon Insights. For information on how to use the device group cards, see Monitor Device and Network Health.

SNMPv3 Inform Configurations

To enable devices to send inform notifications, you must configure SNMPv3 USM user(s).

To create USM users in Paragon Insights:

  1. Navigate to Settings > Ingest.
  2. Select the SNMP Notification tab on the Ingest Settings page.
  3. Click the Usm Users section.
  4. Click the plus (+) icon to add a USM user.
  5. In the Add USM User page, enter the username, select the authentication and the privacy protocols, and enter passphrases.

    If you enabled Authentication None and Privacy none, the protocol menu and passphrase fields do not appear.

  6. Click Save to only save the configuration and Save and Deploy to deploy the configuration in Insights.

After adding USM users, you can configure the following details in the Add Device(s) page in Device Configuration or Add Device Group page in Device Group Configuration.

Table 7: SNMP Configuration for Informs

Attributes

Description

SNMP

SNMP Port Number

Port number required for SNMP. The port number is set to the standard value of 161.

SNMP Version

Select v3 in the drop-down menu.

Notification Ports (Device Groups only)

Enter notification ports separated by comma.

Paragon Insights listens on these notification ports for traps and inform notification messages from device groups.

Context Engine ID (Devices only)

This field appears if you selected v3 in SNMP Version field.

The Engine ID must be set to engine-id of the SNMP agent.

Source IP Address (Devices only)

This field appears if you selected v3 in SNMP Version field.

Enter the source-IP-address of the device.

If you use NAT or an SNMP Proxy, the virtual IP address you configure for the SNMP Proxy must be set as the source IP address.

To set virtual IP address for SNMP Proxy, go to Settings > Deployment in the left navigation bar. In the Loadbalancer page, enter the virtual IP address and click Save and Deploy.

Starting with Paragon Insights 4.2.0, you can collect _device_timestamp_ as a raw data metric. Device timestamp (in seconds) denotes the time when the inform notification is sent from a device.

Note

Device timestamp for an inform notification is captured by the SNMP sensor only if you configured authentication for an inform message.

The device timestamp data for SNMP trap notifications are the same as the received time metric in Paragon Insights. The device timestamp for traps and the time metric for trap notifications denote the time when the SNMP notification is received by Paragon insights.

Port Configuration

By default, Paragon Insights listens for traps and informs in the standard SNMP trap port 162. This can be changed if needed either at the global level which is applicable to all device groups or at the device group level applicable to a specific device group.

Port configured under ingest will apply to all device groups. Trap and Inform messages received through any other port are discarded.

To configure port number at the ingest level:

  1. Navigate to Settings > Ingest in the left-nav bar.
  2. Select the SNMP Notification tab on the Ingest Settings page.
  3. In the Port section, enter the port number.
  4. Click Save to only save the configuration and Save and Deploy to deploy the configuration in Insights.

Port configured under device group will apply to only a specific device group. Traps and informs received through any other port are discarded. To configure port numbers at the device group level, see Table 6.

Rule Configuration

Once the device is configured to send traps or inform notification, you must configure a rule on the device with SNMP trap so that, Paragon Insights can process traps from the device. In device groups, you can apply a playbook instance that has the snmp-notification rule. When you configure SNMP notification in any rule, you must select the MIB name you want to monitor. Go to Juniper MIBS Explorer to browse MIB files for Junos devices and Cisco MIBS Locator to browse MIB files for Cisco devices.

The following example shows how you can configure a rule with SNMP notification to send alerts if an interface comes up for the chassis.interfaces/ topic.

Note

It is assumed that you have configured the device or device group for SNMP trap notification. See Paragon Insights Pull-Model Ingest Methods to configure SNMP trap notifications in devices or device groups.

To configure a rule under topic system.trap/:

  1. Go to Configuration > Rules.
  2. Click the Add Rules button in the Rules page.

    Enter the rule name in the topic/rule-name format in the Rule field and description in the Description field. For example, chassis.interfaces/linkup.

  3. Click Add Sensor button in Sensors tab.
  4. Enter a name in the Sensor Name field and select SNMP Notification from the drop-down menu in Sensor Type.
  5. Enter notification name in MIB-Name::Notification Name format.

    For example, IFMIB::linkDown.

  6. Click Add Field button in the Fields tab.

    The fields for SNMP Notification rule can be derived as described:

    • Variables (varbinds) for the given trap.

      The variables of the trap can be defined as fields. The following steps use the example IfAdminStatus as varbind and IF-MIB:linkDown as the snmp-notification.

      1. Enter IfAdminStatus in the Field Name.

      2. Select Integer as Field Type.

        The Field Type you enter in the GUI must be same as the type defined in the MIB File.

      3. Select Sensor as Ingest Type (field soruce).

        The Ingest Type (field source) must be set to sensor.

      4. Select the sensor name from the drop-down menu under Sensor.

        The sensor name is the name you entered for the snmp-notification sensor.

      5. Enter IfAdminStatus as sensor path.

        The Path must be set the to the variable (varbind) name defined in the MIB file.

      To add a second field for IfOperStatus as variable (varbind) for a given snmp-notification, follow the steps described here but change the field name and sensor path to IfOperStatus.

  7. Click Save to commit the rule or Save & Deploy to deploy the rule in Paragon Insights.

    You can see the new topic name and rule in the list of existing rules.

    You can also configure triggers or functions based on the fields you add. See how to create a new rule in GUI as explained in Paragon Insights Rules and Playbooks.

You must include this rule in a playbook and apply the playbook instance on a device or device group.

To check the new SNMP notifications sent by device groups, log into Paragon Insights server as a root user and type the following command.

healthbot cli --device-group healthbot -s influxdb

You can track new entries sent by SNMP trap notifications to the Paragon Insights server for the fields (for example, IfAdminStatus) you configured.

Release History Table
Release
Description
Starting in Paragon Insights Release 4.0.0, you can clone an existing NetFlow template.
Starting with Paragon Insights Release 4.0.0, you can clone an existing Syslog pattern.
Starting with Paragon Insights Release 4.0.0, you can clone an existing Syslog pattern set.
Starting in Paragon Insights Release 4.0.0, you can configure the pattern for parsing the header portion of a syslog message.
Paragon Insights Release 4.0.0 supports inform and trap notifications that are sent by devices in the network for fault management.
Starting in Paragon Insights Release 4.0.0, you can add more than 50 devices per device group.
Paragon Insights Release 4.0.0 supports collecting log data for SNMP notification.
Starting with Paragon Insights Release 3.2.0, Paragon Insights supports sFlow (v5)
Starting with release 3.1.0, Paragon Insights supports the gRPC Network Management Interface (gNMI) for OpenConfig communication.
Starting with Release 3.0.0, Paragon Insights (formerly HealthBot) supports NetFlow natively as another ingest method
starting with Release 2.1.0, Paragon Insights also supports syslog natively as another data collection method
In Paragon Insights releases prior to 3.1.0, OpenConfig could only be used on Junos OS and certain Cisco devices.