Processor: Service Data Collector

The Service Data Collector processor collects data from the specified service. For example, ‘bgp’ service would be the status of BGP sessions. Objects to be monitored are configured via the graph query and key. In the BGP example, key should evaluate to locallp, localAs, remoteIp, or remote As. For interface-based services such as ‘interface’ and ‘lldp’, key is an interface name.

Input Types - No inputs. This is a source processor.

Output Types - Discrete-State-Set (DSS), Number-Set (NS)


Graph Query (graph_query)

One or more queries on graph specified as strings, or a list of such queries. (String will be deprecated in a future release.) Multiple queries should provide all the named nodes referenced by the expression fields (including additional_properties). Graph query is executed on the “operation” graph. Results of the queries can be accessed using the “query_result” variable with the appropriate index. For example, if querying property set nodes under name “ps”, the result will be available as “query_result[0][“ps”]”.

In collector processors (*_collector, if_counter) it is used to choose a set of nodes for further processing (for example, all leafs, or all interfaces between leaf and spines)

In other processors it is used for general parameterization and it is only supported as a list of queries.

Fabric Interfaces Example
   graph_query: "node("system", role="leaf", name="system").
                 node("interface", name="iface").out("link").
                 node("link", role="spine_leaf")"
Leafs and Spines using two queries Example
   graph_query: ["node("system", role="leaf", name="system")",
                 "node("system", role="spine", name="system")"]
Service name (service_name)
Name of the custom collector service.
List of property names which values will be used as a key parameters for the service. Expression mapping from graph query to whatever key is necessary for the service. For lldp it’s a string with interface name. For bgp it’s a tuple like (src_addr, src_asn, dst_addr, dst_asn, vrf_name, addr_family), where addr_family should be one of ipv4, ipv6, or evpn. For interface it is a string with interface name.
System ID
Expression mapping from graph query to a system_id, e.g. “system.system_id” if “system” is a name in the graph query.
Additional Keys
Each additional key/value pair is used to extend properties of output stages where value is considered as an expression executed in context of the graph query and its result is used as a property value with respective key. The value of this property is evaluated for each item to associate items with metrics provided by a corresponding collector service. The association is done by keys because each collector reports a set of metrics where each metric is identified by a key in a format that is specific for each collector.
Enable Streaming (enable_streaming)
Makes samples of output stages streamed if enabled. An optional boolean that defaults to False. If set to True, all output stages of this processor are streamed in the generic protobuf schema.

Service Data Collector Example

service_name: "interface"
graph_query: "node("system", name="system").out("hosted_interfaces").
              node("interface", name="iface").out("link").
              node("link", role="spine_leaf")"
system_id: "system.system_id"
key: "interface.if_name"
role: "system.role"

In this example, we create a DSS that has an entry for every fabric interface in the system. Each entry is implicitly tagged by “system_id” and “key” (where key happens to be the interface name for the interface service). Furthermore, as we have specified an additional property “role”, each entry is also tagged by system role.

Sample Data
[system_id=spine1,role=spine,key=eth0]: "up"
[system_id=spine2,role=spine,key=eth1]: "down"
[system_id=leaf0,role=leaf, key=swp1]: "up"