Understanding the Custom Collector Pipeline

A Custom Collector operates as a telemetry pipeline—a sequential chain of operations that processes data through interconnected components called nodes. Each node executes a specific task on the incoming data and then passes the results to the next node in the chain. The pipeline includes six distinct node types, with each one performing a single function on the collected data. By connecting these nodes, you can create a continuous data flow that transforms raw ingested data into customized output based on their specifications.

The Telemetry Pipeline includes the following node types:

Table 1: Node Descriptions
Node	Description
	The XML Transfer Node executes the CLI command on devices and extracts fields from the output. Each extracted field forms an accessor definition, mapping to an individual XML path. Accessors identify fields of interest by specifying the path or location in the data source.
	The Select Filter Node transforms an input set of rows into a modified output set of rows by applying expressions to generate new fields. When directing output to a service sink node, you must ensure the output contains two dictionary selectors: keys and values. Each dictionary item is an expression that maps a field to the service schema.
	The Condition Node evaluates a user-defined Boolean expression for each incoming row and keeps only the entries that return true. This process effectively filters the stage before it flows to downstream nodes. Use this node to narrow large inputs to only the relevant data, enabling Latenodes to operate on a focused subset or distinct branches of the pipeline.
	The Group Node groups incoming rows by selected fields and calculates aggregates such as count, sum, or average for each group. The output includes one column for each aggregate. When you configure multiple groups, the node applies the groups sequentially, starting from the outermost (first) and proceeding to the innermost (last), with aggregates computed at the innermost level. Use this node to convert raw, row-level telemetry into concise summaries for devices, interfaces, and neighbors. This node enables downstream processors and dashboards to handle rolled-up metrics instead of high-volume details
	The Join Node connects rows from two input data sets by applying a join condition. Use this node after ingesting data from two XML Transform nodes. You can assign aliases to each input, select an Inner, Left, or Right join type, and keep rows where the Boolean join condition is `true`.
	The Service Sink Node publishes the collector pipeline’s results to a telemetry service (telemetry database). Only one service node is permitted per pipeline.

Note: At minimum, a collector must include three nodes: Xml Transform Node, Select Node, and Service Sink Node.

We've explained the details of a custom collector pipeline. Next, we'll walk through a step-by-step example to build a custom telemetry service for BFD monitoring. Continue to Create a Custom Telemetry Collector to begin.