Device Traffic Probe (aka Headroom probe)


As of version 3.3.0, Headroom Probe is renamed Device Traffic probe. This probe is enabled automatically after blueprint creation.

One of the most useful default probes is the Device Traffic Probe (previously known as Headroom Probe), which provides helpful insights about link capacity between two points in the network. This probe is now extended to provide multiple interface counters (rx, tx, discard, errors etc.) for all managed devices. By default, All counters are collected realtime every 5 seconds. The probe provides utilization on a per port and aggregated utilization per system basis, and raises an anomaly in case of rule violation.

To manually enable this probe, navigate to Analytics > Probes > Create Probe > Instantiate Predefined Probe, and select Device Traffic as per below image.


The predefined probe has the following parameters that you can configure at creation time or update subsequently:

  • Probe Label: Custom name for the probe.
  • Interface counters average period: The average period duration for interface counters in seconds. sDefault value = 2 Minutes.
  • Enable Interface counters history: Maintain historical interface counters data.
  • Interface counters history retention period: Duration to maintain historical interface counters data. Default value = 30 Days.
  • Enable system counters history: Maintain historical system interface counters data.
  • System interface counters history retention period: Duration to maintain historical system interface counters data.**sDefault value** = 30 Days.


User can change the probe inputs but if user changes the probe processor then the probe is not a predefined probe anymore and the traffic layer view will disappear. See Topology Selection Views for more information about traffic layer view.

Once the probe has been created, you can click it to see a breakdown of the probe. Click the desired option on the left side, you can select the processors Live Interface Counters, System Interface Counters.


Live Interface Counters processor

Purpose: Wires in Interface traffic counters every 5 seconds (by default) for all managed devices and keeps historical data based on retention period specified during probe creation.

Outputs of “Live Interface Counters” processor

‘Average Interface Counters’: set of interface counters samples (for each port of each managed device) based on specified average time with historical data.



Use Query All to filter outputs based on any value in the table.

Enable Real Time to filter real time outputs based on any value in the table for a particular aggregation time in history.


‘Live Interface Counters’: set of live interface counters samples (for each port of each managed device).

System Counters processor

Purpose: This processor consumes in ‘Average Interface Counters’ for calculating Interface Counters per system with historical data. It uses properties rx_bps_average, rx_utilization_average, tx_bps_average, and tx_utilization_average to compute the system TX and RX utilization and to compute headroom between the specified source and destination systems.

Outputs of “System Counters” processor

‘System Interface Counters’: set of system interface counters samples (for each device of managed devices) indicating Aggregated TX/RX, Aggregated TX/RX %, and Max interface TX/RX utilization %.



System Level RX/TX Calculation: It aggregate the Tx/RX of all the device interfaces that are “up”.


Max Interface RX/TX Calculation: This is the device interface with the highest Rx and the device interface with highest Tx.

Traffic between a Source and Destination

To visualize traffic between a particular source and destination, first click on System Interface Counters, Check the Box Show Context, and Choose Source and Destination.


The probe uses different colours to describe the remaining link capacity, where greenmeans plenty of capacity and red means that our link is running out of capacity.

The following capture displays an example of a 100G with 70.1Gbps available between a leaf and another leaf showing two available paths via spine1 and spine2.


The next example displays capacity available between two endpoints in a network. It is showing an example of a 100G with 70.1Gbps available between two servers.


Probe Disk Consumption

The web interface displays the amount of disk space utilized by each probe. To view the disk space utilized by probe, go to specific processor output.



In order to ensure there is enough controller disk space to store the probe for the desired retention time, please work with the Support team. With insufficient disk space, older telemetry data files will automatically be deleted as a safety mechanism.

The Device Traffic probe first identifies all the links (across all ECMP paths) that may carry the traffic between the two nodes (source and target). It then calculates headroom for each of them. Headroom is defined as difference between the link capacity and the actual traffic. It then calculates minimum headroom per each path as the smallest headroom across all the links in the path. The semantic of this value is that if you send additional traffic from the source node (new app) which is smaller than path minimum headroom the path will be able to support it, otherwise there will be packet loss. But with ECMP you don’t know which path the traffic will take. To address that, probe then calculates minimum headroom across all ECMP paths. The semantic of this value is that if you send additional traffic from the source which is less then this value, it will make it to the target without the packet loss regardless of which path is taken. Probe also calculates the maximum headroom across all the paths, which is the maximum value among all the minimum path headroom values, for each ECMP path. The semantic of this value is that if you add more traffic from the source (new app), there is NO path that can support it therefore you are guaranteed packet loss. And if you add more traffic from the source which is between minimum and minimum headroom it may or may not be able to make it without a loss, i.e. the performance is unpredictable. The smaller the difference between these two values is, the more predictable the system is, which is why proper ECMP balancing is so important. And all of these calculations are up to date and in sync in presence of any topology change (addition/removal of leafs/links).

For more information, see IBA.