Apstra IBA Getting Started Tutorial

This tutorial gives the Apstra Customer with an existing, deployed AOS Blueprint instructions to get started with AOS Intent Based Analytics (IBA) with pre-defined probes. In addition, this document provides instructions on configuring the “sending” of IBA information from AOS to other systems using protocols such as Syslog.

There are two steps to be taken before installing the IBA Probes:

  1. Install AOS CLI. This CLI is an Apstra development effort to provide with extract functionalities to the Apstra Server, including Analytics.
  2. Install Custom Collector package. This is required in order to collect Telemetry data from the devices.

Install AOS CLI

To install AOS CLI please follow the AOS CLI step by step guide.

Install Custom Collector Package

To use IBA probes, you must install custom collector packages with the device system agent. These are installed via the web interface. Extract the specific platform Custom Collector (Cumulus, NXOS, Arista) Custom Collector: aosstdcollectors_custom_<platform>-0.1.0.post<version>-py2-none-any.whl file from the latest Apstra SDK package downloaded from Juniper Support Knowledge Base KB37156.

In addition, if the environment does not have Internet access, manually add the following Python library package files that are requirements for the Custom Collector. They can be downloaded from the python official repository. Please make sure to download the correct version as described below:

  • netaddr-0.7.17-py2.py3-none-any.whl
  • gtextfsm-0.2.1.tar.gz
  • pyeapi-0.8.2.tar.gz

These three packages are installed in the Apstra Server as well as in the selected devices along with the platform custom Collector as plugins (they can be found in the device folder /tmp/plugins).

To upload these packages to the Apstra Server, navigate to Devices > System Agents > Packages.

_images/image5.png

If you have manually uploaded all the required packages, you can verify that they have been properly installed by checking on Devices / Agents and clicking on the specific device’s agent that you would like to check.

From the Config section you can verify what you have selected to be installed, and scrolling down, in the Telemetry section, you can verify what it has been installed.

Note

If Apstra has Internet connection you will not be able to verify that netaddr, gtextfsm and pyeapi has been installed from the UI, only the custom Collector could be checked. You can verify it by logging into your device and checking on the /tmp/plugins folder as described above.

As an example for Arista EOS with an Apstra Server without Internet connection (otherwise the above three packages - netaddr, gtextfsm,pyeapi - will be automatically installed) click the “Upload Packages” button and select the “netaddr-0.7.17-py2.py3-none-any.whl”, “gtextfsm-0.2.1.tar.gz”, “pyeapi-0.8.2.tar.gz”, and “aosstdcollectors_custom_eos-0.1.0.post10-py2-none-any.whl” files.

_images/image9.png
_images/image8.png

Click “Upload” and the Package files will be uploaded to the Apstra Server.

To enable the package on the Device the two packages need to be enabled for a new Agent Profile. To add a new Agent Profile go to Devices / System Agents / Agent Profiles. Click “Create Agent Profile”. Enter “eos” for the “Platform” field. Since the usernames and passwords are already set, these fields do not need to be enabled.

_images/image2.png

Scroll down and under “Packages” the four packages that were uploaded will be available. Check all four packages to associate these with the Agent Profile then click “Create”.

_images/image12.png

After the required System Agent profile is created with the Custom Collector Package, it needs to be mapped to each System Agent. To enable the package on the Device, from Devices / System Agents / Agent select the device and under the device, provide appropriate Agent Profile beneath System Agent Profile option as per the screenshot below:

_images/image4.png

Note

It is recommended that when a System Agent is created, the appropriate System Agent Profile is mapped to enable Custom Collector Package. This helps take care of package update (i.e addition of more packages to profile) or installation etc. on all the associated System Agents by editing just one Agent profile.

You can verify now that the packages has been successfully installed from the Devices / Agents tab.

Config will display the packages that the user has specified to be installed.

_images/image19.png

Telemetry will show the packages that have been correctly installed.

_images/image20.png

For more information about System Agents and Agent Profiles, please refer to the Agent Profile Documentation.

Alternatively, the following AOS-CLI procedure to “bulk” update Device System Agents with an Agent Profile can be used.

Bulk Update System Agent Profile

In AOS-CLI (build 423 or later), there is a bulk edit option to “update” the Agent Profile in System Agents based on IP/ID or OS type (os_type) (i.e “eos”).

Use the “system-agents update-profile” command to update System Agents by IP range with a specific Agent Profile.

When setting the --profile option, AOS-CLI shows available Agent Profiles. Use the up and down arrow keys to select.

aos> system-agents update-profile --ip 172.20.120.6-11 --profile
                                                                  EOS-IBA  eos

For example.

aos> system-agents update-profile --ip 172.20.120.6-11 --profile 692bb0bb-c5e0-4d7e-a70c-c24b0d5650a8
Successfully updated agent 172.20.120.9 with given profile
Successfully updated agent 172.20.120.6 with given profile
Successfully updated agent 172.20.120.11 with given profile
Successfully updated agent 172.20.120.7 with given profile
Successfully updated agent 172.20.120.10 with given profile
Successfully updated agent 172.20.120.8 with given profile
aos>

Apstra IBA Probes

IBA probes can be implemented using two methods:

  • Create them or instantiate predefined ones from the web interface. See Probes for more information.
  • Install them from AOS CLI as described below.

Probe Installation

The pre-defined IBA Probes in this document are installed via the Apstra AOS-CLI utility.

All probes described in this document are included in AOS-CLI build 412 or later. Probe .j2 files may be made available if the probe file is not built into the AOS-CLI build.

Some of these IBA Probes require an updated service registry. Download the latest AOS SDK and extract the json-schemas.tar.gz file. Copy the file to the /home/admin directory of the Apstra Server so it is available in the AOS-CLI /mytmp directory.

aos> service-registry import-from --file /mytmp/json-schemas.tar.gz
Successfully imported service registry entry for interface_details
Successfully imported service registry entry for route_count
Successfully imported service registry entry for multicast_groups
Successfully imported service registry entry for sfp
Successfully imported service registry entry for resource_usage
Successfully imported service registry entry for mlag_domain
Successfully imported service registry entry for stp
Successfully imported service registry entry for vtep_counters
Successfully imported service registry entry for vlan
Successfully imported service registry entry for evpn_type5
Successfully imported service registry entry for ping
Successfully imported service registry entry for vxlan_info
Successfully imported service registry entry for pim_neighbor_count
Successfully imported service registry entry for lldp_details
Successfully imported service registry entry for evpn_type3
Successfully imported service registry entry for multicast_info
Successfully imported service registry entry for bgp_vrf
Successfully imported service registry entry for traceroute
Successfully imported service registry entry for vrf
Successfully imported service registry entry for table_usage
Successfully imported service registry entry for vxlan_address_table
Successfully imported service registry entry for acl_stats
Successfully imported service registry entry for device_info
Successfully imported service registry entry for power_supply
Successfully imported service registry entry for interface_buffer
Successfully imported service registry entry for pim_rp
Successfully imported service registry entry for anycast_rp
Successfully imported service registry entry for bgp_iba
Successfully imported service registry entry for interface_iba
aos>

Use the probe create AOS-CLI command to create the IBA Probes. You will be prompted for additional options.

aos> probe create
--blueprint           Id of the blueprint
--file                Filename of json file with probe data. Choose from dropdown or specify custom path
--skip-service-check  [Optional] By default, required telemetry services are checked and enabled on target
--check-status        [Optional] Wait for probe to become operational. Default: False
--service-interval    When skip-service-check is False and service is not alreadypresent, this indicates

Use --blueprint and tab-completion to select the Blueprint ID.

aos> probe create --blueprint 67cd936d-c2de-49f8-8708-df465f0cdc68
                              L2 Virtual  two_stage_l3clos

Using --file and tab-completion will list available probes supplied with AOS-CLI. Scroll through the list with the up and down arrow keys.

aos> probe create --blueprint 67cd936d-c2de-49f8-8708-df465f0cdc68 --file
                                                                      evpn.j2
                                                                      sfp.j2
                                                                      memory_usage_threshold_anomalies.j2
                                                                      bandwidth_utilization_history.j2
                                                                      power_supply_anomalies.j2
                                                                      virtual_infra_vlan_mismatch.j2
                                                                      hardware_vtep_counters_enabled.j2

Some probes will need additional Probe template variables.

aos> probe create --blueprint 67cd936d-c2de-49f8-8708-df465f0cdc68 --file /usr/local/lib/python2.7/site-packages/aos_cli/resources/probes/memory_usage_threshold_anomalies.j2
--skip-service-check  [Optional] By default, required telemetry services are checked and enabled on target
--check-status        [Optional] Wait for probe to become operational. Default: False
--service-interval    When skip-service-check is False and service is not alreadypresent, this indicates
--process             Probe template variable
--os_family           Probe template variable

IBA Probes Examples

The following section describes the process to install some of the most interesting probes which are not available by default.

Packet Drops

Packet drop IBA probes detect an abnormal amount of packet drops on interfaces of devices managed by Apstra based on interface telemetry collected by Device System Agents.

Filename Description
pkt_discard_anomalies.j2 Detect Fabric interfaces having sustained packet discards

To install the pkt_discard_anomalies.j2 IBA Probe:

aos> probe create --blueprint 67cd936d-c2de-49f8-8708-df465f0cdc68 --file /usr/local/lib/python2.7/site-packages/aos_cli/resources/probes/pkt_discard_anomalies.j2
Ensuring needed telemetry services for probe are enabled...
Successfully created probe f472ba21-d60f-44dc-9f5d-8318c8b9c07b in blueprint 67cd936d-c2de-49f8-8708-df465f0cdc68
aos>

SFP Optics

SFP optic IBA probes detect high and/or low warning thresholds in SFP RX power, TX power, temperature, voltage, or current for compatible optical modules on interfaces of devices managed by Apstra based on SFP telemetry collected by Device System Agents.

Filename Description
sfp.j2 Detect high and/or low warning thresholds in SFP RX Power, TX Power, Temperature, Voltage, or Current

To install the sfp.j2 IBA Probe:

aos> probe create --blueprint 67cd936d-c2de-49f8-8708-df465f0cdc68 --file /usr/local/lib/python2.7/site-packages/aos_cli/resources/probes/sfp.j2
Ensuring needed telemetry services for probe are enabled...
Enabled service sfp on device l2-virtual-002-leaf1:172.20.60.11
Enabled service sfp on device l2-virtual-001-leaf1:172.20.60.9
Enabled service sfp on device spine2:172.20.60.8
Enabled service sfp on device spine1:172.20.60.6
Enabled service sfp on device l2-virtual-003-leaf1:172.20.60.10
Enabled service sfp on device l2-virtual-004-leaf1:172.20.60.7
Successfully created probe b0c32a46-636c-4f82-b026-e9925b696625 in blueprint 67cd936d-c2de-49f8-8708-df465f0cdc68
aos>

Switch Memory Leak

Switch Memory Leak IBA probes detect abnormal memory leaks in specified processes on devices managed by AOS based on system telemetry collected by Device System Agents.

Filename Description
memory_usage_threshold_anomalies.j2 Detect memory leaks in specified process on all switches in the Fabric
system_memory_usage_threshold_anomalies.j2 Detect switches having potential memory leaks in the Fabric

Note

The Switch Memory Leak IBA probes require device user credentials set in the Device System Agent configuration that has login and access to the device BASH prompt.

Note

This IBA probe is only for Arista EOS devices.

The memory_usage_threshold_anomalies.j2 IBA Probe requires additional “Probe template variables” for os_family and process.

aos> probe create --blueprint 67cd936d-c2de-49f8-8708-df465f0cdc68 --file /usr/local/lib/python2.7/site-packages/aos_cli/resources/probes/memory_usage_threshold_anomalies.j2
  --skip-service-check  [Optional] By default, required telemetry services are checked and enabled on target
  --check-status        [Optional] Wait for probe to become operational. Default: False
  --service-interval    When skip-service-check is False and service is not alreadypresent, this indicates
  --process             Probe template variable
  --os_family           Probe template variable

The only option for os_family is eos for Arista EOS. The (2) options for process are edac-poller and fastcapi or configagent.

aos> probe create --blueprint 67cd936d-c2de-49f8-8708-df465f0cdc68 --file /usr/local/lib/python2.7/site-packages/aos_cli/resources/probes/memory_usage_threshold_anomalies.j2 --os_family eos --process fastcapi
Ensuring needed telemetry services for probe are enabled...
Enabled service resource_usage on device l2-virtual-002-leaf1:172.20.60.11
Enabled service resource_usage on device l2-virtual-001-leaf1:172.20.60.9
Enabled service resource_usage on device spine2:172.20.60.8
Enabled service resource_usage on device spine1:172.20.60.6
Enabled service resource_usage on device l2-virtual-003-leaf1:172.20.60.10
Enabled service resource_usage on device l2-virtual-004-leaf1:172.20.60.7
Successfully created probe 6a258d83-1053-42ad-935c-0550cc500b7d in blueprint 67cd936d-c2de-49f8-8708-df465f0cdc68
aos>
aos> probe create --blueprint rack-based-blueprint-10990707 --file /usr/local/lib/python2.7/site-packages/aos_cli/resources/probes/memory_usage_threshold_anomalies.j2 --os_family eos --process configagent
Ensuring needed telemetry services for probe are enabled...
Successfully created probe ed2c6be1-b4b1-4e1b-bd07-da431e89eeec in blueprint rack-based-blueprint-10990707
aos>

Note

“FastCapi” as service process is valid only for EOS version 4.18. For the newer version of EOS i.e 4.20 and later only ConfigAgent is valid. Please take extra care that service name is in lowercase during probe creation. So it should be configagent instead of ConfigAgent.

To install the IBA Probe for a second process, redo the probe create command for the other process.

The IBA Probe name may be modified to include the process name.

To install the system_memory_usage_threshold_anomalies.j2 IBA Probe:

aos> probe create --blueprint 67cd936d-c2de-49f8-8708-df465f0cdc68 --file /usr/local/lib/python2.7/site-packages/aos_cli/resources/probes/system_memory_usage_threshold_anomalies.j2
Ensuring needed telemetry services for probe are enabled...
Successfully created probe a669ccf8-cba7-414b-ad46-a7d4b4ca3928 in blueprint 67cd936d-c2de-49f8-8708-df465f0cdc68
aos>

Fault Tolerance

Filename Description
spine_fault_tolerance.j2 Find out if failure of given number of spines in the fabric is going to be tolerated. Raise anomaly if total traffic on all spines is more than the available spine capacity, with the specified number of spine failures.
lag_link_fault_tolerance.j2 Find out if failure of one link in a server LAG is going to be tolerated. Monitors total traffic in each LAG against total available capacity of the bond, with one link failure. Raise anomaly for racks with more than 50% of such overused bonds, sustained for certain duration.

Note

These (2) probes require AOS-CLI build 430 or later.

To install the spine_fault_tolerance.j2 IBA Probe:

aos> probe create --blueprint bf7a322c-ee3a-4dcf-aa20-df0560f538da --file /usr/local/lib/python2.7/site-packages/aos_cli/resources/probes/spine_fault_tolerance.j2 --number_of_faulty_spines_to_be_tolerated 1
Successfully created probe 0f0e9bf7-d9b3-43d7-906e-a9f0675e68f2 in blueprint bf7a322c-ee3a-4dcf-aa20-df0560f538da
aos>

Note

number_of_faulty_spines_to_be_tolerated has to be specified.

To install the lag_link_fault_tolerance.j2 IBA Probe:

aos> probe create --blueprint bf7a322c-ee3a-4dcf-aa20-df0560f538da --file /usr/local/lib/python2.7/site-packages/aos_cli/resources/probes/lag_link_fault_tolerance.j2
Successfully created probe 45ce5fe8-555f-41a9-b0ae-267125669d3f in blueprint bf7a322c-ee3a-4dcf-aa20-df0560f538da
aos>

After all the IBA Probes are installed, they will be available in the Blueprint under Analytics.

_images/image1.png

Sending IBA Info with Syslog

See Syslog Configuration for details about using Syslog to send messages to Syslog servers.