Use Junos PyEZ Tables with TextFSM Templates
Junos PyEZ operational (op) Tables can reference a TextFSM template, alone or with a Junos PyEZ View, to parse CLI or VTY command output from any network device.
Understanding TextFSM Templates
Junos PyEZ op Tables can extract data from the CLI or VTY command output of a network device. The Table can reference a View to map fields in the command output to Python objects. Junos PyEZ op Tables can also reference a TextFSM template, alone or with a View, to parse the command output. Junos PyEZ op Tables can use TextFSM templates to parse command output from any network device, regardless of the vendor, network OS, or command.
TextFSM is a Python library that parses semi-formatted CLI output, such
as show command output, from network devices. It was developed by Google and later released
under the Apache 2.0 license. The module requires a template and some input text. The
template uses regular expressions to describe how to parse the data, and you can define and
apply multiple templates to the same data.
TextFSM’s CliTable class enables users to map a command
on a given platform to the template that parses the command output. Network to Code, a network
automation company, has developed a Python wrapper for CliTable along with
a repository of TextFSM templates for network
devices. You can install the ntc-templates library on
your Junos PyEZ server or virtual environment, as appropriate. You then reference the NTC
templates and other TextFSM templates in your Junos PyEZ Tables.
The NTC templates parse show command output from network devices. Each NTC template defines the expected output fields for a given command, and for each item, maps the data to a header. The NTC template filename identifies the platform (vendor and network OS) and the command (with underscores), so the system can easily determine which template to use. The filename format uses the following syntax:
platform_command.textfsm
For example, consider the juniper_junos_show_arp_no-resolve.textfsm template.
Value Required MAC_ADDRESS ([A-Fa-f0-9\:]{17})
Value Required IP_ADDRESS ([A-Fa-f0-9:\.]+)
Value Required INTERFACE (\S+)
Value FLAGS (\S+)
Start
^MAC\s+Address\s+Address\s+Interface\s+Flags\s*$$
^${MAC_ADDRESS}\s+${IP_ADDRESS}\s+${INTERFACE}\s+${FLAGS} -> Record
^Total.*
^\s*$$
^{master:\d+}
^. -> Error
The template parses the show arp no-resolve command output from Juniper
Networks devices running Junos OS or Junos OS Evolved.
user@host> show arp no-resolve MAC Address Address Interface Flags 02:01:00:00:00:05 10.0.0.5 em0.0 none 30:7c:5e:48:4b:40 198.51.100.77 fxp0.0 none f8:c0:01:18:8b:67 198.51.100.254 fxp0.0 none 02:00:00:00:00:10 172.16.0.50 em0.0 none Total entries: 4
Junos PyEZ op Tables can use an NTC template or other TextFSM template to parse
unstructured command output. The Table uses a TextFSM template by defining the following
fields. Junos PyEZ uses the platform and command values to
determine the template’s filename.
-
command: command—Command that generates the output to parse. The command must map to the command string in the filename of an NTC template or other TextFSM template. -
key: key—Field defined in the TextFSM template or Junos PyEZ View that is used to uniquely identify the record item. -
platform: platform—Vendor and OS for the TextFSM template, for example,juniper_junos. The platform value must match the platform string in the filename of an NTC template or other TextFSM template. -
use_textfsm: True—Set toTrueto parse the command output by using the TextFSM template for the given platform and command.
How to Use TextFSM Templates to Parse Command Output
Junos PyEZ Tables can use TextFSM templates, including the predefined NTC templates, to parse show command output from Junos devices.
To use TextFSM templates in a Junos PyEZ Table:
Install the
ntc-templateslibrary on your Junos PyEZ server or virtual environment.user@host:~$ pip install ntc-templates
Create a custom Junos PyEZ Table that includes the
command,key,platform, anduse_textfsmarguments, as well as any additional arguments required for your operations.--- ArpTableTextFSM: command: show arp no-resolve platform: juniper_junos key: MAC_ADDRESS use_textfsm: True
The Junos PyEZ application uses the
platformandcommandvalues to determine the template’s filename, which in this case is juniper_junos_show_arp_no-resolve.textfsm. Thekeyvalue must match a header in the template.Create a Junos PyEZ application that uses the Table to retrieve the data.
import json from pprint import pprint import yaml from yamlloader import ordereddict from jnpr.junos import Device from jnpr.junos.factory.factory_loader import FactoryLoader yaml_table = """ --- ArpTableTextFSM: command: show arp no-resolve platform: juniper_junos key: MAC_ADDRESS use_textfsm: True """ globals().update(FactoryLoader().load(yaml.load(yaml_table, Loader=ordereddict.Loader))) with Device(host='router1.example.net') as dev: arp_stats = ArpTableTextFSM(dev).get() pprint(json.loads(arp_stats.to_json()))
Execute the application.
user@host:~$ python junos-pyez-arptable-textfsm.py {'02:00:00:00:00:10': {'FLAGS': 'none', 'INTERFACE': 'em0.0', 'IP_ADDRESS': '172.16.0.50'}, '02:01:00:00:00:05': {'FLAGS': 'none', 'INTERFACE': 'em0.0', 'IP_ADDRESS': '10.0.0.5'}, '30:7c:5e:48:4b:40': {'FLAGS': 'none', 'INTERFACE': 'fxp0.0', 'IP_ADDRESS': '198.51.100.77'}, 'f8:c0:01:18:8b:67': {'FLAGS': 'none', 'INTERFACE': 'fxp0.0', 'IP_ADDRESS': '198.51.100.254'}}The Table uses the NTC template to extract the output fields. For each Table item, the application returns the defined key and the data for each field.
How to Use TextFSM Templates with Junos PyEZ Views to Parse Command Output
Junos PyEZ Tables can combine a TextFSM template and a Junos PyEZ View to parse command output. The TextFSM template maps the data to a header. In the View, you can map your variable names to the headings defined in the template for the fields you want to return. Combine the template and View, for example, when you want to use different variable names than the ones defined in the template or when you want to return different fields. Junos PyEZ returns only the fields that are common to both the TextFSM template and the Junos PyEZ View.
The following example uses the juniper_junos_show_arp_no-resolve.textfsm template to parse the command output. The Junos PyEZ View maps the data to new variable names and returns only a subset of the fields. To review the template, see Understanding TextFSM Templates.
To use a TextFSM template and a View in a Junos PyEZ Table:
Create a custom Junos PyEZ Table that includes the
command,key,platform,use_textfsm, andviewarguments, as well as any additional arguments required for your operations.--- ArpTableTextFSM2: command: show arp no-resolve platform: juniper_junos key: - ip - mac use_textfsm: True view: ArpViewTextFSM2Create the Junos PyEZ View that defines which template fields to return and the corresponding variable name for each field.
ArpViewTextFSM2: fields: mac: MAC_ADDRESS ip: IP_ADDRESS interface: INTERFACEIn this case, the View does not map the
FLAGSfield defined in the TextFSM template, and the parsed data does not include this value.Create a Junos PyEZ application that uses the Table to retrieve the data.
import json from pprint import pprint import yaml from yamlloader import ordereddict from jnpr.junos import Device from jnpr.junos.factory.factory_loader import FactoryLoader yaml_table = """ --- ArpTableTextFSM2: command: show arp no-resolve platform: juniper_junos key: - ip - mac use_textfsm: True view: ArpViewTextFSM2 ArpViewTextFSM2: fields: mac: MAC_ADDRESS ip: IP_ADDRESS interface: INTERFACE """ globals().update(FactoryLoader().load(yaml.load(yaml_table, Loader=ordereddict.Loader))) with Device(host='router1.example.net') as dev: arp_stats = ArpTableTextFSM2(dev).get() pprint(json.loads(arp_stats.to_json()))
Execute the application.
user@host:~$ python junos-pyez-arptable-textfsm2.py {"('10.0.0.5', '02:01:00:00:00:05')": {'interface': 'em0.0', 'ip': '10.0.0.5', 'mac': '02:01:00:00:00:05'}, "('172.16.0.50', '02:00:00:00:00:10')": {'interface': 'em0.0', 'ip': '172.16.0.50', 'mac': '02:00:00:00:00:10'}, "('198.51.100.254', 'f8:c0:01:18:8b:67')": {'interface': 'fxp0.0', 'ip': '198.51.100.254', 'mac': 'f8:c0:01:18:8b:67'}, "('198.51.100.77', '30:7c:5e:48:4b:40')": {'interface': 'fxp0.0', 'ip': '198.51.100.77', 'mac': '30:7c:5e:48:4b:40'}}The Table uses the NTC template and Junos PyEZ View to extract the output fields. For each Table item, the application returns the defined key and the data for the fields defined in the View.
How to Use Custom TextFSM Templates
Junos PyEZ Tables can use the TextFSM templates that are installed as part of the
ntc-templates package. They can also reference custom TextFSM templates.
To use custom TextFSM templates in your Junos PyEZ application, you must first stage the
template. Then, when you define the Table instance in your application, specify the absolute
path to the template directory.
To use a custom TextFSM template in a Junos PyEZ Table:
Create a directory for your custom templates.
user@host:~$ mkdir TextFSMTemplates
In the templates directory, create your template and name the file using the platform_command.textfsm filename convention.
user@host:~$ vi TextFSMTemplates/my_platform_show_arp_no-resolve.textfsm
Create a Junos PyEZ Table that defines the same
platformandcommandvalues as the template filename.--- ArpTableTextFSM3: command: show arp no-resolve platform: my_platform key: - MAC_ADDRESS - IP_ADDRESS use_textfsm: TrueIn your Junos PyEZ application, specify the absolute path to the custom templates directory when you define the Table instance.
import json from pprint import pprint import yaml from yamlloader import ordereddict from jnpr.junos import Device from jnpr.junos.factory.factory_loader import FactoryLoader yaml_table = """ --- ArpTableTextFSM3: command: show arp no-resolve platform: my_platform key: - MAC_ADDRESS - IP_ADDRESS use_textfsm: True """ globals().update(FactoryLoader().load(yaml.load(yaml_table, Loader=ordereddict.Loader))) with Device(host='router1.example.net') as dev: arp_stats = ArpTableTextFSM3(dev, template_dir='/home/user/TextFSMTemplates').get() pprint(json.loads(arp_stats.to_json()))
Execute the application.
user@host:~$ python junos-pyez-arptable-textfsm3.py
How to Use Junos PyEZ Tables with TextFSM Templates to Parse Any Vendor’s Command Output
Junos PyEZ Tables can use TextFSM templates to parse command output from any vendor’s network
device. Your Python application can retrieve the output from a device or it can read the
output from a file. Then, when you create the Junos PyEZ Table instance, you can pass the
command output string to the Table’s raw argument instead of passing in a
Device instance.
For example, consider the following TextFSM template:
user@host:~$ cat TextFSMTemplates/cisco_xr_show_alarms_detail.textfsm
Value Required description (.+?)
Value Required location (\S+)
Value aid (\S+)
Value tag (\S+)
Value module (\S+)
Start
# Match the timestamp at beginning of command output
^Active Alarms (Detail) for .+
^-+ -> Continue.Record
^Description:\s+${description}\s*$$
^Location:\s+${location}
^AID:\s+${aid}
^Tag String:\s+${tag}
^Module Name:\s+${module}
The template parses the show alarms detail command output from the given device.
RP/0/RP0/CPU0:host#show alarms detail Wed May 5 12:17:00.187 UTC -------------------------------------------------------------------------------- Active Alarms (Detail) for 0/RP0 -------------------------------------------------------------------------------- Description: hw_optics: RX LOS LANE-0 ALARM Location: 0/RP0/CPU0 AID: XR/HW_OPTICS/5 Tag String: DEV_SFP_OPTICS_PORT_RX_LOS_LANE0 Module Name: Optics0/0/0/31 EID: CHASSIS/LCC/1:CONTAINER/CC/1:PORT/OPTICS/31 Reporting Agent ID: 196713 Pending Sync: false Severity: Major Status: Set Group: Software Set Time: 04/27/2021 09:47:16 UTC Clear Time: - Service Affecting: NotServiceAffecting Transport Direction: NotSpecified Transport Source: NotSpecified Interface: N/A Alarm Name: OPTICS RX LOS LANE-0 -------------------------------------------------------------------------------- Description: hw_optics: RX LOS LANE-1 ALARM Location: 0/RP0/CPU0 AID: XR/HW_OPTICS/6 Tag String: DEV_SFP_OPTICS_PORT_RX_LOS_LANE1 Module Name: Optics0/0/0/31 EID: CHASSIS/LCC/1:CONTAINER/CC/1:PORT/OPTICS/31 Reporting Agent ID: 196713 Pending Sync: false Severity: Major Status: Set Group: Software Set Time: 04/27/2021 09:47:16 UTC Clear Time: - Service Affecting: NotServiceAffecting Transport Direction: NotSpecified Transport Source: NotSpecified Interface: N/A Alarm Name: OPTICS RX LOS LANE-1 ...
The following example defines a Junos PyEZ Table that uses both the custom TextFSM template,
cisco_xr_show_alarms_detail.textfsm, and a View to parse the
show alarms detail command output. The example uses the
netmiko library to retrieve the data directly from the device. When the
application creates the Table instance, the raw argument passes in the
command output, and the template_dir argument defines the path to the
directory containing the custom template.
import json from pprint import pprint import yaml from yamlloader import ordereddict from netmiko import ConnectHandler from jnpr.junos.factory.factory_loader import FactoryLoader yaml_table = """ --- XRAlarmsTable: command: show alarms detail key: - description - location platform: cisco_xr use_textfsm: True view: XRAlarmsView XRAlarmsView: fields: description: description location: location aid: aid tag: tag module: module severity: severity status: status group: group time: time affect: affect """ dev_credentials = { 'device_type': 'cisco_xr', 'host': '198.51.100.101', 'username': 'admin', 'password': 'password', } net_connect = ConnectHandler(**dev_credentials) output = net_connect.send_command('show alarms detail') globals().update(FactoryLoader().load(yaml.load(yaml_table, Loader=ordereddict.Loader))) stats = XRAlarmsTable(raw=output, template_dir='/home/user/TextFSMTemplates').get() pprint(json.loads(stats.to_json())) with open('show-alarms-detail.txt', 'w') as fp: fp.write(output)
When you execute the application, it retrieves the command output from the device. The application then uses the TextFSM template in the specified directory along with the View to parse the output. Junos PyEZ returns only the fields that are common to both the TextFSM template and the Junos PyEZ View. The application saves the command output to a file, so you can process the output later, as shown in the next example.
user@host:~$ python junos-pyez-textfsm-alarms.py
{"('hw_optics: RX LOL LANE-0 ALARM', '0/RP0/CPU0')": {'aid': 'XR/HW_OPTICS/29',
'description': 'hw_optics: '
'RX LOL '
'LANE-0 '
'ALARM',
'location': '0/RP0/CPU0',
'module': 'Optics0/0/0/31',
'tag': 'DEV_SFP_OPTICS_PORT_RX_CDR_LOL_LANE0'},
"('hw_optics: RX LOL LANE-1 ALARM', '0/RP0/CPU0')": {'aid': 'XR/HW_OPTICS/30',
'description': 'hw_optics: '
'RX LOL '
'LANE-1 '
'ALARM',
'location': '0/RP0/CPU0',
'module': 'Optics0/0/0/31',
'tag': 'DEV_SFP_OPTICS_PORT_RX_CDR_LOL_LANE1'},
...The following example uses the same TextFSM template and Junos PyEZ View as the previous example, but in this case, the application reads the command output from a file.
import json from pprint import pprint import yaml from yamlloader import ordereddict from jnpr.junos.factory.factory_loader import FactoryLoader yaml_table = """ --- XRAlarmsTable: command: show alarms detail key: - description - location platform: cisco_xr use_textfsm: True view: XRAlarmsView XRAlarmsView: fields: description: description location: location aid: aid tag: tag module: module severity: severity status: status group: group time: time affect: affect """ with open('show-alarms-detail.txt') as fp: output = fp.read() globals().update(FactoryLoader().load(yaml.load(yaml_table, Loader=ordereddict.Loader))) stats = XRAlarmsTable(raw=output, template_dir='/home/user/TextFSMTemplates').get() pprint(json.loads(stats.to_json()))