How to Automatically Update Prefix Lists
Throughout this book you have worked extensively with prefix lists. A prefix list contains one or more prefixes that can be used in routing policies. When you build your network, you usually start with manually-created and manually-updated prefix lists. Your own set of prefixes (the ones that you will originate) doesn’t change very often; you accept everything from your transit providers anyway, and when you do not have a lot of peers or customers there are not many changes to their prefix lists, either. We’ve all been there.
However, as your network and your customer base grow, it is not feasible to keep manually updating prefix lists. In this Appendix we show you how we use the NETCONF functionality of Junos OS, along with a few Python scripts, to automatically upload prefix list changes.
This Appendix is a bit different from the rest of the book, since we cannot provide ready-made scripts that you can cut-and-paste into your network’s management system. Instead we show you how we do it, and invite you to talk to your software development team, encouraging them to be enthusiastic about your new secure routing table, too, and have them implement automatic prefix list changes on your network.
There are three basic steps:
Make sure that for each customer, your customer database contains the AS-SET (or AS number) that your customers announce to you.
Retrieve the prefix list for that AS-SET from the IRR.
Upload the prefix list to your router(s).
Doing this periodically (for instance, twice a day) means that you don’t have to worry about customers who wish to change their prefix list. They do not have to create tickets asking you to change their prefix list, and what’s more: that means you don’t have to handle these tickets any longer.
Customer Database
Your database needs to contain at least two fields for each customer:
Their AS number (we will use this in the name of the prefix list as it is uploaded to the router).
Their AS-SET.
Your customer will have created their own AS-SET in the IRR.
When connecting your customer to the network, you will check to see
if their AS-SET makes sense. It should contain the AS number that
they use for their own network (from which they will announce their
own prefixes) as well as the AS numbers and/or AS-SETs of their customers.
By walking through this “tree”, the bgpq3 tool will create one single prefix list that collapses all prefixes
that your customer may announce.
Retrieve Prefix-list: Use bgpq3
The tool to convert AS-SETs to prefix lists is bgpq3. The official bgpq3 web site is at https://github.com/snar/bgpq3 where you are invited to read, learn, and contribute. After installing
bgpq3 on a machine, you can use it to create prefix lists like the
one below:
And for IPv6:
Using bgpq3, create a .txt file in
the directory prefix-list.d/ for each AS-SET
that you wish to auto-update. The .txt file should have the name of
the prefix list as you want it pushed to the router; and the contents
should be a prefix per line. Creating a script that does this periodically
is left as an exercise to the reader.
Upload to Your Router
The Junos OS supports multiple ways of uploading external information into the configuration database. Since this is an Appendix, it is not exhaustive and does not offer a ready-made, field-tested configuration. Only an example is provided here. In the example, the NETCONF interface for Junos OS is used to upload information. This example specifically uses the Python 3 language and the Junos PyEZ library created and maintained by Juniper Networks. Note that PyEZ may also be used with Python 2.7 if preferable.
Check out the Day One PyEZ book here: https://www.juniper.net/us/en/training/jnbooks/day-one/automation-series/junos-pyez-cookbook/.
In this example, a fictitious Juniper device is used as the configuration target. However, the example should work on any of the Junos OS devices that use prefix lists in the same manner. Our requirement is to update the list of IP prefixes present in the prefix list based on the contents of a text file. The script that will accomplish this is run from a remote host.
Requirements
For the Junos OS device:
Have the NETCONF SSH subsystem (see: https://www.juniper.net/documentation/en_US/junos/topics/topic-map/netconf-ssh-connection.html) enabled through:
#
set system services netconf sshAllow access from remote server on NETCONF port (default: 830)
For the remote server:
Have Python 3.5 installed
(alternatively, use Python 2 and adjust syntax accordingly in code below)
Have Junos PyEZ installed (see: https://github.com/Juniper/py-junos-eznc), for example, through:
#
pip install junos-eznc
Example Code
The Python 3 code below looks for *.txt files in a directory
named prefix-list.d inside the parent directory
of the script itself. Each such file represents a prefix list to update
and should contain prefixes, one per line, that the prefix list should
consist of. Empty lines and lines starting with the hash ( # ) symbol
are ignored. The name of the file (excluding the extension suffix)
is used as the name of the prefix list.
Update the configuration variables HOST, USER, and PASS at the top of the script to match your setup. Refer to the comments inside the script for further explanation:
<code> from pathlib import Path from jnpr import junos from jnpr.junos.utils.config import Config from lxml import etree as ET HOST = 'mx0.example.com'
USER = 'automation' PASS = 'secret' # Prefix lists to update. # Dictionary of {name -> list of prefixes}.
prefix_lists = {} # Find files matching 'prefix-list.d/*.txt' # relative to the parent directory of the script.
CONF_DIR = Path(_file_).parent / 'prefix-list.d' for f in CONF_DIR.glob('*.txt'): with f.open('r', encoding='utf-8') as fp: # Use the file basename ('stem') as the prefix list name; # read, split & trim lines, exclude comments ('#') and empty.
lines = [x.strip() for x in fp.read().splitlines()] prefix_lists[f.stem] = [x for x in lines if x and not x.startswith('#')] # Connect to configuration target. # Note: should use agent auth if PASS is None.
print('Connect to [%s]...' % HOST) with junos.Device(host=HOST, user=USER, passwd=PASS) as device:
# Open configuration; use private mode to avoid # committing existing shared candidate configuration.
print('Open private configuration...') with Config(device, mode='private') as config: # Merge XML config for each prefix list.
for name, prefixes in prefix_lists.items(): # Generate an XML configuration with the following structure:
"""
<configuration> <policy-options> <prefix-list replace="replace"> <name>{name}</name> <prefix-list-item> <name>x.x.x.x/n</name> </prefix-list-item> (...) </prefix-list> </policy-options> </configuration>
""" patch = ET.Element('configuration') policy_options = ET.SubElement(patch, 'policy-options') # Note: replace entire prefix-list to remove extraneous entries. prefix_list = ET.SubElement(policy_options, 'prefix-list', replace='replace') ET.SubElement(prefix_list, 'name').text = name for prefix in prefixes: item = ET.SubElement(prefix_list, 'prefix-list-item') ET.SubElement(item, 'name').text = prefix print('\nUpdate prefix list [%s] with [%d] entries:' % (name, len(prefixes)))
print('---\n' + ET.tounicode(patch, pretty_print=True) + '---') config.load(patch) print('\nUpdated [%d] prefix lists; config diff:' % len(prefix_lists))
print('---\n' + config.diff() + '---') while True: choice = input('\nDo you want to commit these changes? [y/n] ').lower()
if choice in ('y', 'yes'): print('Committing changes...')
config.commit()
print('Commit successful!')
break elif choice in ('n', 'no'): print('Changes not committed.')
break </code>
This code is probably not a cut-and-paste into your own network management system, but it should definitely get you up and running, and ready for automated prefix list changes.
The Next Step: Use the Same Mechanism for Filtering Your Peers
Developers are creatively writing software, and network engineers are creatively using it!
Based on the work described above, not only are you now creating
prefix lists to filter your customers, you can use the same mechanism
to filter your peers as well; simply create a ‘pseudo customer’
in your network management system, note down the AS-SET that the peer
should announce, and then you can refer to the prefix-list in the BGP policy for that peer.
This way you can be sure that your customers, as well as your peers, have a degree of security in the prefixes that they are allowed to announce.
Additional Resources
Now that you have a secure routing table, the next step is to express to customers and partners that you are taking responsibility for a safer, more stable Internet.
One way to express this is to join MANRS (Mutually Agreed Norms for Routing Security). MANRS is a global initiative, supported by the Internet Society, that provides crucial fixes to reduce the most common routing threats. More information is available on their website at: https://www.manrs.org/.
In order to get your customers to announce valid, usable information to you, you may have to help them to fix their announcements, IRR registrations, or RPKI ROAs. The more networks that join, the more secure the Internet becomes!
There’s great documentation within the Juniper TechLibrary that can help:
Enabling BGP to Carry Flow-Specification Routes: https://www.juniper.net/documentation/en_US/ junos/topics/example/routing-bgp-flow-specification-routes.html.
BGP Feature Guide for Routing Devices: https://www.juniper.net/techpubs/en_US/junos15.1/information-products/pathway-pages/config-guide-routing/config-guide-routing-bgp.pdf
And these frequently asked questions compiled by NLnet Labs are very handy on RPKI specific issues: https://nlnetlabs.nl/projects/rpki/faq/.
Conclusion
The authors hope that the information provided in this book will help you get started designing and implementing methods for a secure routing table, and likewise a more stable and reliable network.
It is very likely, and maybe even the wish of the authors, that parts of this book soon become obsolete due to all the work being done to get to a more secure routing table. If parts of this book do become obsolete, we will have achieved our goal!
You might have noticed that implementing routing security can consume quite a bit of time. However, we hope that you will take the time to do so, or even better, help build the tools to do so – there are already many tools and scripts available on sources like Github, which you can use to cut down on time spent. And remember that we all have the same goal, so let us know if you think you can help or have suggestions on how to improve those tools.
If you have any ideas, suggestions, or remarks on how to further develop routing security feel free to reach out to the authors (dayone@juniper.net). By joining forces and helping each other we can make the Internet a safer place for all!