URL Filtering Overview
Starting in Junos OS Release 17.2, you can use URL filtering to determine which Web content is not accessible to users.
Components of this feature include the following:
URL filter database file
Configuration of one or more templates (up to eight per profile)
URL Filter Plug-in (jservices-urlf)
URL filtering daemon (url-filterd)
The URL filter database file is stored on the Routing Engine and contains all the blacklisted URLs. Configured templates define which traffic to monitor, what criteria to match, and which actions to take. You configure the templates and the location of the URL filter database file in a profile.
Starting in Junos OS Release 17.2R2 and 17.4R1, you can disable the filtering of HTTP traffic that contains an embedded IP address (for example, http:/10.1.1.1) belonging to a blacklisted domain name in the URL filter database.
To enable the URL filtering feature, you must configure jservices-urlf as the package-name at the [edit chassis fpc slot-number pic pic-number adaptive-services service-package extension-provider] hierarchy level. Once enabled, jservices-urlf maintains the URL filtering profile and receives all traffic to be filtered, the filtering criteria, and the action to be taken on the filtered traffic.
The URL filtering daemon (url-filterd), which also resides on the Routing Engine, resolves the domain name of each URL in the URL filter database to a list of IPv4 and IPv6 addresses. It then downloads the list of IP addresses to the service PIC, which runs jservices-urlf. Then url-filterd interacts with the Dynamic Firewall process (dfwd) to install filters on the Packet Forwarding Engine to punt the selected traffic from the Packet Forwarding Engine to the service PIC.
As new HTTP and HTTPS traffic reaches the router, a decision is made based on the information in the URL filter database file. The filtering rules are checked and either the router accepts the traffic and passes it on or blocks the traffic. If the traffic is blocked, one of the following configured actions is taken:
An HTTP redirect is sent to the user.
A custom page is sent to the user.
An HTTP status code is sent to the user.
A TCP reset is sent.
Accept is also an option. In this case, the traffic is not blocked.
For more details on the URL filtering feature, see the following sections:
URL Filter Database File
The URL filter database file contains entries of URLs and IP
addresses. Create the URL filter database file in the format indicated
in Table 1 and locate
it on the Routing Engine in the
Table 1: URL Filter Database File Format
Fully qualified domain name.
Full string URL without the Layer 7 protocol.
HTTP request on a specific IPv4 address.
HTTP request on a specific IPv6 address.
You must specify a custom URL filter database in the profile. If needed, you can also assign a custom URL filter database file with any template, and that database takes precedence over the database configured at the profile level.
If you change the contents of the URL filter database file, use the request services (url-filter | web-filter) update command. Other commands to help maintain the URL filter database file include the following:
request services (url-filter | web-filter) delete
request services (url-filter | web-filter) force
request services (url-filter | web-filter) validate
URL Filter Profile Caveats
The URL filter profile consists of from one to eight templates. Each template consists of a set of configured logical interfaces where traffic is monitored for URL filtering and one or more terms.
A term is a set of match criteria with actions to be taken if the match criteria is met. You must configure at least one term to configure URL filtering. Each term consists of a from statement and a then statement, where the from statement defines the source IP prefixes and destination ports that are monitored. The then statement specifies the action to be taken. If you omit the from statement, any source IP prefix and any destination port are considered to match. But you can omit only one from statement per template or per profile.
Example configuration of multiple terms without from statements
If you omit more than one from statement per template, you will get the following error message on commit:
URLFD_CONFIG_FAILURE: Configuration not valid: Cannot have two wild card terms in template template1 error: configuration check-out failed