Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
ON THIS PAGE
 

URL Filtering Overview

You can use URL filtering to determine which Web content is not accessible to users.

Components of this feature include the following:

  • URL filter database file

  • Configuration of one or more templates (up to eight per profile)

  • URL Filter Plug-in (jservices-urlf)

  • URL filtering daemon (url-filterd)

The URL filter database file is stored on the Routing Engine and contains all the disallowed URLs. Configured templates define which traffic to monitor, what criteria to match, and which actions to take. You configure the templates and the location of the URL filter database file in a profile.

Starting in Junos OS Release 17.2R2 and 17.4R1, for Adaptive Services, you can disable the filtering of HTTP traffic that contains an embedded IP address (for example, http:/10.1.1.1) belonging to a disallowed domain name in the URL filter database.Starting in Junos OS Release 19.3R2, this same functionaly is supported for Next Gen Services on MX240, MX480, and MX960.

To enable the URL filtering feature, you must configure jservices-urlf as the package-name at the [edit chassis fpc slot-number pic pic-number adaptive-services service-package extension-provider] hierarchy level. Once enabled, jservices-urlf maintains the URL filtering profile and receives all traffic to be filtered, the filtering criteria, and the action to be taken on the filtered traffic.

Note:

MX-SPC3 does not explicitly need jservices-urlf as the package-name at the [edit chassis fpc slot-number pic pic-number adaptive-services service-package extension-provider] hierarchy level. It is supported by default.

The URL filtering daemon (url-filterd), which also resides on the Routing Engine, resolves the domain name of each URL in the URL filter database to a list of IPv4 and IPv6 addresses. It then downloads the list of IP addresses to the service PIC, which runs jservices-urlf. Then url-filterd interacts with the Dynamic Firewall process (dfwd) to install filters on the Packet Forwarding Engine to punt the selected traffic from the Packet Forwarding Engine to the service PIC.

As new HTTP and HTTPS traffic reaches the router, a decision is made based on the information in the URL filter database file. The filtering rules are checked and either the router accepts the traffic and passes it on or blocks the traffic. If the traffic is blocked, one of the following configured actions is taken:

  • An HTTP redirect is sent to the user.

  • A custom page is sent to the user.

  • An HTTP status code is sent to the user.

  • A TCP reset is sent.

Accept is also an option. In this case, the traffic is not blocked.

Figure 1 illustrates the URL filtering for HTTP sessions.

Figure 1: Packet Flow-URL Filtering for HTTP Sessions Packet Flow-URL Filtering for HTTP Sessions

Figure 2 illustrates the URL filtering for HTTPS sessions.

Figure 2: Packet Flow-URL Filtering for HTTPS Sessions Packet Flow-URL Filtering for HTTPS Sessions

For more details on the URL filtering feature, see the following sections:

URL Filter Database File

The URL filter database file contains entries of URLs and IP addresses. Create the URL filter database file in the format indicated in Table 1 and locate it on the Routing Engine in the /var/db/url-filterd directory.

Table 1: URL Filter Database File Format

Entry

Description

Example

FQDN

Fully qualified domain name.

www.badword.com/jjj/bad.jpg

URL

Full string URL without the Layer 7 protocol.

www.srch.com/*badword*/

www.srch.com

www.srch.com/xyz

www.srch.com/xyz*

IPv4 address

HTTP request on a specific IPv4 address.

10.1.1.199

IPv6 address

HTTP request on a specific IPv6 address.

1::1

You must specify a custom URL filter database in the profile. If needed, you can also assign a custom URL filter database file with any template, and that database takes precedence over the database configured at the profile level.

If you change the contents of the URL filter database file, use the request services (url-filter | web-filter) update command. Other commands to help maintain the URL filter database file include the following:

  • request services (url-filter | web-filter) delete

  • request services (url-filter | web-filter) force

  • request services (url-filter | web-filter) validate

URL Filter Profile Caveats

The URL filter profile consists of from one to eight templates. Each template consists of a set of configured logical interfaces where traffic is monitored for URL filtering and one or more terms.

A term is a set of match criteria with actions to be taken if the match criteria is met. You must configure at least one term to configure URL filtering. Each term consists of a from statement and a then statement, where the from statement defines the source IP prefixes and destination ports that are monitored. The then statement specifies the action to be taken. If you omit the from statement, any source IP prefix and any destination port are considered to match. But you can omit only one from statement per template or per profile.

Example configuration of multiple terms without from statements

If you omit more than one from statement per template, you will get the following error message on commit:

Related Documentation

Release History Table
Release
Description
19.3R2
Starting in Junos OS Release 19.3R2, this same functionaly is supported for Next Gen Services on MX240, MX480, and MX960.
17.2R2
Starting in Junos OS Release 17.2R2 and 17.4R1, for Adaptive Services, you can disable the filtering of HTTP traffic that contains an embedded IP address (for example, http:/10.1.1.1) belonging to a disallowed domain name in the URL filter database.