URL Filtering
URL Filtering Overview
You can use URL filtering to determine which Web content is not accessible to users.
Components of this feature include the following:
URL filter database file
Configuration of one or more templates (up to eight per profile)
URL Filter Plug-in (jservices-urlf)
URL filtering daemon (url-filterd)
The URL filter database file is stored on the Routing Engine and contains all the disallowed URLs. Configured templates define which traffic to monitor, what criteria to match, and which actions to take. You configure the templates and the location of the URL filter database file in a profile.
Starting in Junos OS Release 17.2R2 and 17.4R1, for Adaptive Services, you can disable the filtering of HTTP traffic that contains an embedded IP address (for example, http:/10.1.1.1) belonging to a disallowed domain name in the URL filter database.Starting in Junos OS Release 19.3R2, this same functionaly is supported for Next Gen Services on MX240, MX480, and MX960.
To enable the URL filtering feature, you must configure jservices-urlf as the package-name at the [edit chassis fpc slot-number pic pic-number adaptive-services service-package extension-provider] hierarchy level. Once enabled, jservices-urlf maintains the URL filtering profile and receives all traffic to be filtered, the filtering criteria, and the action to be taken on the filtered traffic.
The URL filtering daemon (url-filterd), which also resides on the Routing Engine, resolves the domain name of each URL in the URL filter database to a list of IPv4 and IPv6 addresses. It then downloads the list of IP addresses to the service PIC, which runs jservices-urlf. Then url-filterd interacts with the Dynamic Firewall process (dfwd) to install filters on the Packet Forwarding Engine to punt the selected traffic from the Packet Forwarding Engine to the service PIC.
As new HTTP and HTTPS traffic reaches the router, a decision is made based on the information in the URL filter database file. The filtering rules are checked and either the router accepts the traffic and passes it on or blocks the traffic. If the traffic is blocked, one of the following configured actions is taken:
An HTTP redirect is sent to the user.
A custom page is sent to the user.
An HTTP status code is sent to the user.
A TCP reset is sent.
Accept is also an option. In this case, the traffic is not blocked.
For more details on the URL filtering feature, see the following sections:
URL Filter Database File
The URL filter database file contains entries of URLs and IP
addresses. Create the URL filter database file in the format indicated
in Table 1 and locate
it on the Routing Engine in the /var/db/url-filterd
directory.
Table 1: URL Filter Database File Format
Entry | Description | Example |
---|---|---|
FQDN | Fully qualified domain name. | www.badword.com/jjj/bad.jpg |
URL | Full string URL without the Layer 7 protocol. | www.yahoo.com/*badword*/ |
IPv4 address | HTTP request on a specific IPv4 address. | 10.1.1.199 |
IPv6 address | HTTP request on a specific IPv6 address. | 1::1 |
You must specify a custom URL filter database in the profile. If needed, you can also assign a custom URL filter database file with any template, and that database takes precedence over the database configured at the profile level.
If you change the contents of the URL filter database file, use the request services (url-filter | web-filter) update command. Other commands to help maintain the URL filter database file include the following:
request services (url-filter | web-filter) delete
request services (url-filter | web-filter) force
request services (url-filter | web-filter) validate
URL Filter Profile Caveats
The URL filter profile consists of from one to eight templates. Each template consists of a set of configured logical interfaces where traffic is monitored for URL filtering and one or more terms.
A term is a set of match criteria with actions to be taken if the match criteria is met. You must configure at least one term to configure URL filtering. Each term consists of a from statement and a then statement, where the from statement defines the source IP prefixes and destination ports that are monitored. The then statement specifies the action to be taken. If you omit the from statement, any source IP prefix and any destination port are considered to match. But you can omit only one from statement per template or per profile.
Example configuration of multiple terms without from statements
If you omit more than one from statement per template, you will get the following error message on commit:
URLFD_CONFIG_FAILURE: Configuration not valid: Cannot have two wild card terms in template template1 error: configuration check-out failed
See also
Configuring URL Filtering
URL filtering is configured on a service PIC. The interfaces you are dealing with are services interfaces (which use the ms prefix) or aggregated multiservices (AMS) interfaces (which use the ams prefix). For more information on AMS interfaces, see the Adaptive Services Interfaces User Guide for Routing Devices starting with Understanding Aggregated Multiservices Interfaces.
To configure the URL filtering feature, you must first configure jservices-urlf as the package-name at the [edit chassis fpc slot-number pic pic-number adaptive-services service-package extension-provider] hierarchy level. For more information on configuring the extension-provider package package-name configuration statement, see the package (Loading on PIC) statement.
A URL filtering profile is a collection of templates. Each template consists of a set of criteria that defines which URLs are disallowed and how the recipient is notified.
To configure the URL profile:
- Assign a name to the URL profile.[edit]user@host# edit services (web-filter | url-filter) profile profile-name
Starting in Junos OS Release 18.3R1, for Adaptive Services. configure the profile at the [edit services web-filter] hierarchy level. Before Junos OS Release 18.3R1, configure the profile at the [edit services url-filter] hierarchy level.Starting in Junos OS Release 19.3R2, this same functionality is available for Next Gen Serices on MX240, MX480, and MX960.
- Specify the name of the URL filter database to use.[edit services (web-filter | url-filter) profile profile-name]user@host# set url-filter-database filename
- Configure one or more templates for the profile.
To configure each template:
- Name the template.[edit services (web-filter | url-filter) profile profile-name]user@host# set (url-filter-template template-name | template template-name)
Note Starting in Junos OS Release 18.3R1, configure the template with the url-filter-template statement. Before Junos OS Release 18.3R1, configure the template with the template statement.
- Go to that new template hierarchy level.[edit services (web-filter | url-filter) profile profile-name]user@host# edit (url-filter-template template-name | template template-name)
- Specify the name of the URL filter database to use.[edit services (web-filter | url-filter) profile profile-name (url-filter-template template-name | template template-name)]user@host# set url-filter-database filename
- Specify the loopback interface for which the source IP
address is picked for sending DNS queries.[edit services (web-filter | url-filter) profile profile-name (url-filter-template template-name | template template-name)]user@host# set dns-source-interface loopback-interface-name
- Disable the filtering of HTTP traffic that contains an
embedded IP address (for example, http:/10.1.1.1) belonging to a disallowed
domain name in the URL filter database.[edit services (web-filter | url-filter) profile profile-name (url-filter-template template-name | template template-name)]user@host# set disable-url-filtering
- Configure the DNS resolution time interval in minutes.[edit services (web-filter | url-filter) profile profile-name (url-filter-template template-name | template template-name)]user@host# set dns-resolution-interval minutes
- Configure the number of retries for a DNS query in case
the query fails or times out.[edit services (web-filter | url-filter) profile profile-name]user@host# set dns-retries number
- Specify the IP addresses (IPv4 or IPv6) of DNS servers
to which the DNS queries are sent.[edit services (web-filter | url-filter) profile profile-name (url-filter-template template-name | template template-name)]user@host# set dns-server [ip-address]
- Specify the client-facing logical interfaces on which
the URL filtering is configured.[edit services (web-filter | url-filter) profile profile-name (url-filter-template template-name | template template-name)]user@host# set client-interfaces [ client-interface-name ]
- Specify the server-facing logical interfaces on which
the URL filtering is configured.[edit services (web-filter | url-filter) profile profile-name (url-filter-template template-name | template template-name)]user@host# set server-interfaces [ server-interface-name ]
- Specify the routing instance on which the URL filtering
is configured.[edit services (web-filter | url-filter) profile profile-name (url-filter-template template-name | template template-name)]user@host# set routing-instance routing-instance-name
- Specify the routing instance on which the DNS server is
reachable.[edit services (web-filter | url-filter) profile profile-name (url-filter-template template-name | template template-name)]user@host# dns-routing-instance dns-routing-instance-name
- Name the template.
- Configure the term information.
Terms are used in filters to segment the policy or filter into small match and action pairs.
- Name the term.[edit services (web-filter | url-filter) profile profile-name (url-filter-template template-name | template template-name)]user@host# set term term-name
- Go to the new term hierarchy level.[edit services (web-filter | url-filter) profile profile-name (url-filter-template template-name | template template-name)]user@host# edit term term-name
- Specify the source IP address prefixes for traffic you
want to filter.[edit services (web-filter | url-filter) profile profile-name (url-filter-template template-name | template template-name) term term-name]user@host# set from src-ip-prefix [prefix]
- Specify the destination ports for traffic you want to
filter.[edit services (web-filter | url-filter) profile profile-name (url-filter-template template-name | template template-name) term term-name]user@host# set from dest-port [port]
- Configure an action to take.[edit services (web-filter | url-filter) profile profile-name (url-filter-template template-name | template template-name) term term-name]user@host# set then action
The action can be one of the following:
custom-page custom-page—Send a custom page string to the user.http-status-code http-status-code—Send an HTTP status code to the user.redirect-url redirect-url—Send an HTTP redirect to the user.tcp-reset—Send a TCP reset to the user.
- Name the term.
- Associate the URL profile with a next-hop service set.
Note For URL filtering, you must configure the service set as a next-hop service set.
[edit]user@host# set services service-set service-set-name (web-filter-profile profile-name | url-filter-profile profile-name)user@host# set services service-set service-set-name next-hop-service inside-service-interface interface-name.unit-numberuser@host# set services service-set service-set-name next-hop-service outside-service-interface interface-name.unit-numberNote The service interface can also be of the ams prefix. If you are using ams interfaces at the [edit services service-set service-set-name] hierarchy level for the URL filter, you must also configure the load-balancing-options hash-keys statement at the [edit interfaces ams-interface-name unit number] hierarchy level. .
Note Starting in Junos OS Release 18.3R1, configure the service set with the web-filter-profile statement. Before Junos OS Release 18.3R1, configure the service set with the url-filter-profile statement.
DNS Request Filtering for Disallowed Website Domains
Overview of DNS Request Filtering
Starting in Junos OS Release 18.3R1, you can configure DNS filtering to identify DNS requests for disallowed website domains. Starting in Junos OS Release 19.3R2, you can configure DNS filtering if you are running Next Gen Services with the MX-SPC3 services card. Next Gen Services are supported on MX240, MX480 and MX960 routers. For DNS request types A, AAAA, MX, CNAME, TXT, SRV, and ANY, you configure the action to take for a DNS request for a disallowed domain. You can either:
Block access to the website by sending a DNS response that contains the IP address or fully qualified domain name (FQDN) of a DNS sinkhole server. This ensures that when the client attempts to send traffic to the disallowed domain, the traffic instead goes to the sinkhole server (see Figure 1).
Log the request and allow access.
For other DNS request types for a disallowed domain, the request is logged and access is allowed.
The actions that the sinkhole server takes are not controlled by the DNS request filtering feature; you are responsible for configuring the sinkhole server actions. For example, the sinkhole server could send a message to the requestor that the domain is not reachable and prevent access to the disallowed domain.

Benefits
DNS filtering redirects DNS requests for disallowed website domains to sinkhole servers, while preventing anyone operating the system from seeing the list of disallowed domains. This is because the disallowed domain names are in an encrypted format.
Disallowed Domain Filter Database File
DNS request filtering requires a disallowed domain filter database .txt file, which identifies each disallowed domain name, the action to take on a DNS request for the disallowed domain, and the IP address or fully qualified domain name (FQDN) of a DNS sinkhole server.
DNS Filter Profile
You configure a DNS filter profile to specify which disallowed domain filter database file to use. You can also specify the interfaces on which DNS request filtering is performed, limit the filtering to requests for specific DNS servers, and limit the filtering to requests from specific source IP address prefixes.
How to Configure DNS Request Filtering
To filter DNS requests for disallowed website domains, perform the following:
How to Configure a Domain Filter Database
Create one or more domain filter database files that include an entry for each disallowed domain. Each entry specifies what to do with a DNS request for a disallowed website domain.
To configure a domain filter database file:
- Create the name for the file. The database file name can
have a maximum length of 64 characters and must have a
.txt
extension. - Add a file header with a format such as
20170314_01:domain,sinkhole_ip,v6_sinkhole,sinkhole_fqdn,id,action
. - Add an entry in the file for each disallowed domain. You
can include a maximum of 10,000 domain entries. Each entry in the
database file has the following items:
hashed-domain-name,IPv4 sinkhole address, IPv6 sinkhole address, sinkhole FQDN, ID, action
where:
hashed-domain-name is a hashed value of the disallowed domain name (64 hexadecimal characters). The hash method and hash key that you use to produce the hashed domain value are needed when you configure DNS filtering with the Junos OS CLI.
IPv4 sinkhole address is the address of the DNS sinkhole server for IPv4 DNS requests.
IPv6 sinkhole address is the address of the DNS sinkhole server for IPv6 DNS requests.
sinkhole FQDN is the fully qualified domain name of the DNS sinkhole server.
ID is a 32-bit number that uniquely associates the entry with the hashed domain name.
action is the action to apply to a DNS request that matches the disallowed domain name. If you enter replace, the MX Series router sends the client a DNS response with the IP address or FQDN of the DNS sinkhole server. If you enter report, the DNS request is logged and then sent to the DNS server.
- In the last line of the file, include the file hash, which you calculate by using the same key and hash method that you used to produce the hashed domain names.
- Save the database files on the Routing Engine in the
/var/db/url-filterd
directory. - Validate the domain filter database file.user@host> request services web-filter validate dns-filter-file-name filename hash-key key-string hash-method hash-method-name
- If you make any changes to the database file, apply the
changes.user@host> request services web-filter update dns-filter-database filename
How to Configure a DNS Filter Profile
A DNS filter profile includes general settings for filtering DNS requests for disallowed website domains, and includes up to 32 templates. The template settings apply to DNS requests on specific uplink and downlink logical interfaces or routing instances, or to DNS requests from specific source IP address prefixes, and override the corresponding settings at the DNS profile level. You can configure up to eight DNS filter profiles.
To configure a DNS filter profile:
- Configure the name for a DNS filter profile:[edit]user@host# edit services web-filter profile profile-name
The maximum number of profiles is 8.
- Configure the interval for logging per-client statistics
for DNS filtering. The range is 0 through 60 minutes and the default
is 5 minutes.[edit services web-filter profile profile-name]user@host# set global-dns-stats-log-timer minutes
- Configure general DNS filtering settings for the profile.
These values are used if a DNS request does not match a specific template.
- Specify the name of the domain filter database to use
when filtering DNS requests.[edit services web-filter profile profile-name dns-filter]user@host# set database-file filename
- (Optional) To limit DNS filtering to DNS requests that
are destined for specific DNS servers, specify up to three IP addresses
(IPv4 or IPv6).[edit services web-filter profile profile-name dns-filter]user@host# set dns-server [ ip-address ]
- Specify the format for the hash key.[edit services web-filter profile profile-name dns-filter]user@host# set hash-key ascii-text
- Specify the hash key that you used to create the hashed
domain name in the domain filter database file.[edit services web-filter profile profile-name dns-filter]user@host# set hash-key key-string
- Specify the hash method that was used to create the hashed
domain name in the domain filter database file.[edit services web-filter profile profile-name dns-filter]user@host# set hash-method hash-method-name
The only supported hash method is hmac-sha2-256.
- Configure the interval for logging statistics for DNS
requests and for sinkhole actions performed for each customer IP address.
The range is 1 through 60 minutes and the default is 5 minutes.[edit services web-filter profile profile-name dns-filter]user@host# set statistics-log-timer minutes
- Configure the time to live while sending the DNS response
after taking the DNS sinkhole action. The range is 0 through 86,400
seconds and the default is 1800.[edit services web-filter profile profile-name dns-filter]user@host# set dns-resp-ttl seconds
- Configure the level of subdomains that are searched for
a match. The range is 0 through 10. A value of 0 indicates that subdomains
are not searched.[edit services web-filter profile profile-name dns-filter]user@host# set wildcarding-level level
For example, if you set the wildcarding-level to 4 and the database file includes an entry for
example.com
, the following comparisons are made for a DNS request that arrives with the domain198.51.100.0.example.com
:198.51.100.0.example.com
: no match51.100.0.example.com
: no match for one level down100.0.example.com
: no match for two levels down0.example.com
: no match for three levels downexample.com
: match for four levels down
- Specify the name of the domain filter database to use
when filtering DNS requests.
- Configure a template. You can configure a maximum of 8
templates in a profile. Each template identifies filter settings for
DNS requests on specific uplink and downlink logical interfaces or
routing instances, or for DNS requests from specific source IP address
prefixes.
- Configure the name for the template.[edit services web-filter profile profile-name]user@host# set dns-filter-template template-name
- (Optional) Specify the client-facing logical interfaces
(uplink) to which the DNS filtering is applied.[edit services web-filter profile profile-name dns-filter-template template-name]user@host# set client-interfaces client-interface-name
- (Optional) Specify the server-facing logical interfaces
(downlink) to which the DNS filtering is applied.[edit services web-filter profile profile-name dns-filter-template template-name]user@host# set server-interfaces server-interface-name
- (Optional) Specify the routing instance for the client-facing
logical interface to which the DNS filtering is applied.[edit services web-filter profile profile-name dns-filter-template template-name]user@host# set client-routing-instance client-routing-instance-name
- (Optional) Specify the routing instance for the server-facing
logical interface to which DNS filtering is applied. [edit services web-filter profile profile-name dns-filter-template template-name]user@host# set server-routing-instance server-routing-instance-name
Note If you configure the client and server interfaces or the client and server routing instances, implicit filters are installed on the interfaces or routing instances to direct DNS traffic to the services PIC for DNS filtering. If you configure neither the client and server interfaces nor the routing instances, you must provide a way to direct DNS traffic to the services PIC (for example, via routes).
- Specify the name of the domain filter database to use
when filtering DNS requests.[edit services web-filter profile profile-name dns-filter-template template-name dns-filter]user@host# set database-file filename
- (Optional) To limit DNS filtering to DNS requests that
are destined for specific DNS servers, specify up to three IP addresses
(IPv4 or IPv6).[edit services web-filter profile profile-name dns-filter-template template-name dns-filter]user@host# set dns-server ip-address
- Specify the hash method that was used to create the hashed
domain name in the domain filter database file.[edit services web-filter profile profile-name dns-filter-template template-name dns-filter]user@host# set hash-method hash-method-name
The only supported hash method is hmac-sha2-256.
- Specify the hash key that was used to create the hashed
domain name in the domain filter database file.[edit services web-filter profile profile-name dns-filter-template template-name dns-filter]user@host# set hash-key key-string
- Configure the interval for logging statistics for DNS
requests and for sinkhole actions performed for each customer IP address.
The range is 1 through 60 minutes and the default is 5 minutes.[edit services web-filter profile profile-name dns-filter-template template-name dns-filter]user@host# set statistics-log-timer minutes
- Configure the time to live while sending the DNS response
after taking the DNS sinkhole action. The range is 0 through 86,400
seconds and the default is 1800.[edit services web-filter profile profile-name dns-filter-template template-name dns-filter]user@host# set dns-resp-ttl seconds
- Configure the level of subdomains that are searched for
a match. The range is 0 through 10. A value of 0 indicates that subdomains
are not searched.[edit services web-filter profile profile-name dns-filter-template template-name dns-filter]user@host# set wildcarding-level level
For example, if you set the wildcarding-level to 4 and the database file includes an entry for
example.com
, the following comparisons are made for a DNS request that arrives with the domain198.51.100.0.example.com
:198.51.100.0.example.com
: no match51.100.0.example.com
: no match for one level down100.0.example.com
: no match for two levels down0.example.com
: no match for three levels downexample.com
: match for four levels down
- (Optional) Specify the response error code for SRV and TXT
query types.[edit services web-filter profile profile-name dns-filter-template template-name dns-filter]user@host# set txt-resp-err-code (Noerror | Refused)user@host# set srv-resp-err-code (Noerror | Refused)
- Configure a term for the template. You can configure a
maximum of 64 terms in a template.[edit services web-filter profile profile-name dns-filter-template template-name]user@host# set term term-name
- (Optional) Specify the source IP address prefixes of DNS
requests you want to filter. You can configure a maximum of 64 prefixes
in a term.[edit services web-filter profile profile-name dns-filter-template template-name term term-name]user@host# set from src-ip-prefix source-prefix
- Specify that the sinkhole action identified in the domain
filter database is performed on disallowed DNS requests.[edit services web-filter profile profile-name dns-filter-template template-name term term-name]user@host# set then dns-sinkhole
- Configure the name for the template.
How to Configure a Service Set for DNS Filtering
- Associate the DNS filter profile with a next-hop service
set and enable logging for DNS filtering. The service interface can
be an ms- or vms- interface Next Gen Services with MX-SPC3 services
card), or it can be an aggregated multiservices (AMS) interface.[edit services service-set service-set-name]user@host# set web-filter-profile profile-nameuser@host# set syslog host hostname class urlf-logsuser@host# set next-hop-service inside-service-interface interface-name.unit-numberuser@host# set next-hop-service outside-service-interface interface-name.unit-number
Integration of Juniper ATP Cloud and Web filtering on MX Routers
Overview
Juniper Sky™ Advanced Threat Prevention (Juniper ATP Cloud) is integrated with MX series routers to protect all hosts in your network against evolving security threats by employing cloud-based threat detection software with a next-generation firewall system.
This topic provides an overview of Juniper ATP Cloud, Policy Enforcer, Security Intelligence, Web filtering, and their benefits when integrated on MX Series routers (MX240, MX480 and MX960).
Benefits
Simplifies deployment and enhances the anti-threat capabilities when integrated with the MX routers.
Delivers protection against “zero-day” threats using a combination of tools to provide robust coverage against sophisticated, evasive threats.
Checks inbound and outbound traffic with policy enhancements that allow users to stop malware, quarantine infected systems, prevent data exfiltration, and disrupt lateral movement.
Supports High Availability to provide uninterrupted service.
Provides scalability to handle increasing loads that require more computing resources, increased network bandwidth to receive more customer submissions, and a large storage for malware.
Provides deep inspection, actionable reporting, and inline malware blocking.
Understanding Policy Enforcer and Juniper ATP Cloud
Juniper Networks Security Director comprises a feature called the Policy Enforcer (PE) that enables it to learn from threat conditions, automate the policy creation, and to dynamically deploy enforcement to Juniper devices in the network.
Figure 2 illustrates the traffic flow between the PE, the Juniper ATP Cloud, and the MX router which functions as a firewall.
Policy Enforcer (PE) learns from threat conditions, automates the policy creation, and deploys enforcement to Juniper devices in the network.
Juniper Sky™ Advanced Threat Prevention (Juniper ATP Cloud) protects all hosts in your network by employing cloud-based threat detection software with a next-generation firewall system.
MX router fetches the threat intelligence feeds from Policy Enforcer (PE) and implements those policies to quarantine compromised hosts. It comprises of the following important components:
Security Intelligence process
Web Filtering process
Firewall process


To understand the functionality of the system architecture consider the following example—if a user downloads a file from the Internet and that file passes through an MX firewall, the file can be sent to the Juniper ATP Cloud cloud for malware inspection (depending on your configuration settings.) If the file is determined to be malware, PE identifies the IP address and MAC address of the host that downloaded the file. Based on a user-defined policy, that host can be put into a quarantine VLAN or blocked from accessing the Internet.
MX Series routers (MX240, MX480, and MX960) can be integrated with the Juniper ATP Cloud to prevent compromised hosts (botnets) from communicating with command and control servers:
Starting in Junos OS Release 18.4R1 with the Adaptive Services as an Inline security capability
Starting in Junos OS Release 19.3R2 with the Next Gen Services as an Inline security capability
Security Intelligence (SecIntel) - Overview
The Security Intelligence process (IPFD), is responsible for
downloading the security intelligence feeds and parsing from the feed
connector or ATP Cloud cloud feed server. The IPFD process on the
MX platforms fetches the command and control IPv4/IPv6 feeds from Policy
Enforcer. C&C feeds are essentially a list of servers that are
known command and control servers for botnets. The list also includes
servers that are known sources for malware downloads. The information
thus fetched is saved in a file (urlf_si_cc_db.txt
) created under the /var/db/url-filterd
directory.
The file format of the disallowed IPs sent by IPFD to the web filtering process is as follows:
IPv4 address | IPv6 address, threat-level.
The threat-level is an integer ranging from 1 to 10 to indicate the threat level of files scanned for malware and for infected hosts. Here, 1 represents the lowest threat level and 10 represents the highest threat level.
For example: 178.10.19.20, 4
Here, 178.10.19.20 indicates the disallowed IP and 4 indicates the threat-level.
The C&C feed database is synced onto the backup Routing Engine. IPFD then shares the information to the web filtering process (url-filterd). The web filtering process reads the file contents and configures the filters accordingly.
Configuring Security Intelligence to Download the CC Feed from Policy Enforcer
To download the command and control IPv4/IPv6 feeds from Juniper ATP Cloud/Policy Enforcer, include the security-intelligence statement at the [edit services] hierarchy as shown in the following example:
Web Filtering (URL-Filterd) - Overview
The web filtering process reads the file contents fetched from the IPFD and configures the filters on the Packet Forwarding Engine accordingly. The web filtering process enforces the command and control feeds by programming the filters in the Packet Forwarding Engine to block the packets destined to the blocked IP addresses and to generate logs for reporting the incident.
Figure 3 illustrates the way C&C feed is fetched by the IPFD and then processed by the web filtering process.


The web filter profile can have more than one templates. Each template consists of a set of configured logical interfaces for Web filtering and one or more terms. A term is a set of match criteria with actions to be taken if the match criteria is met. To configure the web filter profile to use dynamically fetched C&C feed, you can configure the security-intelligence-policy command under the [edit services web-filter profile profile-name hierarchy level. You need not configure a term for a security-intelligence-policy based web filter profiles.
You can configure the following threat level actions for the web filter profile at the edit web-filter profile profile-name security-intelligence-policy threat-level threat-level threat-action hierarchy level:
drop
drop-and-log
log
You can configure only one threat-action for each threat level. If the threat-action is not configured for a particular threat level, the default threat-action is accept.
Configuring the Web Filter Profile for Sampling
Starting in Junos OS Release 19.3R1, web filtering process (url-filterd) supports inline sampling of packets as a threat level action. The packets are dropped, logged, and sampled based on the threat-action you configure. For scaled scenarios, sampling of packets is preferred over the logging option. Along with the existing threat level actions, you can configure the following threat level actions on the web filter profile at the edit web-filter profile profile-name security-intelligence-policy threat-level threat-level threat-action hierarchy level:
drop-and-sample
drop-log-and-sample
log-and-sample
sample
The
inline flow monitoring samples the packets and sends the flow records
in IPFIX format to a flow collector. You can derive the threat level
for the sampled packets received at the external collector by matching
the received IP from the sampled packets with the corresponding IP
entry in /var/db/url-filterd/urlf_si_cc_db.txt
. You can configure sampling using any of the following methods:
Associate a sampling instance with the FPC on which the media interface is present at the [edit chassis] hierarchy level. If you are configuring sampling of IPv4 flows, IPv6 flows, or VPLS flows, you can configure the flow hash table size for each family.
Configure the template properties for inline flow monitoring at the [edit services flow-monitoring hierarchy level.
Configure a sampling instance and associate the flow-server IP address, port number, flow export rate, and specify the collectors at the [edit forwarding-options hierarchy level.
Associate a Sampling Instance with the FPC
To associate the defined instance with a particular FPC, MPC, or DPC, you include the sampling-instance statement at the [edit chassis fpc number] hierarchy level, as shown in the following example:
Configure a Sampling Instance and Associate the Template With the Sampling Instance.
To configure the template properties for inline flow monitoring, include the following statements at the edit services flow-monitoring hierarchy level as shown in the following example:
Configure the sample instance and associate the flow-server IP address and other parameters.
To configure a sampling instance and associate the flow-server IP address and other parameters. include the following statements at the [edit forwarding-options] hierarchy, as shown in the following example: