URL Filtering

If you change the contents of the URL filter database file, use the request services (url-filter | web-filter) update command. Other commands to help maintain the URL filter database file include the following:

To configure the URL profile:

DNS Request Filtering for Disallowed Website Domains

• Overview of DNS Request Filtering

• How to Configure DNS Request Filtering

Overview of DNS Request Filtering

Starting in Junos OS Release 18.3R1, you can configure DNS filtering to identify DNS requests for disallowed website domains. Starting in Junos OS Release 19.3R2, you can configure DNS filtering if you are running Next Gen Services with the MX-SPC3 services card. Next Gen Services are supported on MX240, MX480 and MX960 routers. For DNS request types A, AAAA, MX, CNAME, TXT, SRV, and ANY, you configure the action to take for a DNS request for a disallowed domain. You can either:

• Block access to the website by sending a DNS response that contains the IP address or fully qualified domain name (FQDN) of a DNS sinkhole server. This ensures that when the client attempts to send traffic to the disallowed domain, the traffic instead goes to the sinkhole server (see Figure 1).

• Log the request and allow access.

For other DNS request types for a disallowed domain, the request is logged and access is allowed.

The actions that the sinkhole server takes are not controlled by the DNS request filtering feature; you are responsible for configuring the sinkhole server actions. For example, the sinkhole server could send a message to the requestor that the domain is not reachable and prevent access to the disallowed domain.

Figure 1: DNS Request for Disallowed Domain

• Benefits

• Disallowed Domain Filter Database File

• DNS Filter Profile

Benefits

DNS filtering redirects DNS requests for disallowed website domains to sinkhole servers, while preventing anyone operating the system from seeing the list of disallowed domains. This is because the disallowed domain names are in an encrypted format.

Disallowed Domain Filter Database File

DNS request filtering requires a disallowed domain filter database .txt file, which identifies each disallowed domain name, the action to take on a DNS request for the disallowed domain, and the IP address or fully qualified domain name (FQDN) of a DNS sinkhole server.

DNS Filter Profile

You configure a DNS filter profile to specify which disallowed domain filter database file to use. You can also specify the interfaces on which DNS request filtering is performed, limit the filtering to requests for specific DNS servers, and limit the filtering to requests from specific source IP address prefixes.

How to Configure DNS Request Filtering

• How to Configure a Domain Filter Database

• How to Configure a DNS Filter Profile

• How to Configure a Service Set for DNS Filtering

How to Configure a Domain Filter Database

1. Create the name for the file. The database file name can have a maximum length of 64 characters and must have a .txt extension.
2. Add a file header with a format such as 20170314_01:domain,sinkhole_ip,v6_sinkhole,sinkhole_fqdn,id,action.
3. Add an entry in the file for each disallowed domain. You can include a maximum of 10,000 domain entries. Each entry in the database file has the following items:

   hashed-domain-name,IPv4 sinkhole address, IPv6 sinkhole address, sinkhole FQDN, ID, action

   where:

   • hashed-domain-name is a hashed value of the disallowed domain name (64 hexadecimal characters). The hash method and hash key that you use to produce the hashed domain value are needed when you configure DNS filtering with the Junos OS CLI.

   • IPv4 sinkhole address is the address of the DNS sinkhole server for IPv4 DNS requests.

   • IPv6 sinkhole address is the address of the DNS sinkhole server for IPv6 DNS requests.

   • sinkhole FQDN is the fully qualified domain name of the DNS sinkhole server.

   • ID is a 32-bit number that uniquely associates the entry with the hashed domain name.

   • action is the action to apply to a DNS request that matches the disallowed domain name. If you enter replace, the MX Series router sends the client a DNS response with the IP address or FQDN of the DNS sinkhole server. If you enter report, the DNS request is logged and then sent to the DNS server.

4. In the last line of the file, include the file hash, which you calculate by using the same key and hash method that you used to produce the hashed domain names.
5. Save the database files on the Routing Engine in the /var/db/url-filterd directory.
6. Validate the domain filter database file.
7. If you make any changes to the database file, apply the changes.

How to Configure a DNS Filter Profile

1. Configure the name for a DNS filter profile:

   The maximum number of profiles is 8.

2. Configure the interval for logging per-client statistics for DNS filtering. The range is 0 through 60 minutes and the default is 5 minutes.
3. Configure general DNS filtering settings for the profile. These values are used if a DNS request does not match a specific template.
   1. Specify the name of the domain filter database to use when filtering DNS requests.
   2. (Optional) To limit DNS filtering to DNS requests that are destined for specific DNS servers, specify up to three IP addresses (IPv4 or IPv6).
   3. Specify the format for the hash key.
   4. Specify the hash key that you used to create the hashed domain name in the domain filter database file.
   5. Specify the hash method that was used to create the hashed domain name in the domain filter database file.

      The only supported hash method is hmac-sha2-256.

   6. Configure the interval for logging statistics for DNS requests and for sinkhole actions performed for each customer IP address. The range is 1 through 60 minutes and the default is 5 minutes.
   7. Configure the time to live while sending the DNS response after taking the DNS sinkhole action. The range is 0 through 86,400 seconds and the default is 1800.
   8. Configure the level of subdomains that are searched for a match. The range is 0 through 10. A value of 0 indicates that subdomains are not searched.

      For example, if you set the wildcarding-level to 4 and the database file includes an entry for example.com, the following comparisons are made for a DNS request that arrives with the domain 198.51.100.0.example.com:

      • 198.51.100.0.example.com: no match

      • 51.100.0.example.com: no match for one level down

      • 100.0.example.com: no match for two levels down

      • 0.example.com: no match for three levels down

      • example.com: match for four levels down

4. Configure a template. You can configure a maximum of 8 templates in a profile. Each template identifies filter settings for DNS requests on specific uplink and downlink logical interfaces or routing instances, or for DNS requests from specific source IP address prefixes.
   1. Configure the name for the template.
   2. (Optional) Specify the client-facing logical interfaces (uplink) to which the DNS filtering is applied.
   3. (Optional) Specify the server-facing logical interfaces (downlink) to which the DNS filtering is applied.
   4. (Optional) Specify the routing instance for the client-facing logical interface to which the DNS filtering is applied.
   5. (Optional) Specify the routing instance for the server-facing logical interface to which DNS filtering is applied.
      Note

      If you configure the client and server interfaces or the client and server routing instances, implicit filters are installed on the interfaces or routing instances to direct DNS traffic to the services PIC for DNS filtering. If you configure neither the client and server interfaces nor the routing instances, you must provide a way to direct DNS traffic to the services PIC (for example, via routes).

   6. Specify the name of the domain filter database to use when filtering DNS requests.
   7. (Optional) To limit DNS filtering to DNS requests that are destined for specific DNS servers, specify up to three IP addresses (IPv4 or IPv6).
   8. Specify the hash method that was used to create the hashed domain name in the domain filter database file.

      The only supported hash method is hmac-sha2-256.

   9. Specify the hash key that was used to create the hashed domain name in the domain filter database file.
   10. Configure the interval for logging statistics for DNS requests and for sinkhole actions performed for each customer IP address. The range is 1 through 60 minutes and the default is 5 minutes.
   11. Configure the time to live while sending the DNS response after taking the DNS sinkhole action. The range is 0 through 86,400 seconds and the default is 1800.
   12. Configure the level of subdomains that are searched for a match. The range is 0 through 10. A value of 0 indicates that subdomains are not searched.

       For example, if you set the wildcarding-level to 4 and the database file includes an entry for example.com, the following comparisons are made for a DNS request that arrives with the domain 198.51.100.0.example.com:

       • 198.51.100.0.example.com: no match

       • 51.100.0.example.com: no match for one level down

       • 100.0.example.com: no match for two levels down

       • 0.example.com: no match for three levels down

       • example.com: match for four levels down

   13. (Optional) Specify the response error code for SRV and TXT query types.
   14. Configure a term for the template. You can configure a maximum of 64 terms in a template.
   15. (Optional) Specify the source IP address prefixes of DNS requests you want to filter. You can configure a maximum of 64 prefixes in a term.
   16. Specify that the sinkhole action identified in the domain filter database is performed on disallowed DNS requests.

How to Configure a Service Set for DNS Filtering

• Associate the DNS filter profile with a next-hop service set and enable logging for DNS filtering. The service interface can be an ms- or vms- interface Next Gen Services with MX-SPC3 services card), or it can be an aggregated multiservices (AMS) interface.

Integration of Juniper ATP Cloud and Web filtering on MX Routers

• Overview

• Configuring the Web Filter Profile for Sampling

Overview

• Benefits

• Understanding Policy Enforcer and Juniper ATP Cloud

• Security Intelligence (SecIntel) - Overview

• Web Filtering (URL-Filterd) - Overview

Benefits

• Simplifies deployment and enhances the anti-threat capabilities when integrated with the MX routers.

• Delivers protection against “zero-day” threats using a combination of tools to provide robust coverage against sophisticated, evasive threats.

• Checks inbound and outbound traffic with policy enhancements that allow users to stop malware, quarantine infected systems, prevent data exfiltration, and disrupt lateral movement.

• Supports High Availability to provide uninterrupted service.

• Provides scalability to handle increasing loads that require more computing resources, increased network bandwidth to receive more customer submissions, and a large storage for malware.

• Provides deep inspection, actionable reporting, and inline malware blocking.

Understanding Policy Enforcer and Juniper ATP Cloud

Juniper Networks Security Director comprises a feature called the Policy Enforcer (PE) that enables it to learn from threat conditions, automate the policy creation, and to dynamically deploy enforcement to Juniper devices in the network.

Figure 2 illustrates the traffic flow between the PE, the Juniper ATP Cloud, and the MX router which functions as a firewall.

• Policy Enforcer (PE) learns from threat conditions, automates the policy creation, and deploys enforcement to Juniper devices in the network.

• Juniper Sky™ Advanced Threat Prevention (Juniper ATP Cloud) protects all hosts in your network by employing cloud-based threat detection software with a next-generation firewall system.

• MX router fetches the threat intelligence feeds from Policy Enforcer (PE) and implements those policies to quarantine compromised hosts. It comprises of the following important components:

  • Security Intelligence process

  • Web Filtering process

  • Firewall process

Figure 2: System Architecture

To understand the functionality of the system architecture consider the following example—if a user downloads a file from the Internet and that file passes through an MX firewall, the file can be sent to the Juniper ATP Cloud cloud for malware inspection (depending on your configuration settings.) If the file is determined to be malware, PE identifies the IP address and MAC address of the host that downloaded the file. Based on a user-defined policy, that host can be put into a quarantine VLAN or blocked from accessing the Internet.

MX Series routers (MX240, MX480, and MX960) can be integrated with the Juniper ATP Cloud to prevent compromised hosts (botnets) from communicating with command and control servers:

• Starting in Junos OS Release 18.4R1 with the Adaptive Services as an Inline security capability

• Starting in Junos OS Release 19.3R2 with the Next Gen Services as an Inline security capability

Security Intelligence (SecIntel) - Overview

The Security Intelligence process (IPFD), is responsible for downloading the security intelligence feeds and parsing from the feed connector or ATP Cloud cloud feed server. The IPFD process on the MX platforms fetches the command and control IPv4/IPv6 feeds from Policy Enforcer. C&C feeds are essentially a list of servers that are known command and control servers for botnets. The list also includes servers that are known sources for malware downloads. The information thus fetched is saved in a file (urlf_si_cc_db.txt) created under the /var/db/url-filterd directory.

The file format of the disallowed IPs sent by IPFD to the web filtering process is as follows:

IPv4 address | IPv6 address, threat-level.

The threat-level is an integer ranging from 1 to 10 to indicate the threat level of files scanned for malware and for infected hosts. Here, 1 represents the lowest threat level and 10 represents the highest threat level.

For example: 178.10.19.20, 4

Here, 178.10.19.20 indicates the disallowed IP and 4 indicates the threat-level.

The C&C feed database is synced onto the backup Routing Engine. IPFD then shares the information to the web filtering process (url-filterd). The web filtering process reads the file contents and configures the filters accordingly.

Configuring Security Intelligence to Download the CC Feed from Policy Enforcer

To download the command and control IPv4/IPv6 feeds from Juniper ATP Cloud/Policy Enforcer, include the security-intelligence statement at the [edit services] hierarchy as shown in the following example:

Web Filtering (URL-Filterd) - Overview

The web filtering process reads the file contents fetched from the IPFD and configures the filters on the Packet Forwarding Engine accordingly. The web filtering process enforces the command and control feeds by programming the filters in the Packet Forwarding Engine to block the packets destined to the blocked IP addresses and to generate logs for reporting the incident.

Figure 3 illustrates the way C&C feed is fetched by the IPFD and then processed by the web filtering process.

Figure 3: Web Filtering

The web filter profile can have more than one templates. Each template consists of a set of configured logical interfaces for Web filtering and one or more terms. A term is a set of match criteria with actions to be taken if the match criteria is met. To configure the web filter profile to use dynamically fetched C&C feed, you can configure the security-intelligence-policy command under the [edit services web-filter profile profile-name hierarchy level. You need not configure a term for a security-intelligence-policy based web filter profiles.

You can configure the following threat level actions for the web filter profile at the edit web-filter profile profile-name security-intelligence-policy threat-level threat-level threat-action hierarchy level:

• drop

• drop-and-log

• log

You can configure only one threat-action for each threat level. If the threat-action is not configured for a particular threat level, the default threat-action is accept.

Configuring the Web Filter Profile for Sampling

Starting in Junos OS Release 19.3R1, web filtering process (url-filterd) supports inline sampling of packets as a threat level action. The packets are dropped, logged, and sampled based on the threat-action you configure. For scaled scenarios, sampling of packets is preferred over the logging option. Along with the existing threat level actions, you can configure the following threat level actions on the web filter profile at the edit web-filter profile profile-name security-intelligence-policy threat-level threat-level threat-action hierarchy level:

• drop-and-sample

• drop-log-and-sample

• log-and-sample

• sample

The inline flow monitoring samples the packets and sends the flow records in IPFIX format to a flow collector. You can derive the threat level for the sampled packets received at the external collector by matching the received IP from the sampled packets with the corresponding IP entry in /var/db/url-filterd/urlf_si_cc_db.txt. You can configure sampling using any of the following methods:

• Associate a sampling instance with the FPC on which the media interface is present at the [edit chassis] hierarchy level. If you are configuring sampling of IPv4 flows, IPv6 flows, or VPLS flows, you can configure the flow hash table size for each family.

• Configure the template properties for inline flow monitoring at the [edit services flow-monitoring hierarchy level.

• Configure a sampling instance and associate the flow-server IP address, port number, flow export rate, and specify the collectors at the [edit forwarding-options hierarchy level.

Associate a Sampling Instance with the FPC

To associate the defined instance with a particular FPC, MPC, or DPC, you include the sampling-instance statement at the [edit chassis fpc number] hierarchy level, as shown in the following example:

Configure a Sampling Instance and Associate the Template With the Sampling Instance.

To configure the template properties for inline flow monitoring, include the following statements at the edit services flow-monitoring hierarchy level as shown in the following example:

Configure the sample instance and associate the flow-server IP address and other parameters.

To configure a sampling instance and associate the flow-server IP address and other parameters. include the following statements at the [edit forwarding-options] hierarchy, as shown in the following example:

Example: Configuring Web-filter Profile to Define Different Threat-Levels