URL Filtering

Benefits

DNS filtering redirects DNS requests for disallowed website domains to sinkhole servers, while preventing anyone operating the system from seeing the list of disallowed domains. This is because the disallowed domain names are in an encrypted format.

Disallowed Domain Filter Database File

DNS request filtering requires a disallowed domain filter database .txt file, which identifies each disallowed domain name, the action to take on a DNS request for the disallowed domain, and the IP address or fully qualified domain name (FQDN) of a DNS sinkhole server.

DNS Filter Profile

You configure a DNS filter profile to specify which disallowed domain filter database file to use. You can also specify the interfaces on which DNS request filtering is performed, limit the filtering to requests for specific DNS servers, and limit the filtering to requests from specific source IP address prefixes.

Overview

Starting in Junos OS Release 21.1R1, you can configure custom domain feeds per customer or IP subgroup. You can :

• Configure domain names and actions for multiple tenants such that domain feeds can be managed on a per tenant basis.
• Configure hierarchical domain feed management per profile, per dns-filter-template or per dns-filter-term.
• Exempt domain feeds at the IP, subnet, or CIDR level.

To implement the mutiltenant support for DNS filtering, creating the domain filter database file under template or profile level is disabled. You need not specify a file at the template or profile level. Starting in Junos OS 21.1R1, by default, a global file with a fixed name, nsf_multi_tenant_dn_custom_file.txt (plain text format) or dnsf_multi_tenant_dn_custom_file_hashed.txt (encrypted file) is available.

Each entry in the database file has the following items:

hashed-domain-name, IPv4 sinkhole address, IPv6 sinkhole address, sinkhole FQDN, ID, action, feed-name.

The file hash is calculated and appended to the list of domain name entries in the file. The file hash is calculated using a global key and method ,which is validated with the file hash computed using the hash key configured at the [edit services web-filter] hierarchy. The file validation is successful only if the calculated file-hash matches the file hash present in the file.

Each entry in nsf_multi_tenant_dn_custom_file.txt file consists of an additional field called feed-name. This feed-name s used as an indicator to group set of domain-names and map them to a tenant (profile, template, term, or IP address).

When the DNS packets are received from a particular SRC IP address, the corresponding feed-name is fetched and lookup happens against the domain-names mapped with the feed-name associated with the term. If the feed-name is not provisioned for that IP address, then it falls back to the feed-name configured at the template-level and lookup happens against the domain-names mapped with the feed-name associated with the template. If the feed-name is not configured at template, then the lookup is against the domain-names mapped against the feed-name associated with the profile.

Benefits

• Simplifies deployment and enhances the anti-threat capabilities when integrated with the MX routers.

• Delivers protection against “zero-day” threats using a combination of tools to provide robust coverage against sophisticated, evasive threats.

• Checks inbound and outbound traffic with policy enhancements that allow users to stop malware, quarantine infected systems, prevent data exfiltration, and disrupt lateral movement.

• Supports High Availability to provide uninterrupted service.

• Provides scalability to handle increasing loads that require more computing resources, increased network bandwidth to receive more customer submissions, and a large storage for malware.

• Provides deep inspection, actionable reporting, and inline malware blocking.

Understanding Policy Enforcer and Juniper ATP Cloud

Juniper Networks Security Director comprises a feature called the Policy Enforcer (PE) that enables it to learn from threat conditions, automate the policy creation, and to dynamically deploy enforcement to Juniper devices in the network.

Figure 4 illustrates the traffic flow between the PE, the Juniper ATP Cloud, and the MX router which functions as a firewall.

• Policy Enforcer (PE) learns from threat conditions, automates the policy creation, and deploys enforcement to Juniper devices in the network.

• Juniper Advanced Threat Prevention (Juniper ATP Cloud) protects all hosts in your network by employing cloud-based threat detection software with a next-generation firewall system.

• MX router fetches the threat intelligence feeds from Policy Enforcer (PE) and implements those policies to quarantine compromised hosts. It comprises of the following important components:

  • Security Intelligence process

  • Web Filtering process

  • Firewall process

Figure 4: System Architecture

To understand the functionality of the system architecture consider the following example—if a user downloads a file from the Internet and that file passes through an MX firewall, the file can be sent to the Juniper ATP Cloud cloud for malware inspection (depending on your configuration settings.) If the file is determined to be malware, PE identifies the IP address and MAC address of the host that downloaded the file. Based on a user-defined policy, that host can be put into a quarantine VLAN or blocked from accessing the Internet.

MX Series routers (MX240, MX480, and MX960) can be integrated with the Juniper ATP Cloud to prevent compromised hosts (botnets) from communicating with command and control servers:

• Starting in Junos OS Release 18.4R1 with the Adaptive Services as an Inline security capability

• Starting in Junos OS Release 19.3R2 with the Next Gen Services as an Inline security capability

Security Intelligence (SecIntel) - Overview

The Security Intelligence process (IPFD), is responsible for downloading the security intelligence feeds and parsing from the feed connector or ATP Cloud cloud feed server. The IPFD process on the MX platforms fetches the command and control IPv4/IPv6 feeds from Policy Enforcer. C&C feeds are essentially a list of servers that are known command and control servers for botnets. The list also includes servers that are known sources for malware downloads. The information thus fetched is saved in a file (urlf_si_cc_db.txt) created under the /var/db/url-filterd directory.

The file format of the disallowed IPs sent by IPFD to the web filtering process is as follows:

IPv4 address | IPv6 address, threat-level.

The threat-level is an integer ranging from 1 to 10 to indicate the threat level of files scanned for malware and for infected hosts. Here, 1 represents the lowest threat level and 10 represents the highest threat level.

For example: 178.10.19.20, 4

Here, 178.10.19.20 indicates the disallowed IP and 4 indicates the threat-level.

The C&C feed database is synced onto the backup Routing Engine. IPFD then shares the information to the web filtering process (url-filterd). The web filtering process reads the file contents and configures the filters accordingly.

Web Filtering (URL-Filterd) - Overview

The web filtering process reads the file contents fetched from the IPFD and configures the filters on the Packet Forwarding Engine accordingly. The web filtering process enforces the command and control feeds by programming the filters in the Packet Forwarding Engine to block the packets destined to the blocked IP addresses and to generate logs for reporting the incident.

Figure 5 illustrates the way C&C feed is fetched by the IPFD and then processed by the web filtering process.

Figure 5: Web Filtering

The web filter profile can have more than one templates. Each template consists of a set of configured logical interfaces for Web filtering and one or more terms. A term is a set of match criteria with actions to be taken if the match criteria is met. To configure the web filter profile to use dynamically fetched C&C feed, you can configure the security-intelligence-policy command under the [edit services web-filter profile profile-name hierarchy level. You need not configure a term for a security-intelligence-policy based web filter profiles.

You can configure the following threat level actions for the web filter profile at the edit web-filter profile profile-name security-intelligence-policy threat-level threat-level threat-action hierarchy level:

• drop

• drop-and-log

• log

You can configure only one threat-action for each threat level. If the threat-action is not configured for a particular threat level, the default threat-action is accept.

Associate a Sampling Instance with the FPC

To associate the defined instance with a particular FPC, MPC, or DPC, you include the sampling-instance statement at the [edit chassis fpc number] hierarchy level, as shown in the following example:

Configure a Sampling Instance and Associate the Template With the Sampling Instance.

To configure the template properties for inline flow monitoring, include the following statements at the edit services flow-monitoring hierarchy level as shown in the following example:

Configure the sample instance and associate the flow-server IP address and other parameters.

To configure a sampling instance and associate the flow-server IP address and other parameters. include the following statements at the [edit forwarding-options] hierarchy, as shown in the following example:

Example: Configuring Web-filter Profile to Define Different Threat-Levels

Overview

The GeoIP feeds are essentially a list of IP address to country code mappings. Starting in Junos OS 21.4R1, you can configure IP-based Geo locations on MX Series routers to fetch the GeoIP feeds from Policy Enforcer. By deploying the GeoIP feeds, you can enable the network to prevent devices from communicating with IP addresses belonging to specific countries.

You can configure the security intelligence process (IPFD) on MX series routers to fetch the GeoIP feeds from Policy Enforcer. Similar to existing C&C IP or IPv6 feeds, IPFD downloads the GeoIP feeds from the Policy Enforcer. IPFD translates the feed in the file format that is processed by the web-filtering process (url-filterd) subsequently.

Starting in Junos OS 22.1R1, you can configure the security intelligence process (IPFD) on MX series routers to fetch the GeoIP feeds from Juniper ATP Cloud. Similar to existing C&C IP or IPv6 feeds, IPFD downloads the GeoIP feeds from the Juniper ATP Cloud.

How to Configure GeoIP Filtering on MX Series Routers

The information fetched by the IPFD is saved in a file (urlf_si_geoip_db.txt) created at the /var/db/url-filterd location.

The format of the file sent by IPFD to the web filtering process is as follows:

IPv4 address|IPv6 address,Prefix,threat-level,VRF-name,Gen-num. Gen-num is always 0. VRF-name refers to a country code.

For example, 178.10.19.22,12,255,US,0

IPFD and the web-filtering process maintain a pconn connection for communicating the creation or update of files containing GeoIP feeds. The Web-Filtering process enforces the GeoIP feeds by programming the filters in the PFE to block the packets destined to the blocked countries. The APIs provided by liburlf are used to validate and parse the files.

The web-filtering process reads the file containing the list of IP addresses and the PFE filters are programmed with the destination IP addresses listed in the feed and the action configured for the associated country.

• Global filter- Countries are configured under global rule within a profile. All IP addresses for countries specific to that global rule are programmed in a single filter and applied to all templates in the profile. You can configure a profile to dynamically fetch GeoIP feed by configuring geo-ip rule match country country-name at the [edit services web-filter profile profile-name security-intelligence-policy] hierarchy .

• Group filter- Groups of countries are configured under a template. All IP addresses associated with the countries for a Group are programmed in a group filter applied to the templates under which that group is configured. Group is a list of countries defined in a json file that is parsed by liburlf.

  To configure a group filter, you must configure a json file at the /var/db/url-filterd location, where the group.json file contains the group mappings.

  The format of the json file is as follows:

  [

  {

  "group_name" : "group1",

  "country" : ["ZA","YE"]

  },

  {

  "group_name" : "group2",

  "country" : ["YT"]

  }

  ]

  To dynamically fetch GeoIP feeds, you can configure a global filter using a single profile or configure multiple group filters using templates. We do not support both the configurations together.

  The groups created in the json file are referred in the GeoIP match clause defined at the [edit services web-filter profile profile-name url-filter-template template-name security-intelligence-policy geo-ip rule match group group-name] hierarchy.

You can choose to customize the IP feed by adding your own allowlist and blocklist. This can be helpful to manage intelligence feeds that are custom to your security operations center or as a temporary measure for false positives. Starting in Junos OS release 21.4R1, you can allow or block certain IP addresses based on configuration through a CLI or a file. You can either configure separate list for allowlist and a separate list for blocklist or include the IP addresses in a file and include the file name in the CLI configuration.

You can create an IP-address-list at the [edit services web-filter] hierarchy. Here, IP-address-list contains the list of IP addresses that must be allowed or blocked. You can also create a file containing the IP addresses that need to be allowed or blocked in the /var/db/url-filterd location. The IP addresses configured as a part of the file or IP address list are programmed as a part of the global filter, which is attached to all templates.

You can define a global allowlist by configuring white-list (IP-address-list | file-name) at the edit services web-filter profile profile-name security-intelligence-policy hierarchy. You can define a global blocklist by configuring the black-list (IP-address-list | file-name) at the edit services web-filter profile profile-name security-intelligence-policy hierarchy. Here, the IP-address-list, refers to the name of IP address-list specified at the [edit services web-filter] hierarchy. The file-name refers to the name of the file which contains the list of the IP addresses that must be allowed or blocked. The file must be in the /var/db/url-filterd location and must have the same name as in the configuration.

The format of the global allowlist file is as follows:

The format of the global blocklist file is as follows:

The web-filtering process parses the list of global allowlist or global blocklist IP addresses and programs the implicit filter terms with the configured IP addresses to either allow or block the packets.