Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Integrated Web Filtering

 

Enhanced Web Filtering (EWF) with Websense is an integrated URL filtering solution. When you enable the solution on the device, the firewall intercepts the HTTP and the HTTPS requests and sends the HTTP URL or the HTTPS source IP to the Websense ThreatSeeker Cloud (TSC). For more information, see the following topics:

Understanding Integrated Web Filtering

The Integrated Web Filtering is not supported from Junos OS Release 15.1X49-D10 and Junos OS Release 17.3R1 onwards. For previous releases, with integrated Web filtering, the firewall intercepts every HTTP request in a TCP connection and extracts the URL from the HTTP request. Each individual HTTP request is blocked or permitted based on URL filtering profiles defined by you. The decision making is done on the device after it identifies a category for a URL.

The Surf-Control feature is not supported from Junos OS Release 15.1X49-D10 and Junos OS Release 17.3R1 onwards.

A URL category is a list of URLs grouped by content. URL categories are predefined and maintained by Surf-Control or are defined by you. Surf-Control maintains about 40 predefined categories. When defining your own URL categories, you can group URLs and create categories specific to your needs.

You define your own categories using URL pattern list and custom URL category list custom objects. Once defined, you can select your categories when you configure your Web filtering profile. Each category can have a maximum of 20 URLs. When you create a category, you can add either the URL or the IP address of a site. When you add a URL to a user-defined category, the device performs DNS lookup, resolves the host name into IP addresses, and caches this information. When a user tries to access a site with the IP address of the site, the device checks the cached list of IP addresses and tries to resolve the hostname. Many sites have dynamic IP addresses, meaning that their IP addresses change periodically. A user attempting to access a site can type an IP address that is not in the cached list on the device. Therefore, if you know the IP addresses of sites you are adding to a category, enter both the URL and the IP address(es) of the site.

If a URL appears in both a user-defined category and a predefined category, the device matches the URL to the user-defined category.

Web filtering is performed on all the methods defined in HTTP 1.0 and HTTP 1.1.

The integrated Web filtering solution intercepts every HTTP request in a TCP connection. In this case, the decision making is done on the device after it identifies the category for a URL either from user-defined categories or from a category server (SurfControl Content Portal Authority provided by Websense). The Integrated Web filtering is not supported from Junos OS Release 15.1X49-D10 onwards.

The integrated Web filtering feature is a separately licensed subscription service. When the license key for Web filtering has expired, no URLs are sent to the category server for checking, only local user-defined categories are checked.

Integrated Web filtering solution is supported only on SRX210, SRX220, SRX240, SRX550, and SRX650 devices.

This topic contains the following sections:

Integrated Web Filtering Process

This is a general description of how Web traffic is intercepted and acted upon by the Web filtering module.

  1. The device intercepts a TCP connection.
  2. The device intercepts each HTTP request in the TCP connection.
  3. The device extracts each URL in the HTTP request and checks its URL filter cache.
  4. Global Web filtering allowlists and blocklists are checked first for block or permit.
  5. If the HTTP request URL is allowed based on cached parameters, it is forwarded to the webserver. If there is no cache match, a request for categorization is sent to the SurfControl server. (If the HTTP request URL is blocked, the request is not forwarded and a notification message is logged.)
  6. In the allowed case, the SurfControl server responds with the corresponding category.
  7. Based on the identified category, if the URL is permitted, the device forwards the HTTP request to the webserver. If the URL is not permitted, then a deny page is sent to the HTTP client.

Integrated Web Filtering Cache

By default, the device retrieves and caches the URL categories from the SurfControl CPA server. This process reduces the overhead of accessing the SurfControl CPA server each time the device receives a new request for previously requested URLs. You can configure the size and duration of the cache, according to the performance and memory requirements of your networking environment. The lifetime of cached items is configurable between 1 and 1800 seconds with a default value of 300 seconds.

Caches are not preserved across device reboots or power losses.

Integrated Web Filtering Profiles

You configure Web filtering profiles that permit or block URLs according to defined categories. A Web filtering profile consists of a group of URL categories assigned one of the following actions:

  • Permit — The device always allows access to the websites in this category.

  • Block — The device blocks access to the websites in this category. When the device blocks access to this category of websites, it displays a message in your browser indicating the URL category.

  • Blocklist — The device always blocks access to the websites in this list. You can create a user-defined category.

  • Allowlist — The device always allows access to the websites in this list. You can create a user-defined category.

Note

A predefined profile is provided and can be used if you choose not to define your own profile.

A Web filtering profile may contain one blocklist or one allowlist, multiple user-defined and/or predefined categories each with a permit or block action, and an Other category with a permit or block action. You can define an action for all Other categories in a profile to specify what to do when the incoming URL does not belong to any of the categories defined in the profile. If the action for the Other category is block, the incoming URL is blocked if it does not match any of the categories explicitly defined in the profile. If an action for the Other category is not specified, the default action of permit is applied to the incoming URL not matching any category.

Profile Matching Precedence

When a profile employs several categories for URL matching, those categories are checked for matches in the following order:

  1. If present, the global blocklist is checked first. If a match is made, the URL is blocked. If no match is found...
  2. The global allowlist is checked next. If a match is made, the URL is permitted. If no match is found...
  3. User-defined categories are checked next. If a match is made, the URL is blocked or permitted as specified. If no match is found...
  4. Predefined categories are checked next. If a match is made, the URL is blocked or permitted as specified. If no match is found...
  5. The Other category is checked next. If a match is made, the URL is blocked or permitted as specified.

Example: Configuring Integrated Web Filtering

The Integrated Web Filtering is not supported from Junos OS Release 15.1X49-D10 and Junos OS Release 17.3R1 onwards. For previous releases, this example shows how to configure integrated Web filtering.

Requirements

Before you begin, learn more about Web filtering. See Web Filtering Overview.

Overview

In this example you configure integrated Web filtering custom objects, integrated Web filtering feature profiles, and integrated Web filtering UTM policies. You also attach integrated Web filtering UTM policies to security policies.

In the first example configuration you create a custom object called urllist3 that contains the pattern http://www.example.net 1.2.3.4. The urllist3 custom object is then added to the custom URL category custurl3.

In the second example configuration, you configure the Web filtering feature profile. You set the URL blocklist filtering category to custblacklist, set the allowlist filtering category to custwhitelist and the type of Web filtering engine to surf-control-integrated. Then you set the cache size parameters for Web filtering to 500 KB, which is the default, and the cache timeout parameters to 1800.

You name the Surf Control server as surfcontrolserver and enter 8080 as the port number for communicating with it. (Default ports are 80, 8080, and 8081.) Then you create a surf-control-integrated profile name called surfprofile1.

Next you select a category from the included allowlist and blocklist categories or select a custom URL category list you created for filtering against. Then you enter an action (permit, log and permit, block) to go with the filter. You do this as many times as necessary to compile your allowlists and blocklists and their accompanying actions. This example blocks URLs in the custurl3 category.

Then you enter a custom message to be sent when HTTP requests are blocked. This example configures the device to send an ***access denied*** message. You select a default action (permit, log and permit, block) for this profile for requests that experience errors. This example sets the default action to block. You select fallback settings (block or log and permit) for this profile, in case errors occur in each configured category. This example sets fallback settings to block.

Finally, you enter a timeout value in seconds. Once this limit is reached, fail mode settings are applied. The default is 10 seconds, and you can enter a value from 10 to 240 seconds. This example sets the timeout value to 10.

In the third example configuration, you create UTM policy utmp5 and attach it to profile surfprofile1.

In the final example configuration, you attach the UTM policy utmp5 to the security policy p5.

Configuration

Configuring Integrated Web Filtering Custom Objects

CLI Quick Configuration

To quickly configure this section of the example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

Custom category does not take precedence over predefined categories when it has the same name as one of the predefined categories. We do not recommend having a custom category name be the same as the predefined category name.

Step-by-Step Procedure

To configure integrated Web filtering:

  1. Create custom objects and create the URL pattern list.
  2. Configure the custom URL category list custom object using the URL pattern list.
  3. Create a list of untrusted sites.
  4. Configure the custom URL category list custom object using the URL pattern list of untrusted sites.
  5. Create a list of trusted sites.
  6. Configure the custom URL category list custom object using the URL pattern list of trusted sites.

Results

From configuration mode, confirm your configuration by entering the show security utm custom-objects command. If the output does not display the intended configuration, repeat the configuration instructions in this example to correct it.

For brevity, this show command output includes only the configuration that is relevant to this example. Any other configuration on the system has been replaced with ellipses (...).

If you are done configuring the device, enter commit from configuration mode.

Configuring the Integrated Web Filtering Feature Profiles

CLI Quick Configuration

To quickly configure this section of the example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

Step-by-Step Procedure

The following example requires you to navigate various levels in the configuration hierarchy. For instructions on how to do that, see Using the CLI Editor in Configuration Mode in the CLI User Guide.

To configure integrated Web filtering feature profiles:

  1. Configure the Web filtering URL Blocklist.
  2. Configure the Web filtering URL Allowlist.
  3. Specify the surf-control-integrated Web filtering engine and set the cache size parameters.
  4. Set the cache timeout parameters.
  5. Set the server name or IP address.
  6. Enter the port number for communicating with the server.
  7. Create a profile name and select a category from the included allowlist and blocklist categories.
  8. Enter a custom message to be sent when HTTP requests are blocked.
  9. Select a default action (permit, log and permit, block) for this profile for requests that experience errors.
  10. Select fallback settings (block or log and permit) for this profile.
  11. Enter a timeout value, in seconds.

Results

From configuration mode, confirm your configuration by entering the show security utm feature-profile command. If the output does not display the intended configuration, repeat the configuration instructions in this example to correct it.

If you are done configuring the device, enter commit from configuration mode.

Configuring Integrated Web Filtering UTM Policies

CLI Quick Configuration

To quickly configure this section of the example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

Step-by-Step Procedure

To configure a UTM policy:

  1. Create the UTM policy referencing a profile.

Results

From configuration mode, confirm your configuration by entering the show security utm utm-policy command. If the output does not display the intended configuration, repeat the configuration instructions in this example to correct it.

If you are done configuring the device, enter commit from configuration mode.

Attaching Integrated Web Filtering UTM Policies to Security Policies

CLI Quick Configuration

To quickly configure this section of the example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

Step-by-Step Procedure

To attach a UTM policy to a security policy:

  1. Create and configure the security policy.
  2. Attach the UTM policy to the security policy.

Results

From configuration mode, confirm your configuration by entering the show security policies command. If the output does not display the intended configuration, repeat the configuration instructions in this example to correct it.

If you are done configuring the device, enter commit from configuration mode.

Verification

To confirm that the configuration is working properly, perform these tasks:

Verifying the Configuration of Integrated Web Filtering Custom Objects

Purpose

Verify the configuration of integrated Web filtering custom objects.

Action

From the top of the configuration in configuration mode, enter the show security utm custom-objects command.

Verifying the Configuration of Integrated Web Filtering Feature Profiles

Purpose

Verify the configuration of integrated Web filtering feature profiles.

Action

From the top of the configuration in configuration mode, enter the show security utm feature-profile command.

Verifying the Configuration of Integrated Web Filtering UTM Policies

Purpose

Verify the configuration of integrated Web filtering UTM policies.

Action

From the top of the configuration in configuration mode, enter the show security utm command.

Verifying the Attachment of Integrated Web Filtering UTM Policies to Security Policies

Purpose

Verify the attachment of integrated Web filtering UTM policies to security policies.

Action

From the top of the configuration in configuration mode, enter the show security policies command.

Displaying Global SurfControl URL Categories

Purpose

The Surf-Control feature is not supported from Junos OS Release 15.1X49-D10 and Junos OS Release 17.3R1 onwards. For previous releases, view global URL categories defined and maintained by SurfControl.

Action

Enter the user@host# show groups junos-defaults CLI command. You can also look for custom-url-category.

Release History Table
Release
Description
The Integrated Web Filtering is not supported from Junos OS Release 15.1X49-D10 and Junos OS Release 17.3R1 onwards.
The Surf-Control feature is not supported from Junos OS Release 15.1X49-D10 and Junos OS Release 17.3R1 onwards.
The Integrated Web filtering is not supported from Junos OS Release 15.1X49-D10 onwards.
The Integrated Web Filtering is not supported from Junos OS Release 15.1X49-D10 and Junos OS Release 17.3R1 onwards.
The Surf-Control feature is not supported from Junos OS Release 15.1X49-D10 and Junos OS Release 17.3R1 onwards.