Enhanced Web Filtering

 

Web Filtering provides URL filtering capability by using either a local Websense server or Internet-based SurfControl server. For more information, see the following topics:

Enhanced Web Filtering Overview

Enhanced Web Filtering (EWF) with Websense is an integrated URL filtering solution. When you enable the solution on the device, it intercepts the HTTP and the HTTPS requests and sends the HTTP URL or the HTTPS source IP to the Websense ThreatSeeker Cloud (TSC). The TSC categorizes the URL into one of the 95 or more categories that are predefined and also provides site reputation information. The TSC further returns the URL category and the site reputation information to the device. The device determines if it can permit or block the request based on the information provided by the TSC.

The Surf-Contol feature is not supported from Junos OS Release 15.1X49-D10 and Junos OS Release 17.3R1 onwards.

Starting in Junos OS Release 15.1X49-D40 and Junos OS Release 17.3R1, EWF supports HTTPS traffic by intercepting HTTPS traffic passing through the SRX Series device. The security channel from the SRX Series device is divided as one SSL channel between the client and the SRX Series device and another SSL channel between the SRX Series device and the HTTPS server. SSL forward proxy acts as the terminal for both channels and forwards the cleartext traffic to the UTM. UTM extracts the URL from the HTTP request message.

You can consider the EWF solution as the next-generation URL filtering solution, building upon the existing SurfControl solution.

Enhanced Web Filtering supports the following HTTP methods:

  • GET

  • POST

  • OPTIONS

  • HEAD

  • PUT

  • DELETE

  • TRACE

  • CONNECT

User Messages and Redirect URLs for Enhanced Web Filtering (EWF) on SRX Series Devices

Starting with Junos OS Release 15.1X49-D110, a new option, custom-message, is added for the custom-objects command that enables you to configure user messages and redirect URLs to notify users when a URL is blocked or quarantined for each EWF category. The custom-message option has the following mandatory attributes:

  • Name: Name of the custom message; maximum length is 59 bytes.

  • Type: Type of custom message: user-message or redirect-url.

  • Content: Content of the custom message; maximum length is 1024 bytes.

You configure a user message or redirect URL as a custom object and assign the custom object to an EWF category.

  • User messages indicate that website access has been blocked by an organization's access policy. To configure a user message, include the type user-message content message-text statement at the [edit security utm custom-objects custom-message message] hierarchy level.

  • Redirect URLs redirect a blocked or quarantined URL to a user-defined URL. To configure a redirect URL, include the type redirect-url content redirect-url statement at the [edit security utm custom-objects custom-message message] hierarchy level.

The custom-message option provides the following benefits:

  • You can configure a separate custom message or redirect URL for each EWF category.

  • The custom-message option allows you to fine-tune messages to support your polices to know which URL is blocked or quarantined.

Note

Only one custom-message configuration option is applied for each category. The custom-message configuration is supported only on Enhanced Web Filtering (EWF). Therefore, only the Juniper EWF engine type is supported.

Starting with Junos OS Release 17.4R1, support for custom category configuration is available for local and Websense redirect profiles.

Understanding the Enhanced Web Filtering Process

Web filtering enables you to manage Internet access, preventing access to inappropriate Web content. The Enhanced Web Filtering (EWF) feature intercepts, scans, and acts upon HTTP or HTTPS traffic in the following way:

  1. The device creates TCP socket connections to the Websense ThreatSeeker Cloud (TSC).

  2. The device intercepts an HTTP or an HTTPS connection and extracts each URL (in the HTTP request) or IP (in the HTTPS request). For an HTTPS connection, EWF is supported through SSL forward proxy.

    Note

    Starting with Junos OS Release 12.3X48-D25 and Junos OS Release 17.3R1, Enhanced Web Filtering (EWF) over SSL forward proxy supports HTTPS traffic.

  3. The device looks for the URL in the user-configured blacklist or whitelist.

    Note

    A blacklist or a whitelist action type is a user-defined category in which all the URLs or IP addresses are always blocked or permitted and optionally logged.

    • If the URL is in the user-configured blacklist, the device blocks the URL.

    • If the URL is in the user-configured whitelist, the device permits the URL.

  4. The device checks the user-defined categories and blocks or permits the URL based on the user-specified action for the category.

  5. The device looks for the URL in the URL filtering cache.

    • If the URL is not available in the URL filtering cache, the device sends the URL in HTTP format to the TSC with a request for categorization. The device uses one of the connections made available to the TSC to send the request.

    • The TSC responds to the device with the categorization and a reputation score.

  6. The device performs the following actions based on the identified category:

    • If the URL is permitted, the device forwards the HTTP request to the HTTP server.

    • If the URL is blocked, the device sends a deny page to the HTTP client and also sends a reset message to the HTTP server to close the connection

    • If the URL is quarantined, the device sends a redirect response to the HTTP client and the URL is redirected to the HTTP server.

    • If the category is configured and the category action is available, the device permits or blocks the URL based on the category action.

    • If the category is not configured, the device permits or blocks the URL based on the global reputation action.

    • If the global reputation is not configured, the device permits or blocks the URL based on the default action configured in the Web filtering profile.

Functional Requirements for Enhanced Web Filtering

The following items are required to use Enhanced Web Filtering (EWF):

  • License key—The EWF solution builds upon the SurfControl integrated feature on the device. Two different valid license keys are required for the SurfControl integrated solution and for EWF. You need to install a new license to upgrade to the EWF solution.

    Note

    You can ignore the warning message "requires 'wf_key_websense_ewf' license” because it is generated by routine EWF license validation check.

    A grace period of 30 days, consistent with other UTM features, is provided for the EWF feature after the license key expires.

    Note

    The device will continue to support the SurfControl integrated solution after the upgrade.

    When the grace period for the EWF feature has passed (or if the feature has not been installed), Web filtering is disabled, all HTTP requests bypass Web filtering, and any connections to the TSC are disabled. When you install a valid license, the connections to the server are established again.

  • The debug command provides the following information to each TCP connection available on the device:

    • Number of processed requests

    • Number of pending requests

    • Number of errors (dropped or timed-out requests)

  • TCP connection between a Web client and a webserver—An application identification (APPID) module is used to identify an HTTP connection. The EWF solution identifies an HTTP connection after the device receives the first SYN packet. If an HTTP request has to be blocked, EWF sends a block message from the device to the Web client. EWF further sends a TCP FIN request to the client and a TCP reset (RST) to the server to disable the connection. The device sends all the messages through the flow session. The messages follow the entire service chain.

  • HTTP request interception—EWF intercepts the first HTTP request on the device and performs URL filtering on all methods defined in HTTP 1.0 and HTTP 1.1. The device holds the original request while waiting for a response from the TSC. If the first packet in the HTTP URL is fragmented or if the device cannot extract the URL for some reason, then the destination IP address is used for the categorization.

    Note

    For HTTP 1.1 persistent connections, the subsequent requests on that session are ignored by the EWF module.

    If the device holds the original request for a long time, then the client will retransmit the request. The URL filtering code will detect the retransmitted packets. If the original HTTP request has already been forwarded, then EWF forwards the retransmitted packet to the server. However, if EWF is in the middle of first-packet processing or makes the calculation to block the session, then the solution drops the retransmitted packet. A counter tracks the number of retransmitted packets received by the device.

    If the TSC does not respond in time to the categorization request from the device, then the original client request is blocked or permitted according to the timeout fallback setting.

  • HTTPS request interceptionStarting with Junos OS 15.1X49-D40 and Junos OS Release 17.3R1, EWF intercepts HTTPS traffic passing through the SRX Series device. The security channel from the SRX Series device is divided as one SSL channel between the client and the SRX Series device and another SSL channel between the SRX Series device and the HTTPS server. SSL forward proxy acts as the terminal for both channels and forwards the cleartext traffic to the UTM. UTM extracts the URL from the HTTP request message.

  • Blocking message—The blocking message sent to the Web client is user-configurable and is of the following types:

    • The Juniper Networks blocking message is the default message defined in the device that can be modified by the user. The default blocking message contains the reason why the request is blocked and the category name (if it is blocked because of a category).

    • Syslog message.

    For example, if you have set the action for Enhanced_Search_Engines_and_Portals to block, and you try to access www.example.com, the blocking message is of the following form: Juniper Web Filtering:Juniper Web Filtering has been set to block this site. CATEGORY: Enhanced_Search_Engines_and_Portals REASON: BY_PRE_DEFINED . However, the corresponding syslog message on the device under test (DUT) is: WEBFILTER_URL_BLOCKED: WebFilter: ACTION="URL Blocked" 56.56.56.2(59418)->74.125.224.48(80) CATEGORY="Enhanced_Search_Engines_and_Portals" REASON="by predefined category" PROFILE="web-ewf" URL=www.example.com OBJ=/ .

  • Monitoring the Websense server—The URL filtering module uses two methods to determine if the TSC is active: socket connections and heartbeat. EWF maintains persistent TCP sockets to the TSC. The server responds with a TCP ACK if it is enabled. EWF sends an application layer NOOP keepalive to the TSC. If the device does not receive responses to three consecutive NOOP keepalives in a specific period, it determines the socket to be inactive. The EWF module attempts to open a new connection to the TSC. If all sockets are inactive, the TSC is considered to be inactive. Therefore an error occurs. The error is displayed and logged. Subsequent requests and pending requests are either blocked or passed according to the server connectivity fallback setting until new connections to the TSC are opened again.

  • HTTP protocol communication with the TSC—EWF uses the HTTP 1.1 protocol to communicate with the TSC. This ensures a persistent connection and transmission of multiple HTTP requests through the same connection. A single HTTP request or response is used for client or server communication. The TSC can handle queued requests; for optimal performance, an asynchronous request or response mechanism is used. The requests are sent over TCP, so TCP retransmission is used to ensure request or response delivery. TCP also ensures that valid in-order, non-retransmitted HTTP stream data is sent to the HTTP client on the device.

  • Responses—The responses adhere to the basic HTTP conventions. Successful responses include a 20x response code (typically 200). An error response includes a 4xx or 5xx code. Error responses in the 4xx series indicate issues in the custom code. Error responses in the 5xx series indicate issues with the service.

    Error codes and meanings are as follows:

    • 400–Bad request

    • 403–Forbidden

    • 404–Not found

    • 408–Request canceled or null response

    • 500–Internal server error

    Errors in the 400 series indicate issues with the request. Errors in the 500 series indicate issues with the TSC service. Websense is notified of these errors automatically and responds accordingly.

    You can configure the default fallback setting to determine whether to pass or block the request:

    set security utm feature-profile web-filtering juniper-enhanced profile juniper-enhanced fallback-settings default ?

    The response also contains the site categorization and site reputation information.

  • Categories—A category list is available on the device. This list consists of categories, each containing a category code, a name, and a parent ID. Categories can also be user-defined. Each category consists of a list of URLs or IP addresses. Categories are not updated dynamically and are tied to the Junos OS release because they have to be compiled into the Junos OS image. Any update in categories needs to be synchronized with the Junos OS release cycle.

    Starting with Junos OS Release 17.4R1, you can download and dynamically load new EWF categories. The downloading and dynamic loading of the new EWF categories do not require a software upgrade. Websense occasionally releases new EWF categories. EWF classifies websites into categories according to host, URL, or IP address and performs filtering based on the categories.

    Note

    On SRX Series devices, if the category file transfer fails between the primary and secondary devices, then the file transfer results in an upgrading error and an error log is generated.

    During new category file installation, if the category filename is changed, then the new category file overwrites the old category file in the internal system and all related output information is replaced with the new category name.

    Starting with Junos OS Release 17.4R1, predefined base filters, defined in a category file, are supported for individual EWF categories. Each EWF category has a default action in a base filter, which is attached to the user profile to act as a backup filter. If the categories are not configured in the user profile, then the base filter takes the action.

    A base filter is an object that contains a category-action pair for all categories defined in the category file. A base filter is a structured object, and is defined with the help of a filter name and an array of category-action pairs.

    The following is an example of a base filter with an array of category-action pairs. For the Enhanced_Adult_Material category, the action is block; for the Enhanced_Blog_Posting category, the action is permit; and so on.

    EWF supports up to 16 base filters. Junos OS Release 17.4R1 also supports online upgradation of base filters.

    Note

    If the user profile has the same name as the base filter, then the Web filter uses the wrong profile.

  • Caching—Successfully categorized responses are cached on the device. Uncategorized URLs are not cached. The size of the cache can be configured by the user.

  • Safe search (HTTP support only, not HTTPS)—A safe-search solution is used to ensure that the embedded objects, such as images on the URLs received from the search engines, are safe and that no undesirable content is returned to the client.

    A URL is provided to the TSC to provide categorization information. If it is a search URL, the TSC also returns a safe-search string. For instance, the safe-search string is safe=active. This safe-search string is appended to the URL, and a redirect response for redirecting the client's query with safe search is turned on. This ensures that no unsafe content is returned to the client. If the TSC indicates that it needs to be safe-searched, then you can perform the safe-search redirect.

    For example, the client makes a request to the URL http://images.example.com/images?hl=en&source=imghp&biw=1183&bih=626&q

    =adult+movies&gbv=2&aq=f&aqi=&aql=&oq=&gs_rfai= No category action is defined for this URL
    . TSC returns safe-search string safe=active. The EWF code on the DUT generates a HTTP 302 response, with the redirect URL: http://images.example.com/images?hl=en&source=imghp&biw=1183&bih=626&q=adult+

    movies&gbv=2&aq=f&aqi=&aql=&oq=&gs_rfai=&safe=active
    . This response is returned to the client. The client now sends out a safe redirect request to this URL.

    Note

    Safe-search redirect supports HTTP only. You cannot extract the URL for HTTPS. Therefore it is not possible to generate a redirect response for HTTPS search URLs. Safe-search redirects can be disabled by using the CLI option no-safe-search.

  • Site reputation—The TSC provides site reputation information. Based on these reputations, you can choose a block or a permit action. If the URL is not handled by a whitelist or a blacklist and does not fall in a user or predefined category, then the reputation can be used to perform a URL filtering decision.

    Starting with Junos OS Release 17.4R1, the reputation base scores are configurable. Users can apply global reputation values, provided by the Websense ThreatSeeker Cloud (TSC). For the non-category URLs, the global reputation value is used to perform filtering,

    The reputation scores are as follows:

    • 100-90–Site is considered very safe.

    • 80-89–Site is considered moderately safe.

    • 70-79–Site is considered fairly safe.

    • 60-69–Site is considered suspicious.

    • 0-59–Site is considered harmful.

    The device maintains a log for URLs that are blocked or permitted based on site reputation scores.

  • Profiles—A URL filtering profile is defined as a list of categories, with each profile having an action type (permit, log-and-permit, block, quarantine) associated with it. A predefined profile, junos-wf-enhanced-default, is provided to users if they choose not to define their own profile.

    You can also define an action based on site reputations in a profile to specify the action when the incoming URL does not belong to any of the categories defined in the profile. If you do not configure the site reputation handling information, then you can define a default action. All URLs that do not have a defined category or defined reputation action in their profile will be blocked, permitted, logged-and-permitted, or quarantined depending on the block or permit handling for the default action explicitly defined in the profile. If you do not specify a default action, then the URLs will be permitted. For search engine requests, if there is no explicit user-defined configuration, and the URL request is without the safe-search option, then EWF generates a redirect response and sends it to the client. The client will generate a new search request with the safe-search option enabled.

    Note

    A URL filtering profile can contain the following items:

    • Multiple user-defined and predefined categories, each with a permit or block action

    • Multiple site reputation handling categories, each with a permit or block action

    • One default action with a permit or block action

    The order of search is blacklist, whitelist, user-defined category, predefined category, safe-search, site reputation, and default action.

User Messages and Redirect URLs for Enhanced Web Filtering (EWF) on SRX Series Devices

Starting with Junos OS Release 15.1X49-D110, a new option, custom-message, is added for the custom-objects statement that enables you to configure user messages and redirect URLs to notify users when a URL is blocked or quarantined for each EWF category. The custom-message option has the following mandatory attributes:

  • Name: Name of the custom message; maximum length is 59 ASCII characters.

  • Type: Type of custom message: user-message or redirect-url.

  • Content: Content of the custom message; maximum length is 1024 ASCII characters.

You configure a user message or redirect URL as a custom object and assign the custom object to an EWF category.

  • User messages indicate that website access has been blocked by an organization's access policy. To configure a user message, include the type user-message content message-text statement at the [edit security utm custom-objects custom-message message] hierarchy level.

  • Redirect URLs redirect a blocked or quarantined URL to a user-defined URL. To configure a redirect URL, include the type redirect-url content redirect-url statement at the [edit security utm custom-objects custom-message message] hierarchy level.

The custom-message option provides the following benefits:

  • You can configure a separate custom message or redirect URL for each EWF category.

  • The custom-message option allows you to fine-tune messages to support your polices to know which URL is blocked or quarantined.

Note

Only one custom-message configuration option is applied for each category. The custom-message configuration is supported only on Enhanced Web Filtering (EWF). Therefore, only the Juniper EWF engine type is supported.

Starting with Junos OS Release 17.4R1, support for custom category configuration is available for local and Websense redirect profiles.

Predefined Category Upgrading and Base Filter Configuration Overview

You can download and dynamically load new Enhanced Web Filtering (EWF) categories without any software upgrade. The predefined base filters defined in a category file are supported for individual EWF categories.

To configure a predefined category upgrade without any software upgrade:

  1. Configure UTM custom objects for the UTM features. Set the interval, set the start time, and enter the URL of category package download:
  2. Configure the predefined base filters. Each EWF category has a default action in a base filter, which is attached to the user profile to act as a backup filter. If the categories are not configured in the user profile, then the base filter takes the action. You can also upgrade the base filters online.

show security utm custom-objects

show security utm feature-profile web-filtering juniper-enhanced

Example: Configuring Enhanced Web Filtering

This example shows how to configure Enhanced Web filtering (EWF) for managing website access. This feature is supported on all SRX Series devices. The EWF solution intercepts HTTP and the HTTPS requests and sends the HTTP URL or the HTTPS source IP to the Websense ThreatSeeker Cloud (TSC). The TSC categorizes the URL into one of the 151 or more predefined categories and also provides site reputation information. The TSC further returns the URL category and the site reputation information to the device. The SRX Series device determines whether it can permit or block the request based on the information provided by the TSC.

Requirements

This example uses the following hardware and software components:

  • SRX5600 device

  • Junos OS Release 12.1X46-D10 or later

Before you begin, you should be familiar with Web filtering and Enhanced Web filtering (EWF). See Web Filtering Overview and Understanding Enhanced Web Filtering Process.

Overview

Web filtering is used to monitor and control how users access the website over HTTP and HTTPS. In this example, you configure a URL pattern list (whitelist) of URLs or addresses that you want to bypass. After you create the URL pattern list, define the custom objects. After defining the custom objects, you apply them to feature profiles to define the activity on each profile, apply the feature profile to the UTM policy, and finally attach the Web filtering UTM policies to the security policies. Table 1 shows information about EWF configuration type, steps, and parameters used in this example.

Table 1: Enhanced Web filtering (EWF) Configuration Type, Steps, and Parameters

Configuration Type

Configuration Steps

Configuration Parameters

URL pattern and custom objects

Configure a URL pattern list (whitelist) of URLs or addresses that you want to bypass.

Create a custom object called urllist3 that contains the pattern http://www.example.net 1.2.3.4

  • [http://www.example.net 1.2.3.4]

  • value urllist3

  • http://www.untrusted.com

  • http://www.trusted.com

Add the urllist3 custom object to the custom URL category custurl3.

  • urllistblack

  • urllistwhite

Feature profiles

Configure the Web filtering feature profile:

 
  • Set the URL blacklist filtering category to custblacklist, set the whitelist filtering category to custwhitelist, and set the type of Web filtering engine to juniper-enhanced. Then you set the cache size and cache timeout parameters.

  • custwhitelist

  • custblacklist

  • type juniper-enhanced

  • cache size 500

  • cache timeout 1800

  • Name the EWF server and enter the port number for communicating with it. (Default port is 80.) Then you create an EWF profile name.

  • rp.cloud.threatseeker.com

  • port 80

  • http-profile my_ewfprofile01

  • Select a category from the included whitelist and blacklist categories or select a custom URL category list you created for filtering against.

  • http-reassemble

  • http-persist

  • Action: log-and-permit

  • site-reputation-action:

    • very-safe permit

  • Enter a custom message to be sent when HTTP requests are blocked. Finally, enter a timeout value in seconds.

  • ewf_my_profile-default block

  • custom-block-message "***access denied ***"

  • fallback-settings:

    • server-connectivity block

    • timeout block

    • too-many-requests block

  • quarantine-custom-message “**The requested webpage is blocked by your organization's access policy**”.

  • quarantine-message type custom-redirect-url

  • quarantine-message url besgas.spglab.example.net

  • ewf_my_profile-default:

    • timeout 10

    • no-safe-search

Configuration

This example shows how to configure custom URL patterns, custom objects, feature profiles, and security policies.

Configuring Enhanced Web Filtering Custom Objects and URL Patterns

CLI Quick Configuration

To quickly configure this section of the example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

Starting with Junos OS Release 15.1X49-D110, the “* “ in a wildcard syntax, required to create URL pattern for Web filtering profile, matches all subdomains. For example, *.example.net matches:

  • http://a.example.net

  • http://example.net

  • a.b.example.net

Warning

A custom category does not take precedence over a predefined category when it has the same name as one of the predefined categories. Do not use the same name for a custom category that you have used for a predefined category.

Step-by-Step Procedure

The following example requires you to navigate various levels in the configuration hierarchy. For instructions on how to do that, see Using the CLI Editor in Configuration Mode in the CLI User Guide.

To configure custom objects and URL patterns in Enhanced Web Filtering:

  1. Configure a URL pattern list (whitelist) of URLs or addresses that you want to bypass. After you create the URL pattern list, you create a custom URL category list and add the pattern list to it. Configure a URL pattern list custom object by creating the list name and adding values to it as follows:Note

    Because you use URL pattern lists to create custom URL category lists, you must configure URL pattern list custom objects before you configure custom URL category lists.

    Note

    The guideline to use a URL pattern wildcard is as follows: Use \*\.[]\?* and precede all wildcard URLs with http://. You can use “*” only if it is at the beginning of the URL and is followed by “.”. You can use “?” only at the end of the URL.

    The following wildcard syntaxes are supported: http://*.example.net, http://www.example.ne?, http://www.example.n??. The following wildcard syntaxes are not supported: *.example.???, http://*example.net, http://?.

  2. Create a custom object called urllist3 that contains the pattern http://www.example.net and then add the urllist3 custom object to the custom URL category custurl3.
  3. Create a list of untrusted and trusted sites.
  4. Configure the custom URL category list custom object by using the URL pattern list of untrusted and trusted sites.

Results

From configuration mode, confirm your configuration by entering the show security utm custom-objects command. If the output does not display the intended configuration, repeat the instructions in this example to correct.

If you are done configuring the device, enter commit from configuration mode.

Configuring Enhanced Web Filtering Feature Profiles

CLI Quick Configuration

To quickly configure this section of the example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

Note

Starting in Junos OS Release 12.3X48-D25, new CLI options are available. The http-reassemble and http-persist options are added in the show security utm feature-profile web-filtering command.

Step-by-Step Procedure

The following example requires you to navigate various levels in the configuration hierarchy. For instructions on how to do that, see Using the CLI Editor in Configuration Mode in the CLI User Guide.

To configure the EWF feature profiles:

  1. Configure the Web filtering URL blacklist, URL whitelist, and the Web filtering engine.
  2. Set the cache size and cache timeout parameters for the configured EWF engine.
  3. Set the server name or IP address and the port number for communicating with the server. The default host value in the system is rp.cloud.threatseeker.com.
  4. Set the http-reassemble statement to reassemble the requested packet and the http-persist statement to check every HTTP request packet in the same session. If the http-reassemble statement is not configured for cleartext HTTP traffic, then EWF does not reassemble the fragmented HTTP request to avoid incomplete parsing in the packet-based inspection. If the http-persist statement is not configured for cleartext HTTP traffic, then EWF does not check every HTTP request packet in the same session.
  5. Create a profile name, and select a category from the included whitelist and blacklist categories.
  6. Specify the action to be taken depending on the site reputation returned for the URL if there is no category match found.
  7. Enter a custom message to be sent when HTTP requests are blocked.

  8. Define a redirect URL server so that instead of the device sending a block page with plain text HTML, the device will send an HTTP 302 redirect to this redirect server with some special variables embedded in the HTTP redirect location field. These special variables can be parsed by the redirect server and serve a special block page to the client with rich images and formatting.
    Note

    If you configure the security utm feature-profile web-filtering juniper-enhanced profile ewf_my_profile block-message statement, then the default block message configuration takes precedence over the security utm feature-profile web-filtering juniper-enhanced profile ewf_my_profile custom-block-message configuration.

  9. Specify a default action (permit, log and permit, block, or quarantine) for the profile, when no other explicitly configured action (blacklist, whitelist, custom category, predefined category actions, or site reputation actions) is matched .
  10. Configure the fallback settings (block or log and permit) for this profile.
  11. Enter a timeout value in seconds. When this limit is reached, fallback settings are applied. This example sets the timeout value to 10. You can also disable the safe-search functionality. By default, search requests have safe-search strings attached to them, and a redirect response is sent to ensure that all search requests are safe or strict.
    Note

    The timeout value range for SRX210, SRX220, SRX240, SRX300, SRX320, SRX345, SRX550, SRX1500, SRX4100, and SRX4200 is 0 through 1800 seconds and the default value is 15 seconds. The timeout value range for SRX3400 and SRX3600 is 1 through 120 seconds and the default value is 3 seconds.

  12. Configure a UTM policy (mypolicy) for the Web-filtering HTTP protocol, associating ewf_my_profile to the UTM policy, and attach this policy to a security profile to implement it.

Results

From configuration mode, confirm your configuration by entering the show security utm feature-profile command. If the output does not display the intended configuration, repeat the instructions in this example to correct.

If you are done configuring the device, enter commit from configuration mode.

Attaching Web Filtering UTM Policies to Security Policies

CLI Quick Configuration

To quickly configure this section of the example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

Step-by-Step Procedure

The following example requires you to navigate various levels in the configuration hierarchy. For instructions on how to do that, see Using the CLI Editor in Configuration Mode in the CLI User Guide.

To attach a UTM policy to a security policy:

  1. Create the security policy sec_policy.
  2. Specify the match conditions for sec-policy.
  3. Attach the UTM policy mypolicy to the security policy sec_policy.

Results

From configuration mode, confirm your configuration by entering the show security policies command. If the output does not display the intended configuration, repeat the configuration instructions in this example to correct it.

After you are done configuring the device, enter commit from configuration mode.

Verification

To confirm that the configuration is working properly, perform these tasks:

Verifying the Status of the Web Filtering Server

Purpose

Verify the Web filtering server status.

Action

From the top of the configuration in operational mode, enter the show security utm web-filtering status command.

user@host> show security utm web-filtering status

Meaning

The command output shows that the Web filtering server connection is up.

Verifying that Web Filtering Statistics Have Increased

Purpose

Verify the increase in Web filtering statistics. The initial counter value is 0; if there is an HTTP request URL hit, then there is a increase in the Web filtering statistics.

Action

From the top of the configuration in operational mode, enter the show security utm web-filtering statistics command.

user@host> show security utm web-filtering statistics

Meaning

The output displays Web filtering statistics for connections including whitelist and blacklist hits and custom category hits. If there is an HTTP request URL hit, then there is a increase in the Web filtering statistics from an earlier value.

Verifying That the Web Filtering UTM Policy Is Attached to the Security Policy

Purpose

Verify that the Web filtering UTM policy mypolicy is attached to the security policy sec_policy.

Action

From operational mode, enter the show security policy command.

user@host> show security policies global policy-name mypolicy detail

Meaning

The output displays a summary of all security policies configured on the device. If a particular policy is specified, it displays information specific to that policy. If UTM is enabled, then mypolicy is attached to sec_policy.

Understanding the Quarantine Action for Enhanced Web Filtering

UTM Enhanced Web Filtering supports block, log-and-permit, and permit actions for HTTP/HTTPS requests. In addition to this, UTM Enhanced Web Filtering now supports the quarantine action which allows or denies access to the blocked site based on the user’s response to the message.

The following sequence explains how the HTTP or HTTPs request is intercepted, redirected, and acted upon by the quarantine action:

  • The HTTP client requests URL access.

  • The device intercepts the HTTP request and sends the extracted URL to the Websense Thread Seeker Cloud (TSC).

  • The TSC returns the URL category and the site reputation information to the device.

  • If the action configured for the category is quarantine, the device logs the quarantine action and sends a redirect response to HTTP client.

  • The URL is sent to the HTTP server for redirecting.

  • The device shows a warning message stating that the access to the URL is blocked according to the organization’s security policies and prompts the user to respond.

  • If the user response is “No,” the session is terminated. If the user response is “Yes,” the user is allowed access to the site and such access is logged and reported to the administrator.

Note

On all SRX Series devices, the quarantine action is supported only for UTM Enhanced Web Filtering or Juniper enhanced type of Web filtering.

Quarantine Message

The quarantine message sent to the HTTP client is user-configurable and is of the following types:

  • Default message

    The default quarantine message is displayed when a user attempts to access a quarantined website and it contains the following information:

    • URL name

    • Quarantine reason

    • Category (if available)

    • Site-reputation (if available)

    For example, if you have set the action for Enhanced_Search_Engines_and_Portals to quarantine, and you try to access www.search.example.com, the quarantine message is as follows:

    ***The requested webpage is blocked by your organization's access policy***.

  • Syslog message.

    The syslog message will be logged by the system when the user access the web page that has already been quarantined and marked as block or permit.

    The corresponding syslog message on the device under test is:

    Jan 25 15:10:40 rodian utmd[3871]: WEBFILTER_URL_BLOCKED: WebFilter: ACTION="URL Blocked" 99.99.99.4(60525)->74.125.224.114(80) CATEGORY="Enhanced_Search_Engines_and_Portals" REASON="by predefined category(quarantine)" PROFILE="ewf-test-profile" URL=www.search.example.com OBJ=/

    Starting in Junos OS 12.1X47-D40 and Junos OS Release 17.3R1, the structured log fields have changed. The structured log field changes in the UTM Web filter logs WEBFILTER_URL_BLOCKED, WEBFILTER_URL_REDIRECTED, and WEBFILTER_URL_PERMITTED are as follows:

    • name -> category

    • error-message -> reason

    • profile-name -> profile

    • object-name -> url

    • pathname -> obj

User Messages and Redirect URLs for Enhanced Web Filtering (EWF) on SRX Series devices

Starting with Junos OS Release 15.1X49-D110, a new option, custom-message, is added for the custom-objects statement that enables you to configure user messages and redirect URLs to notify users when a URL is blocked or quarantined for each EWF category. The custom-message option has the following mandatory attributes:

  • Name: Name of the custom message; maximum length is 59 ASCII characters.

  • Type: Type of custom message: user-message or redirect-url.

  • Content: Content of the custom message; maximum length is 1024 ASCII characters.

You configure a user message or redirect URL as a custom object and assign the custom object to an EWF category.

  • User messages indicate that website access has been blocked by an organization's access policy. To configure a user message, include the type user-message content message-text statement at the [edit security utm custom-objects custom-message message] hierarchy level.

  • Redirect URLs redirect a blocked or quarantined URL to a user-defined URL. To configure a redirect URL, include the type redirect-url content redirect-url statement at the [edit security utm custom-objects custom-message message] hierarchy level.

The custom-message option provides the following benefits:

  • You can configure a separate custom message or redirect URL for each EWF category.

  • The custom-message option enables you to fine-tune messages to support your polices to know which URL is blocked or quarantined.

Note

Only one custom-message configuration option is applied for each category. The custom-message configuration is supported only on Enhanced Web Filtering (EWF). Therefore, only the Juniper EWF engine type is supported.

Starting with Junos OS Release 17.4R1, support for custom category configuration is available for local and Websense redirect profiles.

Example: Configuring Site Reputation Action for Enhanced Web Filtering

This example shows how to configure the site reputation action for both categorized and uncategorized URLs.

Requirements

Before you begin, you should be familiar with Web Filtering and Enhanced Web Filtering. See Web Filtering Overview and Understanding Enhanced Web Filtering Process.

Overview

In this example, you configure Web Filtering profiles to URLs according to defined categories using the site reputation action. You set the URL whitelist filtering category to url-cat-white and the type of Web Filtering engine to juniper-enhanced. Then you set the cache size parameters for Web Filtering and the cache timeout parameters to 1.

Then you create a juniper-enhanced profile called profile ewf-test-profile, set the URL whitelist category to cust-cat-quarantine, and set the reputation action to quarantine.

You enter a custom message to be sent when HTTP requests are quarantined. In this example, the following message is sent: The requested webpage is blocked by your organization's access policy.

You block URLs in the Enhanced_News_and_Media category and permit URLs in the Enhanced_Education category. Then you quarantine the URLs in the Enhanced_Streaming_Media category and configure the device to send the following message: The requested webpage is blocked by your organization's access policy.

In this example, you set the default action to permit. You select fallback settings (block or log-and-permit) for this profile in case errors occur in each configured category. Finally, you set the fallback settings to block.

Configuration

Configuring Site Reputation Action

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

Step-by-Step Procedure

The following example requires you to navigate various levels in the configuration hierarchy. For instructions on how to do that, see Using the CLI Editor in Configuration Mode in the CLI User Guide.

To configure the site reputation action:

  1. Configure the Web Filtering URL whitelist.
  2. Specify the Enhanced Web Filtering engine, and set the cache size parameters.
  3. Configure the base reputation scores.
    Note

    The base reputation value must be ordered.

  4. Set the cache timeout parameters.
  5. Create a profile name, and select a category from the whitelist categories.
  6. Create a profile name, and select a category from the whitelist categories.
  7. Enter a warning message to be sent when HTTP requests are quarantined.
  8. Select a default action (permit, log-and-permit, block, or quarantine) for the profile, when no other explicitly configured action (blacklist, whitelist, custom category, predefined category or site reputation ) is matched .
  9. Select fallback settings (block or log-and-permit) for this profile.

Results

From configuration mode, confirm your configuration by entering the show security utm command. If the output does not display the intended configuration, repeat the instructions in this example to correct.

If you are done configuring the device, enter commit from configuration mode.

Verification

Confirm that the configuration is working properly.

Verifying the Status of UTM Service

Purpose

Verify the UTM service status.

Action

From operational mode, enter the show security utm status command.

Sample Output

user@host>show security utm status

Verifying the Status of UTM Session

Purpose

Verify the UTM session status.

Action

From operational mode, enter the show security utm session command.

Sample Output

user@host>show security utm session

Verifying the Status of UTM Web Filtering

Purpose

Verify the UTM Web filtering status.

Action

From operational mode, enter the show security utm web-filtering status command.

Sample Output

user@host>show security utm web-filtering status

Verifying the Statistics of UTM Web Filtering

Purpose

Verify the Web filtering statistics for connections including whitelist and blacklist hits and custom category hits.

Action

From operational mode, enter the show security utm web-filtering statistics command.

Sample Output

user@host>show security utm web-filtering statistics
Release History Table
Release
Description
Starting with Junos OS Release 17.4R1, support for custom category configuration is available for local and Websense redirect profiles.
Starting with Junos OS Release 17.4R1, you can download and dynamically load new EWF categories. The downloading and dynamic loading of the new EWF categories do not require a software upgrade. Websense occasionally releases new EWF categories. EWF classifies websites into categories according to host, URL, or IP address and performs filtering based on the categories.
Starting with Junos OS Release 17.4R1, predefined base filters, defined in a category file, are supported for individual EWF categories. Each EWF category has a default action in a base filter, which is attached to the user profile to act as a backup filter. If the categories are not configured in the user profile, then the base filter takes the action.
Starting with Junos OS Release 17.4R1, the reputation base scores are configurable. Users can apply global reputation values, provided by the Websense ThreatSeeker Cloud (TSC). For the non-category URLs, the global reputation value is used to perform filtering,
Starting with Junos OS Release 17.4R1, support for custom category configuration is available for local and Websense redirect profiles.
Starting with Junos OS Release 17.4R1, support for custom category configuration is available for local and Websense redirect profiles.
Starting in Junos OS Release 15.1X49-D40 and Junos OS Release 17.3R1, EWF supports HTTPS traffic by intercepting HTTPS traffic passing through the SRX Series device.
Starting with Junos OS 15.1X49-D40 and Junos OS Release 17.3R1, EWF intercepts HTTPS traffic passing through the SRX Series device. The security channel from the SRX Series device is divided as one SSL channel between the client and the SRX Series device and another SSL channel between the SRX Series device and the HTTPS server. SSL forward proxy acts as the terminal for both channels and forwards the cleartext traffic to the UTM. UTM extracts the URL from the HTTP request message.
Starting with Junos OS Release 15.1X49-D110, a new option, custom-message, is added for the custom-objects command that enables you to configure user messages and redirect URLs to notify users when a URL is blocked or quarantined for each EWF category.
Starting with Junos OS Release 15.1X49-D110, a new option, custom-message, is added for the custom-objects statement that enables you to configure user messages and redirect URLs to notify users when a URL is blocked or quarantined for each EWF category.
Starting with Junos OS Release 15.1X49-D110, the “* “ in a wildcard syntax, required to create URL pattern for Web filtering profile, matches all subdomains.
Starting with Junos OS Release 15.1X49-D110, a new option, custom-message, is added for the custom-objects statement that enables you to configure user messages and redirect URLs to notify users when a URL is blocked or quarantined for each EWF category.
The Surf-Contol feature is not supported from Junos OS Release 15.1X49-D10 and Junos OS Release 17.3R1 onwards.
Starting with Junos OS Release 12.3X48-D25 and Junos OS Release 17.3R1, Enhanced Web Filtering (EWF) over SSL forward proxy supports HTTPS traffic.
Starting in Junos OS 12.1X47-D40 and Junos OS Release 17.3R1, the structured log fields have changed.