Enhanced Web Filtering
Web Filtering provides URL filtering capability by using either a local Websense server or Internet-based SurfControl server. For more information, see the following topics:
Enhanced Web Filtering Overview
Enhanced Web Filtering (EWF) with Websense is an integrated URL filtering solution. When you enable the solution on the device, it intercepts the HTTP and the HTTPS requests and sends the HTTP URL or the HTTPS source IP to the Websense ThreatSeeker Cloud (TSC). The TSC categorizes the URL into one of the 95 or more categories that are predefined and also provides site reputation information. The TSC further returns the URL category and the site reputation information to the device. The device determines if it can permit or block the request based on the information provided by the TSC.
Starting in Junos OS Release 15.1X49-D40 and Junos OS Release 17.3R1, EWF supports HTTPS traffic by intercepting HTTPS traffic passing through the SRX Series device. The security channel from the device is divided as one SSL channel between the client and the device and another SSL channel between the device and the HTTPS server. SSL forward proxy acts as the terminal for both channels and forwards the cleartext traffic to the UTM. UTM extracts the URL from the HTTP request message.
You can consider the EWF solution as the next-generation URL filtering solution, building upon the existing Surf-Control solution.
Enhanced Web Filtering supports the following HTTP methods:
GET
POST
OPTIONS
HEAD
PUT
DELETE
TRACE
CONNECT
User Messages and Redirect URLs for Enhanced Web Filtering (EWF)
Starting with Junos OS Release 15.1X49-D110, a new option, custom-message, is added for the custom-objects command that enables you to configure user messages and redirect URLs to notify users when a URL is blocked or quarantined for each EWF category. The custom-message option has the following mandatory attributes:
Name: Name of the custom message; maximum length is 59 bytes.
Type: Type of custom message: user-message or redirect-url.
Content: Content of the custom message; maximum length is 1024 bytes.
You configure a user message or redirect URL as a custom object and assign the custom object to an EWF category.
User messages indicate that website access has been blocked by an organization's access policy. To configure a user message, include the type user-message content message-text statement at the [edit security utm custom-objects custom-message message] hierarchy level.
Redirect URLs redirect a blocked or quarantined URL to a user-defined URL. To configure a redirect URL, include the type redirect-url content redirect-url statement at the [edit security utm custom-objects custom-message message] hierarchy level.
The custom-message option provides the following benefits:
You can configure a separate custom message or redirect URL for each EWF category.
The custom-message option allows you to fine-tune messages to support your polices to know which URL is blocked or quarantined.
Only one custom-message configuration option is applied for each category. The custom-message configuration is supported only on Enhanced Web Filtering (EWF). Therefore, only the Juniper EWF engine type is supported.
Starting with Junos OS Release 17.4R1, support for custom category configuration is available for local and Websense redirect profiles.
See also
Understanding the Enhanced Web Filtering Process
Web filtering enables you to manage Internet access, preventing access to inappropriate Web content. The Enhanced Web Filtering (EWF) feature intercepts, scans, and acts upon HTTP or HTTPS traffic in the following way:
The device creates TCP socket connections to the Websense ThreatSeeker Cloud (TSC).
The device intercepts an HTTP or an HTTPS connection and extracts URL or hostname or IP address to perform Web filtering. For an HTTPS connection, EWF is supported through SSL forward proxy.
Starting with Junos OS Release 12.3X48-D25 and Junos OS Release 17.3R1, Enhanced Web Filtering (EWF) over SSL forward proxy supports HTTPS traffic.
The device looks for the URL in the user-configured blocklist or allowlist.
A blocklist or a allowlist action type is a user-defined category in which all the URLs or IP addresses are always blocked or permitted and optionally logged.
If the URL is in the user-configured blocklist, the device blocks the URL.
If the URL is in the user-configured allowlist, the device permits the URL.
The device checks the user-defined categories and blocks or permits the URL based on the user-specified action for the category.
The device looks for predefiend category in local cache or from cloud service.
If the URL is not available in the URL filtering cache, the device sends the URL in HTTP format to the TSC with a request for categorization. The device uses one of the connections made available to the TSC to send the request.
The TSC responds to the device with the categorization and a reputation score.
The device performs the following actions based on the identified category:
If the URL is permitted, the device forwards the HTTP request to the HTTP server.
If the URL is blocked, the device sends a deny page to the HTTP client and also sends a reset message to the HTTP server to close the connection
If the URL is quarantined, the device sends a quarantine page with set-cookie to the HTTP client. If the client decided to continue, the device permits new request with cookie.
If the category is configured and the category action is available, the device permits or blocks the URL based on the category action.
If the category is not configured, the device permits or blocks the URL based on the global reputation action.
If the global reputation is not configured, the device permits or blocks the URL based on the default action configured in the Web filtering profile.
By default, the EWF processes a URL in the order of blocklist, allowlist, custom category, and then predefined category.
Functional Requirements for Enhanced Web Filtering
The following items are required to use Enhanced Web Filtering (EWF):
License key— You need to install a new license to upgrade to the EWF solution.
You can ignore the warning message "requires 'wf_key_websense_ewf' license” because it is generated by routine EWF license validation check.
A grace period of 30 days, consistent with other UTM features, is provided for the EWF feature after the license key expires.
This feature requires a license. To understand more about UTM Licensing, see, Understanding UTM Licensing. Please refer to the Licensing Guide for general information about License Management. Please refer to the product Data Sheets at SRX Series Services Gateways for details, or contact your Juniper Account Team or Juniper Partner.
When the grace period for the EWF feature has passed (or if the feature has not been installed), Web filtering is disabled, all HTTP requests bypass Web filtering, and any connections to the TSC are disabled. When you install a valid license, the connections to the server are established again.
The debug command provides the following information to each TCP connection available on the device:
Number of processed requests
Number of pending requests
Number of errors (dropped or timed-out requests)
TCP connection between a Web client and a webserver—An application identification (APPID) module is used to identify an HTTP connection. The EWF solution identifies an HTTP connection after the device receives the first SYN packet. If an HTTP request has to be blocked, EWF sends a block message from the device to the Web client. EWF further sends a TCP FIN request to the client and a TCP reset (RST) to the server to disable the connection. The device sends all the messages through the flow session. The messages follow the entire service chain.
HTTP request interception—EWF intercepts the first HTTP request on the device and performs URL filtering on all methods defined in HTTP 1.0 and HTTP 1.1. The device holds the original request while waiting for a response from the TSC. If the first packet in the HTTP URL is fragmented or if the device cannot extract the URL for some reason, then the destination IP address is used for the categorization. If you turn on http-reassemble, EWF can recover the whole request from fragment and get URL.
For HTTP 1.1 persistent connections, the subsequent requests on that session are ignored by the EWF module.
If the device holds the original request for a long time, then the client will retransmit the request. The URL filtering code will detect the retransmitted packets. If the original HTTP request has already been forwarded, then EWF forwards the retransmitted packet to the server. However, if EWF is in the middle of first-packet processing or makes the calculation to block the session, then the solution drops the retransmitted packet. A counter tracks the number of retransmitted packets received by the device.
If the TSC does not respond in time to the categorization request from the device, then the original client request is blocked or permitted according to the timeout fallback setting.
HTTPS request interception—Starting with Junos OS 15.1X49-D40 and Junos OS Release 17.3R1, EWF intercepts HTTPS traffic passing through the SRX Series device. The security channel from the device is divided as one SSL channel between the client and the device and another SSL channel between the device and the HTTPS server. SSL forward proxy acts as the terminal for both channels and forwards the cleartext traffic to the UTM. UTM extracts the URL from the HTTP request message.
Blocking message—The blocking message sent to the Web client is user-configurable and is of the following types:
The Juniper Networks blocking message is the default message defined in the device that can be modified by the user. The default blocking message contains the reason why the request is blocked and the category name (if it is blocked because of a category).
Syslog message.
For example, if you have set the action for Enhanced_Search_Engines_and_Portals to block, and you try to access www.example.com, the blocking message is of the following form: Juniper Web Filtering:Juniper Web Filtering has been set to block this site. CATEGORY: Enhanced_Search_Engines_and_Portals REASON: BY_PRE_DEFINED . However, the corresponding syslog message on the device under test (DUT) is: WEBFILTER_URL_BLOCKED: WebFilter: ACTION="URL Blocked" 56.56.56.2(59418)->74.125.224.48(80) CATEGORY="Enhanced_Search_Engines_and_Portals" REASON="by predefined category" PROFILE="web-ewf" URL=www.example.com OBJ=/ .
Monitoring the Websense server—The URL filtering module uses two methods to determine if the TSC is active: socket connections and heartbeat. EWF maintains persistent TCP sockets to the TSC. The server responds with a TCP ACK if it is enabled. EWF sends an application layer NOOP keepalive to the TSC. If the device does not receive responses to three consecutive NOOP keepalives in a specific period, it determines the socket to be inactive. The EWF module attempts to open a new connection to the TSC. If all sockets are inactive, the TSC is considered to be inactive. Therefore an error occurs. The error is displayed and logged. Subsequent requests and pending requests are either blocked or passed according to the server connectivity fallback setting until new connections to the TSC are opened again.
HTTP protocol communication with the TSC—EWF uses the HTTP 1.1 protocol to communicate with the TSC. This ensures a persistent connection and transmission of multiple HTTP requests through the same connection. A single HTTP request or response is used for client or server communication. The TSC can handle queued requests; for optimal performance, an asynchronous request or response mechanism is used. The requests are sent over TCP, so TCP retransmission is used to ensure request or response delivery. TCP also ensures that valid in-order, non-retransmitted HTTP stream data is sent to the HTTP client on the device.
Responses—The responses adhere to the basic HTTP conventions. Successful responses include a 20x response code (typically 200). An error response includes a 4xx or 5xx code. Error responses in the 4xx series indicate issues in the custom code. Error responses in the 5xx series indicate issues with the service.
Error codes and meanings are as follows:
400–Bad request
403–Forbidden
404–Not found
408–Request canceled or null response
500–Internal server error
Errors in the 400 series indicate issues with the request. Errors in the 500 series indicate issues with the TSC service. Websense is notified of these errors automatically and responds accordingly.
You can configure the default fallback setting to determine whether to pass or block the request:
set security utm feature-profile web-filtering juniper-enhanced profile juniper-enhanced fallback-settings default ?The response also contains the site categorization and site reputation information.
Categories—A category list is available on the device. This list consists of categories, each containing a category code, a name, and a parent ID. Categories can also be user-defined. Each category consists of a list of URLs or IP addresses. Categories are not updated dynamically and are tied to the Junos OS release because they have to be compiled into the Junos OS image. Any update in categories needs to be synchronized with the Junos OS release cycle.
Starting with Junos OS Release 17.4R1, you can download and dynamically load new EWF categories. The downloading and dynamic loading of the new EWF categories do not require a software upgrade. Websense occasionally releases new EWF categories. EWF classifies websites into categories according to host, URL, or IP address and performs filtering based on the categories.
If the category file transfer fails between the primary and secondary devices, then the file transfer results in an upgrading error and an error log is generated.
During new category file installation, if the category filename is changed, then the new category file overwrites the old category file in the internal system and all related output information is replaced with the new category name.
Starting with Junos OS Release 17.4R1, predefined base filters, defined in a category file, are supported for individual EWF categories. Each EWF category has a default action in a base filter, which is attached to the user profile to act as a backup filter. If the categories are not configured in the user profile, then the base filter takes the action.
A base filter is an object that contains a category-action pair for all categories defined in the category file. A base filter is a structured object, and is defined with the help of a filter name and an array of category-action pairs.
The following is an example of a base filter with an array of category-action pairs. For the Enhanced_Adult_Material category, the action is block; for the Enhanced_Blog_Posting category, the action is permit; and so on.
{ "predefined-filter": [ { "filter-name": "ewf-default-filter", "cat-action-table": [ {"name":"Enhanced_Adult_Material","action":"block"}, {"name":"Enhanced_Blog_Posting","action":"permit"}, {"name":"Enhanced_Blog_Commenting","action":"permit"} ] } ] }EWF supports up to 16 base filters. Junos OS Release 17.4R1 also supports online upgradation of base filters.
If the user profile has the same name as the base filter, then the Web filter uses the wrong profile.
Caching—Successfully categorized responses are cached on the device. Uncategorized URLs are not cached. The size of the cache can be configured by the user.
Safe search (HTTP support only, not HTTPS)—A safe-search solution is used to ensure that the embedded objects, such as images on the URLs received from the search engines, are safe and that no undesirable content is returned to the client.
A URL is provided to the TSC to provide categorization information. If it is a search URL, the TSC also returns a safe-search string. For instance, the safe-search string is safe=active. This safe-search string is appended to the URL, and a redirect response for redirecting the client's query with safe search is turned on. This ensures that no unsafe content is returned to the client. If the TSC indicates that it needs to be safe-searched, then you can perform the safe-search redirect.
For example, the client makes a request to the URL https://www.google.com/search?q=test, which is permitted by EWF profile. On packet mode, the EWF on the DUT will generate a HTTP 302 response, with the redirect URL: https://www.google.com/search?q=test&safe=active. This response returns to the client. The client now sends out a safe redirect request to this URL. On stream mode, the EWF on the DUT rewrites the URL to https://www.google.com/search?q=test&safe=active and forwards it.
Note Safe-search redirect supports HTTP only. You cannot extract the URL for HTTPS. Therefore it is not possible to generate a redirect response for HTTPS search URLs. Safe-search redirects can be disabled by using the CLI option no-safe-search.
Site reputation—The TSC provides site reputation information. Based on these reputations, you can choose a block or a permit action. If the URL is not handled by a allowlist or a blocklist and does not fall in a user or predefined category, then the reputation can be used to perform a URL filtering decision.
Starting with Junos OS Release 17.4R1, the reputation base scores are configurable. Users can apply global reputation values, provided by the Websense ThreatSeeker Cloud (TSC). For the non-category URLs, the global reputation value is used to perform filtering,
The reputation scores are as follows:
100-90–Site is considered very safe.
80-89–Site is considered moderately safe.
70-79–Site is considered fairly safe.
60-69–Site is considered suspicious.
0-59–Site is considered harmful.
The device maintains a log for URLs that are blocked or permitted based on site reputation scores.
Profiles—A URL filtering profile is defined as a list of categories, with each profile having an action type (permit, log-and-permit, block, quarantine) associated with it. A predefined profile, junos-wf-enhanced-default, is provided to users if they choose not to define their own profile.
You can also define an action based on site reputations in a profile to specify the action when the incoming URL does not belong to any of the categories defined in the profile. If you do not configure the site reputation handling information, then you can define a default action. All URLs that do not have a defined category or defined reputation action in their profile will be blocked, permitted, logged-and-permitted, or quarantined depending on the block or permit handling for the default action explicitly defined in the profile. If you do not specify a default action, then the URLs will be permitted. For search engine requests, if there is no explicit user-defined configuration, and the URL request is without the safe-search option, then EWF generates a redirect response and sends it to the client. The client will generate a new search request with the safe-search option enabled.
A URL filtering profile can contain the following items:
Multiple user-defined and predefined categories, each with a permit or block action
Multiple site reputation handling categories, each with a permit or block action
One default action with a permit or block action
The order of search is blocklist, allowlist, user-defined category, predefined category, safe-search, site reputation, and default action.
User Messages and Redirect URLs for Enhanced Web Filtering (EWF)
Starting with Junos OS Release 15.1X49-D110, a new option, custom-message, is added for the custom-objects statement that enables you to configure user messages and redirect URLs to notify users when a URL is blocked or quarantined for each EWF category. The custom-message option has the following mandatory attributes:
Name: Name of the custom message; maximum length is 59 ASCII characters.
Type: Type of custom message: user-message or redirect-url.
Content: Content of the custom message; maximum length is 1024 ASCII characters.
You configure a user message or redirect URL as a custom object and assign the custom object to an EWF category.
User messages indicate that website access has been blocked by an organization's access policy. To configure a user message, include the type user-message content message-text statement at the [edit security utm custom-objects custom-message message] hierarchy level.
Redirect URLs redirect a blocked or quarantined URL to a user-defined URL. To configure a redirect URL, include the type redirect-url content redirect-url statement at the [edit security utm custom-objects custom-message message] hierarchy level.
The custom-message option provides the following benefits:
You can configure a separate custom message or redirect URL for each EWF category.
The custom-message option allows you to fine-tune messages to support your polices to know which URL is blocked or quarantined.
Only one custom-message configuration option is applied for each category. The custom-message configuration is supported only on Enhanced Web Filtering (EWF). Therefore, only the Juniper EWF engine type is supported.
Starting with Junos OS Release 17.4R1, support for custom category configuration is available for local and Websense redirect profiles.
See also
Predefined Category Upgrading and Base Filter Configuration Overview
You can download and dynamically load new Enhanced Web Filtering (EWF) categories without any software upgrade. The predefined base filters defined in a category file are supported for individual EWF categories.
To configure a predefined category upgrade without any software upgrade:
- Configure UTM custom objects for the UTM features. Set
the interval, set the start time, and enter the URL of category package
download:user@host# set security utm custom-objectsuser@host# set security utm custom-objects category-packageuser@host# set security utm custom-objects category-package automaticuser@host# set security utm custom-objects category-package automatic interval 60user@host# set security utm custom-objects category-package automatic interval 60 enableuser@host# set security utm custom-objects category-package automatic interval 60 enable start-time 2017-09-05.08.08.08user@host# set security utm custom-objects category-package automatic route-instance VRFuser@host# set security utm custom-objects category-package automatic route-instance VRF url https://update.juniper-updates.net/EWF
- Configure the predefined base filters. Each EWF category
has a default action in a base filter, which is attached to the user
profile to act as a backup filter. If the categories are not configured
in the user profile, then the base filter takes the action. You can
also upgrade the base filters online.user@host# set security utm feature-profile web-filtering juniper-enhanced juniper-enhanced-profileuser@host# set security utm feature-profile web-filtering juniper-enhanced juniper-enhanced-profile base-filter [base-filter]user@host# set security utm feature-profile web-filtering juniper-enhanced juniper-enhanced-profile base-filter [base-filter] category <category-action >user@host# set security utm feature-profile web-filtering juniper-enhanced juniper-enhanced-profile base-filter [base-filter] category category-action default <default-action>user@host# set security utm feature-profile web-filtering juniper-enhanced juniper-enhanced-profile base-filter [base-filter] category category-action default <default-action>site-reputation-action <reputation-action>
show security utm custom-objects
category-package{
automatic{
interval 60;
enable;
start-time "2017-09-05.08.08.08";
}
route-instance VRF;
url https://update.juniper-updates.net/EWF;
}
show security utm feature-profile web-filtering juniper-enhanced
server {
host rp.cloud.threatseeker.com;
}
sockets 8;
profile ewf_p1 {
+ base-filter gov-filter;
default log-and-permit;
timeout 15;
}
+reputation {
reputation-very-safe 90;
reputation-moderately-safe 80;
reputation-fairly-safe 70;
reputation-suspicious 60;
}
See also
Example: Configuring Enhanced Web Filtering
This example shows how to configure Enhanced Web filtering (EWF) for managing website access. This feature is supported on all SRX Series devices. The EWF solution intercepts HTTP and the HTTPS requests and sends the HTTP URL or the HTTPS source IP to the Websense ThreatSeeker Cloud (TSC). The TSC categorizes the URL into one of the 151 or more predefined categories and also provides site reputation information. The TSC further returns the URL category and the site reputation information to the device. The SRX Series device determines whether it can permit or block the request based on the information provided by the TSC.
Requirements
This example uses the following hardware and software components:
SRX5600 device
Junos OS Release 12.1X46-D10 or later
Before you begin, you should be familiar with Web filtering and Enhanced Web filtering (EWF). See Web Filtering Overview and Understanding Enhanced Web Filtering Process.
Overview
Web filtering is used to monitor and control how users access the website over HTTP and HTTPS. In this example, you configure a URL pattern list (allowlist) of URLs or addresses that you want to bypass. After you create the URL pattern list, define the custom objects. After defining the custom objects, you apply them to feature profiles to define the activity on each profile, apply the feature profile to the UTM policy, and finally attach the Web filtering UTM policies to the security policies. Table 1 shows information about EWF configuration type, steps, and parameters used in this example.
Table 1: Enhanced Web filtering (EWF) Configuration Type, Steps, and Parameters
Configuration Type | Configuration Steps | Configuration Parameters |
|---|---|---|
URL pattern and custom objects | Configure a URL pattern list (allowlist) of URLs or addresses that you want to bypass. Create a custom object called urllist3 that contains the pattern http://www.example.net 1.2.3.4 |
|
Add the urllist3 custom object to the custom URL category custurl3. |
| |
Feature profiles | Configure the Web filtering feature profile: | |
|
| |
|
| |
|
| |
|
|
Configuration
This example shows how to configure custom URL patterns, custom objects, feature profiles, and security policies.
Configuring Enhanced Web Filtering Custom Objects and URL Patterns
CLI Quick Configuration
To quickly configure this section of the example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.
Starting with Junos OS Release 15.1X49-D110, the “* “ in a wildcard syntax, required to create URL pattern for Web filtering profile, matches all subdomains. For example, *.example.net matches:
http://a.example.net
http://example.net
a.b.example.net
A custom category does not take precedence over a predefined category when it has the same name as one of the predefined categories. Do not use the same name for a custom category that you have used for a predefined category.
Step-by-Step Procedure
The following example requires you to navigate various levels in the configuration hierarchy. For instructions on how to do that, see Using the CLI Editor in Configuration Mode in the CLI User Guide.
To configure custom objects and URL patterns in Enhanced Web Filtering:
- Configure a URL pattern list (allowlist) of URLs or addresses
that you want to bypass. After you create the URL pattern list, you
create a custom URL category list and add the pattern list to it.
Configure a URL pattern list custom object by creating the list name
and adding values to it as follows:
Note Because you use URL pattern lists to create custom URL category lists, you must configure URL pattern list custom objects before you configure custom URL category lists.
[edit security utm]user@host# set custom-objects url-pattern urllist3 value [http://www. example.net 1.2.3.4]Note The guideline to use a URL pattern wildcard is as follows: Use \*\.[]\?* and precede all wildcard URLs with http://. You can use “*” only if it is at the beginning of the URL and is followed by “.”. You can use “?” only at the end of the URL.
The following wildcard syntaxes are supported: http://*.example.net, http://www.example.ne?, http://www.example.n??. The following wildcard syntaxes are not supported: *.example.???, http://*example.net, http://?.
- Create a custom object called urllist3 that contains the
pattern http://www.example.net and then add the urllist3 custom object
to the custom URL category custurl3.[edit security utm]user@host# set custom-objects custom-url-category custurl3 value urllist3
- Create a list of untrusted and trusted sites.[edit security utm]user@host# set custom-objects url-pattern urllistblack value [http://www.untrusted.com 13.13.13.13]user@host# set custom-objects url-pattern urllistwhite value [http://www.trusted.com 11.11.11.11]
- Configure the custom URL category list custom object by
using the URL pattern list of untrusted and trusted sites. [edit security utm]user@host# set custom-objects custom-url-category custblacklist value urllistblackuser@host# set custom-objects custom-url-category custwhitelist value urllistwhite
Results
From configuration mode, confirm your configuration by entering the show security utm custom-objects command. If the output does not display the intended configuration, repeat the instructions in this example to correct.
If you are done configuring the device, enter commit from configuration mode.
Configuring Enhanced Web Filtering Feature Profiles
CLI Quick Configuration
To quickly configure this section of the example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.
Starting in Junos OS Release 12.3X48-D25, new CLI options are available. The http-reassemble and http-persist options are added in the show security utm feature-profile web-filtering command.
Step-by-Step Procedure
The following example requires you to navigate various levels in the configuration hierarchy. For instructions on how to do that, see Using the CLI Editor in Configuration Mode in the CLI User Guide.
To configure the EWF feature profiles:
- Configure the Web filtering URL blocklist, URL allowlist,
and the Web filtering engine.[edit security utm feature-profile web-filtering]user@host# set url-blacklist custblacklistuser@host# set url-whitelist custwhitelistuser@host# set type juniper-enhanced
- Set the cache size and cache timeout parameters for the
configured EWF engine.[edit security utm feature-profile web-filtering]user@host# set juniper-enhanced cache size 500user@host# set juniper-enhanced cache timeout 1800
- Set the server name or IP address and the port number
for communicating with the server. The default host value in the system
is rp.cloud.threatseeker.com.[edit security utm feature-profile web-filtering]user@host# set juniper-enhanced server host rp.cloud.threatseeker.comuser@host# set juniper-enhanced server port 80
- Set the http-reassemble statement to reassemble
the requested packet and the http-persist statement to
check every HTTP request packet in the same session. If the http-reassemble statement is not configured for cleartext HTTP traffic, then EWF
does not reassemble the fragmented HTTP request to avoid incomplete
parsing in the packet-based inspection. If the http-persist statement is not configured for cleartext HTTP traffic, then EWF
does not check every HTTP request packet in the same session.[edit security utm feature-profile web-filtering]user@host# set http-reassembleuser@host# set http-persist
- Create a profile name, and select a category from the
included allowlist and blocklist categories.[edit security utm feature-profile web-filtering]user@host# set juniper-enhanced profile ewf_my_profile category Enhanced_Hacking action log-and-permituser@host# set security utm feature-profile web-filtering juniper-enhanced profile ewf_my_profile category Enhanced_Government action quarantine
- Specify the action to be taken depending on the site reputation
returned for the URL if there is no category match found.[edit security utm feature-profile web-filtering]user@host#set juniper-enhanced profile ewf_my_profile site-reputation-action very-safe permit
- Enter a custom message to be sent when HTTP requests are
blocked. [edit security utm feature-profile web-filtering]user@host# set juniper-enhanced profile ewf_my_profile custom-block-message "***access denied ***"
- Define a redirect URL server so that instead of the device
sending a block page with plain text HTML, the device will send an
HTTP 302 redirect to this redirect server with some special variables
embedded in the HTTP redirect location field. These special variables
can be parsed by the redirect server and serve a special block page
to the client with rich images and formatting.[edit security utm feature-profile web-filtering]user@host# set juniper-enhanced profile ewf_my_profile block-message type custom-redirect-url http://10.10.1.1user@host# set juniper-enhanced profile ewf_my_profile block-message url http://10.10.121.18
If you configure the security utm feature-profile web-filtering juniper-enhanced profile ewf_my_profile block-message statement, then the default block message configuration takes precedence over the security utm feature-profile web-filtering juniper-enhanced profile ewf_my_profile custom-block-message configuration.
- Specify a default action (permit, log and permit, block,
or quarantine) for the profile, when no other explicitly configured
action (blocklist, allowlist, custom category, predefined category
actions, or site reputation actions) is matched .[edit security utm feature-profile web-filtering]user@host# set juniper-enhanced profile ewf_my_profile default block
- Configure the fallback settings (block or log and permit)
for this profile. [edit security utm feature-profile web-filtering]user@host# set juniper-enhanced profile ewf_my_profile fallback-settings default blockuser@host# set juniper-enhanced profile ewf_my_profile fallback-settings server-connectivity blockuser@host# set juniper-enhanced profile ewf_my_profile fallback-settings timeout blockuser@host# set juniper-enhanced profile ewf_my_profile fallback-settings too-many-requests block
- Enter a timeout value in seconds. When this limit is reached,
fallback settings are applied. This example sets the timeout value
to 10. You can also disable the safe-search functionality. By default,
search requests have safe-search strings attached to them, and a redirect
response is sent to ensure that all search requests are safe or strict.[edit security utm feature-profile web-filtering]user@host# set juniper-enhanced profile ewf_my_profile timeout 10user@host# set juniper-enhanced profile ewf_my_profile no-safe-search
Note The timeout value range for SRX210, SRX220, SRX240, SRX300, SRX320, SRX345, SRX380, SRX550, SRX1500, SRX4100, and SRX4200 is 0 through 1800 seconds and the default value is 15 seconds. The timeout value range for SRX3400 and SRX3600 is 1 through 120 seconds and the default value is 3 seconds.
- Configure a UTM policy (mypolicy) for the Web-filtering
HTTP protocol, associating ewf_my_profile to the UTM policy, and attach
this policy to a security profile to implement it.[edit security utm]user@host# set utm-policy mypolicy web-filtering http-profile ewf_my_profileuser@host# set security policies from-zone utm_clients to-zone mgmt policy 1 then permit application-services utm-policy mypolicy
Results
From configuration mode, confirm your configuration by entering the show security utm feature-profile command. If the output does not display the intended configuration, repeat the instructions in this example to correct.
If you are done configuring the device, enter commit from configuration mode.
Attaching Web Filtering UTM Policies to Security Policies
CLI Quick Configuration
To quickly configure this section of the example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.
Step-by-Step Procedure
The following example requires you to navigate various levels in the configuration hierarchy. For instructions on how to do that, see Using the CLI Editor in Configuration Mode in the CLI User Guide.
To attach a UTM policy to a security policy:
- Create the security policy sec_policy.[edit]user@host# set security policies from-zone trust to-zone untrust policy sec_policy
- Specify the match conditions for sec-policy.[edit security policies from-zone trust to-zone untrust policy sec_policy]user@host# set match source-address anyuser@host# set match destination-address anyuser@host# set match application any
- Attach the UTM policy mypolicy to the security policy
sec_policy.[edit security policies from-zone trust to-zone untrust policy sec_policy]user@host# set then permit application-services utm-policy mypolicy
Results
From configuration mode, confirm your configuration by entering the show security policies command. If the output does not display the intended configuration, repeat the configuration instructions in this example to correct it.
After you are done configuring the device, enter commit from configuration mode.
Verification
To confirm that the configuration is working properly, perform these tasks:
Verifying the Status of the Web Filtering Server
Purpose
Verify the Web filtering server status.
Action
From the top of the configuration in operational mode, enter the show security utm web-filtering status command.
user@host> show security utm web-filtering status UTM web-filtering status:
Server status: Juniper Enhanced using Websense server UPMeaning
The command output shows that the Web filtering server connection is up.
Verifying that Web Filtering Statistics Have Increased
Purpose
Verify the increase in Web filtering statistics. The initial counter value is 0; if there is an HTTP request URL hit, then there is a increase in the Web filtering statistics.
Action
From the top of the configuration in operational mode, enter the show security utm web-filtering statistics command.
user@host> show security utm web-filtering statistics UTM web-filtering statistics:
Total requests: 0
white list hit: 0
Black list hit: 0
Queries to server: 0
Server reply permit: 0
Server reply block: 0
Server reply quarantine: 0
Server reply quarantine block: 0
Server reply quarantine permit: 0
Custom category permit: 0
Custom category block: 0
Custom category quarantine: 0
Custom category qurantine block: 0
Custom category quarantine permit: 0
Site reputation permit: 0
Site reputation block: 0
Site reputation quarantine: 0
Site reputation quarantine block: 0
Site reputation quarantine permit: 0
Site reputation by Category 0
Site reputation by Global 0
Cache hit permit: 0
Cache hit block: 0
Cache hit quarantine: 0
Cache hit quarantine block: 0
Cache hit quarantine permit: 0
Safe-search redirect: 0
SNI pre-check queries to server: 1
SNI pre-check server responses: 1
Web-filtering sessions in total: 128000
Web-filtering sessions in use: 0
Fallback: log-and-permit block
Default 0 0
Timeout 0 0
Connectivity 0 0
Too-many-requests 0 0
Meaning
The output displays Web filtering statistics for connections including allowlist and blocklist hits and custom category hits. If there is an HTTP request URL hit, then there is a increase in the Web filtering statistics from an earlier value.
Verifying That the Web Filtering UTM Policy Is Attached to the Security Policy
Purpose
Verify that the Web filtering UTM policy mypolicy is attached to the security policy sec_policy.
Action
From operational mode, enter the show security policy command.
user@host> show security policies global policy-name
mypolicy detailnode0:
-
Global policies:
Policy: mypolicy, State: enabled, Index: 5, Scope Policy: 0, Sequence number: 1
From zones: zone1, zone2
To zones: zone3, zone4
Source addresses: any
Destination addresses: any
Applications: any
Action: permit
Unified Threat Management: enabled
Meaning
The output displays a summary of all security policies configured on the device. If a particular policy is specified, it displays information specific to that policy. If UTM is enabled, then mypolicy is attached to sec_policy.
Understanding the Quarantine Action for Enhanced Web Filtering
UTM Enhanced Web Filtering supports block, log-and-permit, and permit actions for HTTP/HTTPS requests. In addition to this, UTM Enhanced Web Filtering now supports the quarantine action which allows or denies access to the blocked site based on the user’s response to the message.
The following sequence explains how the HTTP or HTTPs request is intercepted, redirected, and acted upon by the quarantine action:
The HTTP client requests URL access.
The device intercepts the HTTP request and sends the extracted URL to the Websense Thread Seeker Cloud (TSC).
The TSC returns the URL category and the site reputation information to the device.
If the action configured for the category is quarantine, the device logs the quarantine action and sends a redirect response to HTTP client.
The URL is sent to the HTTP server for redirecting.
The device shows a warning message stating that the access to the URL is blocked according to the organization’s security policies and prompts the user to respond.
If the user response is “No,” the session is terminated. If the user response is “Yes,” the user is allowed access to the site and such access is logged and reported to the administrator.
The quarantine action is supported only for UTM Enhanced Web Filtering or Juniper enhanced type of Web filtering.
Quarantine Message
The quarantine message sent to the HTTP client is user-configurable and is of the following types:
Default message
The default quarantine message is displayed when a user attempts to access a quarantined website and it contains the following information:
URL name
Quarantine reason
Category (if available)
Site-reputation (if available)
For example, if you have set the action for Enhanced_Search_Engines_and_Portals to quarantine, and you try to access www.search.example.com, the quarantine message is as follows:
***The requested webpage is blocked by your organization's access policy***.
Syslog message.
The syslog message will be logged by the system when the user access the web page that has already been quarantined and marked as block or permit.
The corresponding syslog message on the device under test is:
Jan 25 15:10:40 rodian utmd[3871]: WEBFILTER_URL_BLOCKED: WebFilter: ACTION="URL Blocked" 99.99.99.4(60525)->74.125.224.114(80) CATEGORY="Enhanced_Search_Engines_and_Portals" REASON="by predefined category(quarantine)" PROFILE="ewf-test-profile" URL=www.search.example.com OBJ=/
Starting in Junos OS 12.1X47-D40 and Junos OS Release 17.3R1, the structured log fields have changed. The structured log field changes in the UTM Web filter logs WEBFILTER_URL_BLOCKED, WEBFILTER_URL_REDIRECTED, and WEBFILTER_URL_PERMITTED are as follows:
name -> category
error-message -> reason
profile-name -> profile
object-name -> url
pathname -> obj
User Messages and Redirect URLs for Enhanced Web Filtering (EWF)
Starting with Junos OS Release 15.1X49-D110, a new option, custom-message, is added for the custom-objects statement that enables you to configure user messages and redirect URLs to notify users when a URL is blocked or quarantined for each EWF category. The custom-message option has the following mandatory attributes:
Name: Name of the custom message; maximum length is 59 ASCII characters.
Type: Type of custom message: user-message or redirect-url.
Content: Content of the custom message; maximum length is 1024 ASCII characters.
You configure a user message or redirect URL as a custom object and assign the custom object to an EWF category.
User messages indicate that website access has been blocked by an organization's access policy. To configure a user message, include the type user-message content message-text statement at the [edit security utm custom-objects custom-message message] hierarchy level.
Redirect URLs redirect a blocked or quarantined URL to a user-defined URL. To configure a redirect URL, include the type redirect-url content redirect-url statement at the [edit security utm custom-objects custom-message message] hierarchy level.
The custom-message option provides the following benefits:
You can configure a separate custom message or redirect URL for each EWF category.
The custom-message option enables you to fine-tune messages to support your polices to know which URL is blocked or quarantined.
Only one custom-message configuration option is applied for each category. The custom-message configuration is supported only on Enhanced Web Filtering (EWF). Therefore, only the Juniper EWF engine type is supported.
Starting with Junos OS Release 17.4R1, support for custom category configuration is available for local and Websense redirect profiles.
See also
Example: Configuring Site Reputation Action for Enhanced Web Filtering
This example shows how to configure the site reputation action for both categorized and uncategorized URLs.
Requirements
Before you begin, you should be familiar with Web Filtering and Enhanced Web Filtering. See Web Filtering Overview and Understanding Enhanced Web Filtering Process.
Overview
In this example, you configure Web Filtering profiles to URLs according to defined categories using the site reputation action. You set the URL allowlist filtering category to url-cat-white and the type of Web Filtering engine to juniper-enhanced. Then you set the cache size parameters for Web Filtering and the cache timeout parameters to 1.
Then you create a juniper-enhanced profile called profile ewf-test-profile, set the URL allowlist category to cust-cat-quarantine, and set the reputation action to quarantine.
You enter a custom message to be sent when HTTP requests are quarantined. In this example, the following message is sent: The requested webpage is blocked by your organization's access policy.
You block URLs in the Enhanced_News_and_Media category and permit URLs in the Enhanced_Education category. Then you quarantine the URLs in the Enhanced_Streaming_Media category and configure the device to send the following message: The requested webpage is blocked by your organization's access policy.
In this example, you set the default action to permit. You select fallback settings (block or log-and-permit) for this profile in case errors occur in each configured category. Finally, you set the fallback settings to block.
Configuration
Configuring Site Reputation Action
CLI Quick Configuration
To quickly configure this example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.
Step-by-Step Procedure
The following example requires you to navigate various levels in the configuration hierarchy. For instructions on how to do that, see Using the CLI Editor in Configuration Mode in the CLI User Guide.
To configure the site reputation action:
- Configure the Web Filtering URL allowlist.[edit security utm feature-profile web-filtering]user@host# set url-whitelist custwhitelist
- Specify the Enhanced Web Filtering engine, and set the
cache size parameters.[edit security utm feature-profile web-filtering]user@host# set juniper-enhanced cache size
- Configure the base reputation scores. [edit security utm feature-profile web-filtering]set juniper-enhanced reputation reputation-very-safe 85set juniper-enhanced reputation reputation-moderately-safe 75set juniper-enhanced reputation reputation-fairly-safe 65set juniper-enhanced reputation reputation-suspicious 55
Note The base reputation value must be ordered.
- Set the cache timeout parameters.[edit security utm feature-profile web-filtering]user@host# set juniper-enhanced cache timeout 1
- Create a profile name, and select a category from the
allowlist categories.[edit security utm feature-profile web-filtering]user@host# set juniper-enhanced profile ewf-test-profile category cust-cat-quarantine action quarantine
- Create a profile name, and select a category from the
allowlist categories.[edit security utm feature-profile web-filtering]user@host# set juniper-enhanced profile ewf-test-profile category Enhanced_News_and_Media action block[edit security utm feature-profile web-filtering]user@host# set juniper-enhanced profile ewf-test-profile category Enhanced_Education action permituser@host# set juniper-enhanced profile ewf-test-profile category Enhanced_Education action harmful block[edit security utm feature-profile web-filtering]user@host# set juniper-enhanced profile ewf-test-profile category Enhanced_Streaming_Media action quarantine
- Enter a warning message to be sent when HTTP requests
are quarantined.[edit security utm feature-profile web-filtering]user@host# set juniper-enhanced profile ewf-test-profile quarantine-custom-message "***The requested webpage is blocked by your organization's access policy ***"
- Select a default action (permit, log-and-permit, block,
or quarantine) for the profile, when no other explicitly configured
action (blocklist, allowlist, custom category, predefined category
or site reputation ) is matched .[edit security utm feature-profile web-filtering]user@host# set juniper-enhanced profile ewf-test-profile default permit
- Select fallback settings (block or log-and-permit) for
this profile. [edit security utm feature-profile web-filtering]user@host# set juniper-enhanced profile ewf-test-profile fallback-settings server-connectivity blockuser@host# set juniper-enhanced profile ewf-test-profile fallback-settings timeout block
Results
From configuration mode, confirm your configuration by entering the show security utm command. If the output does not display the intended configuration, repeat the instructions in this example to correct.
If you are done configuring the device, enter commit from configuration mode.
Verification
Confirm that the configuration is working properly.
Verifying the Status of UTM Service
Purpose
Verify the UTM service status.
Action
From operational mode, enter the show security utm status command.
Sample Output
user@host>show security utm statusUTM service status: Running
Verifying the Status of UTM Session
Purpose
Verify the UTM session status.
Action
From operational mode, enter the show security utm session command.
Sample Output
user@host>show security utm sessionUTM session info: Maximum sessions: 4000 Total allocated sessions: 0 Total freed sessions: 0 Active sessions: 0
Verifying the Status of UTM Web Filtering
Purpose
Verify the UTM Web filtering status.
Action
From operational mode, enter the show security utm web-filtering status command.
Sample Output
user@host>show security utm web-filtering statusUTM web-filtering status:
Server status: Juniper Enhanced using Websense server UP
Verifying the Statistics of UTM Web Filtering
Purpose
Verify the Web filtering statistics for connections including allowlist and blocklist hits and custom category hits.
Action
From operational mode, enter the show security utm web-filtering statistics command.
Sample Output
user@host>show security utm web-filtering statisticsUTM web-filtering statistics:
Total requests: 2594
white list hit: 0
Black list hit: 0
Queries to server: 2407
Server reply permit: 1829
Server reply block: 0
Server reply quarantine: 517
Server reply quarantine block: 0
Server reply quarantine permit: 8
Custom category permit: 0
Custom category block: 0
Custom category quarantine: 0
Custom category qurantine block: 0
Custom category quarantine permit: 0
Site reputation permit: 0
Site reputation block: 0
Site reputation quarantine: 0
Site reputation quarantine block: 0
Site reputation quarantine permit: 0
Site reputation by Category 0
Site reputation by Global 0
Cache hit permit: 41
Cache hit block: 0
Cache hit quarantine: 144
Cache hit quarantine block: 0
Cache hit quarantine permit: 1
Safe-search redirect: 0
Web-filtering sessions in total: 16000
Web-filtering sessions in use: 0
Fallback: log-and-permit block
Default 0 0
Timeout 0 0
Connectivity 0 1
Too-many-requests 0 0Verifying the URL status using Log file
Purpose
Verify the blocked and allowed URL status using log file.
Action
To see blocked and allowed URLs, send the utm logs to a syslog server using stream mode. For more information see: Configuring Off-Box Binary Security Log Files.
From operational mode, enter the show log messages | match RT_UTM command.
Sample Output
user@host>show log messages | match RT_UTMRT_UTM: WEBFILTER_URL_BLOCKED: WebFilter: ACTION="URL Blocked" source-zone="trust" destination-zone="untrust" 4.0.0.3(59466)->5.0.0.3(80) SESSION_ID=268436912 APPLICATION="UNKNOWN" NESTED-APPLICATION="UNKNOWN" CATEGORY="URL_Blacklist" REASON="BY_BLACK_LIST" PROFILE="ewf" URL=www.example1.com OBJ=/ username N/A roles N/A application-sub-category N/A urlcategory-risk 0
See also
TAP Mode Support Overview for UTM
In TAP mode, an SRX Series device will be connected to a mirror port of the switch, which provides a copy of the traffic traversing the switch. An SRX Series device in TAP mode processes the incoming traffic from TAP interface and generates security log to display the information on threats detected, application usage, and user details.
Starting in Junos OS Release 19.1R1 you can enable TAP mode on UTM module. When you enable TAP mode on UTM module, the SRX Series device inspects the incoming and outgoing traffic that matches a firewall policy or policies with the enabled UTM service. TAP mode can’t block traffic but generates security logs, reports, and statistics to show the number of threats detected, application usage, and user details. If some packet gets lost in the TAP interface, the UTM terminates the connection, and the TAP mode do not generate any security logs, reports, and statistics for this connection. The UTM configuration remains the same as non-TAP mode.
UTM functionality configured on an SRX Series device continues to work and exchange information from the server. To use UTM functionality when the SRX Series device is configured in TAP mode, you must configure the DNS server to resolve the cloud server’s IP addresses.
To use TAP mode, the SRX device will be connected to a mirror port of the switch, which provides a copy of the traffic traversing the switch. SRX Series device process the incoming traffic from TAP interface and generates security log information to display the information on threats detected, application usage, and user details.
When operating in TAP mode, the SRX Series device performs:
Enhanced Web filtering (EWF) for mirrored HTTP traffic.
Sophos antivirus (SAV) for mirrored HTTP/FTP/SMTP/POP3/IMAP traffic.
Antispam (AS) for mirrored SMTP traffic.