Local Web Filtering
The Web filtering lets you to manage Internet usage by preventing access to inappropriate Web content. There are three types of Web filtering solutions:
Redirect Web filtering
Local Web filtering
Enhanced Web filtering
For more information, see the following topics:
Understanding Local Web Filtering
Local web filtering allows you to define custom URL categories, which can be included in blacklists and whitelists that are evaluated on the SRX Series device. All URLs for each category in a blacklist are denied, while all URLs for each category in a whitelist are permitted.
With local Web filtering, a firewall intercepts every HTTP request in a TCP connection and extracts the URL. A decision is made by the device after it looks up a URL to determine whether it is in the whitelist or blacklist based on its user-defined category. A URL is first compared to the blacklist URLs. If a match is found, the request is blocked. If no match is found, the URL is compared to the whitelist. If a match is found, the request is permitted. If the URL is not in either list, the custom category is taken (block, log-and-permit, or permit). If the URL is not in custom category, the defined default action is taken (block, log-and-permit, or permit). You can permit or block access to a requested site by binding a Web filtering profile to a firewall policy. Local Web filtering provides basic Web filtering without requiring an additional license or external category server.
This topic contains the following sections:
Local Web Filtering Process
The following section describes on how Web traffic is intercepted and acted upon by the Web filtering module.
- The device intercepts a TCP connection.
- The device intercepts each HTTP request in the TCP connection.
- The device extracts each URL in the HTTP request and checks its URL against the user-defined whitelist and blacklist.
- If the URL is found in the blacklist, the request is not permitted and a deny page is sent to the http client. If the URL is found in the whitelist, the request is permitted.
- If the URL is not found in the whitelist or blacklist, the configured default fallback action is applied. If no fallback action is defined, then the request is permitted.
User-Defined Custom URL Categories
To perform local Web filtering, you must define a blacklist and whitelist content that can be applied to the profile.
When defining your own URL categories, you can group URLs and create categories specific to your needs. Each category can have a maximum of 20 URLs. When you create a category, you can add either the URL or the IP address of a site. When you add a URL to a user-defined category, the device performs DNS lookup, resolves the hostname into IP addresses, and caches this information. When a user tries to access a site with the IP address of the site, the device checks the cached list of IP addresses and tries to resolve the hostname. Many sites have dynamic IP addresses, meaning that their IP addresses change periodically. A user attempting to access a site can type an IP address that is not in the cached list on the device. Therefore, if you know the IP addresses of sites you are adding to a category, enter both the URL and the IP address(es) of the site.
You define your own categories using URL pattern list and custom URL category list custom objects. Once defined, you assign your categories to the global user-defined url-blacklist (block) or url-whitelist (permit) categories.
Web filtering is performed on all the methods defined in HTTP 1.0 and HTTP 1.1.
Local Web Filtering Profiles
You configure Web filtering profiles that permit or block URLs according to defined custom categories. A Web filtering profile consists of a group of URL categories assigned one of the following actions:
Blacklist — The device always blocks access to the websites in this list. Only user-defined categories are used with local Web filtering.
Whitelist — The device always allows access to the websites in this list. Only user-defined categories are used with local Web filtering.
A Web filtering profile can contain one blacklist or one whitelist with multiple user-defined categories each with a permit or block action. You can define a default fallback action when the incoming URL does not belong to any of the categories defined in the profile. If the action for the default category is block, the incoming URL is blocked if it does not match any of the categories explicitly defined in the profile. If an action for the default action is not specified, the default action of permit is applied to the incoming URL not matching any category.
Starting with Junos OS Release 17.4R1, custom category configuration is supported for local Web filtering. The custom-message option is also supported in a category for local Web filtering and Websense redirect profiles. Users can create multiple URL lists (custom categories) and apply them to a UTM Web filtering profile with actions such as permit, permit and log, block, and quarantine. To create a global whitelist or blacklist, apply a local Web filtering profile to a UTM policy and attach it to a global rule.
User Messages and Redirect URLs for Web Filtering on SRX Series devices
Starting with Junos OS Release 17.4R1, a new option, custom-message, is added for the custom-objects statement that enables you to configure user messages and redirect URLs to notify users when a URL is blocked or quarantined for each EWF category. The custom-message option has the following mandatory attributes:
Name: Name of the custom message; maximum length is 59 ASCII characters.
Type: Type of custom message: user-message or redirect-url.
Content: Content of the custom message; maximum length is 1024 ASCII characters.
You configure a user message or redirect URL as a custom object and assign the custom object to an EWF category.
User messages indicate that website access has been blocked by an organization's access policy. To configure a user message, include the type user-message content message-text statement at the [edit security utm custom-objects custom-message message] hierarchy level.
Redirect URLs redirect a blocked or quarantined URL to a user-defined URL. To configure a redirect URL, include the type redirect-url content redirect-url statement at the [edit security utm custom-objects custom-message message] hierarchy level.
The custom-message option provides the following benefits:
You can configure a separate custom message or redirect URL for each EWF category.
The custom-message option enables you to fine-tune messages to support your polices to know which URL is blocked or quarantined.
Profile Matching Precedence
When a profile employs several categories for URL matching, those categories are checked for matches in the following order:
- If present, the global blacklist is checked first. If a match is made, the URL is blocked. If no match is found...
- The global whitelist is checked next. If a match is made, the URL is permitted. If no match is found...
- User-defined categories are checked next. If a match is made, the URL is blocked or permitted as specified.
See also
Example: Configuring Local Web Filtering
This example shows how to configure local Web filtering for managing website access.
Requirements
This example uses the following hardware and software components:
SRX1500 device
Junos OS Release 12.1X46-D10 or later
Before you begin, learn more about Web filtering. See Web Filtering Overview.
Overview
In this example you configure local Web filtering custom objects, local Web filtering feature profiles, and local Web filtering UTM policies. You also attach local Web filtering UTM policies to security policies. Table 1 shows information about local Web filtering configuration type, steps, and parameters used in this example.
Table 1: Local Web filtering Configuration Type, Steps, and Parameters
Configuration Type | Configuration Steps | Configuration Parameters |
|---|---|---|
URL pattern and custom objects | Configure a URL pattern list of URLs or addresses that you want to bypass. Create a custom object called urllis1 that contains the pattern [http://www.example1.net 192.0.2.0] Create a custom object called urllist2 that contains the pattern [http://www.example2.net 192.0.2.3] Create a custom object called urllist3 that contains the pattern [http://www.example3.net 192.0.2.9] Create a custom object called urllist4 that contains the pattern [http://www.example4.net 192.0.2.8] |
|
The urllist1 and urllist2 custom objects are then added to the custom URL categories cust-black-list, and cust-permit-list respectively. |
| |
Feature profiles | Configure the Web filtering feature profile: | |
|
| |
|
| |
|
| |
UTM policies | Create the UTM policy utmp5 and attach it to the profile localprofile1. In the final configuration example, attach the UTM policy utmp5 to the security policy p5. |
|
Configuration
Configuring Local Web Filtering Custom Objects and URL Patterns
Attaching Local Web Filtering UTM Policies to Security Policies
Configuring Local Web Filtering Custom Objects and URL Patterns
CLI Quick Configuration
To quickly configure this section of the example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.
Starting in Junos OS Release 15.1X49-D110, the “* “ in a wildcard syntax, used for URL pattern Web filtering profile, matches all subdomains. For example, *.example.net matches:
http://a.example.net
http://example.net
aaa.example.net
Step-by-Step Procedure
To configure local Web filtering using the CLI:
- Configure a URL pattern list custom object by creating
the list name and adding values to it as follows:
Note Because you use URL pattern lists to create custom URL category lists, you must configure URL pattern list custom objects before you configure custom URL category lists.
[edit]user@host# set security utm custom-objects url-pattern urllist1 value [http://www.example1.net 192.0.2.0]user@host# set security utm custom-objects url-pattern urllist2 value [http://www.example2.net 192.0.2.3]user@host# set security utm custom-objects url-pattern urllist3 value [http://www.example3.net 192.0.2.9]user@host# set security utm custom-objects url-pattern urllist4 value [http://www.example4.net 192.0.2.8]Note The guideline to use a URL pattern wildcard is as follows: Use \*\.[]\?* and precede all wildcard URLs with http://. You can use “*” only if it is at the beginning of the URL and is followed by “.”. You can use “?” only at the end of the URL.
The following wildcard syntaxes are supported: http://*.example.net, http://www.example.ne?, http://www.example.n??. The following wildcard syntaxes are not supported: *.example.???, http://*example.net, http://?.
- Applying the URL pattern to a custom URL category.[edit]user@host# set security utm custom-objects custom-url-category cust-black-list value urllist1user@host# set security utm custom-objects custom-url-category cust-permit-list value urllist2user@host# set security utm custom-objects custom-url-category custurl3 value urllist3user@host# set security utm custom-objects custom-url-category custurl4 value urllist4
Results
From configuration mode, confirm your configuration by entering the show security utm custom-objects command. If the output does not display the intended configuration, repeat the configuration instructions in this example to correct it.
If you are done configuring the device, enter commit from configuration mode.
Apply Custom Objects to the Feature Profiles
CLI Quick Configuration
To quickly configure this section of the example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.
Step-by-Step Procedure
The following example requires you to navigate various levels in the configuration hierarchy. For instructions on how to do that, see Using the CLI Editor in Configuration Mode in the CLI User Guide.
To configure local Web filtering feature profiles:
- Configure the Web filtering URL blacklist, URL whitelist,
and the Web filtering engine. [edit security utm feature-profile web-filtering]user@host# set url-whitelist custurl3user@host# set url-blacklist custurl4user@host# set type juniper-local
- Create a profile name, and select a category from the
included permit and blacklist categories. The custom category action
could be block, permit, log-and-permit, and quarantine.[edit security utm feature-profile web-filtering]user@host# set juniper-local profile localprofile1 category cust-black-list action blockuser@host# set juniper-local profile localprofile1 category cust-permit-list action log-and-permit
- Define a redirect URL server so that instead of the device
sending a block page with plain text HTML, the device send an HTTP
302 redirect to this redirect server with special variables embedded
in the HTTP redirect location field. These special variables are parsed
by the redirect server and serve as a special block page to the client
with images and a clear text format.[edit security utm feature-profile web-filtering]user@host# set security utm feature-profile web-filtering juniper-local profile localprofile1 block-message type custom-redirect-urluser@host# set security utm feature-profile web-filtering juniper-local profile localprofile1 block-message url http://192.0.2.10
- Enter a custom message to be sent when HTTP requests are
blocked. [edit security utm feature-profile web-filtering]user@host# set juniper-local profile localprofile1 custom-block-message “Access to this site is not permitted”
- Specify a default action (permit, log and permit, block,
or quarantine) for the profile, when no other explicitly configured
action (blacklist, whitelist, custom category, predefined category
actions, or site reputation actions) is matched .[edit security utm feature-profile web-filtering]user@host# set juniper-local profile localprofile1 default log-and-permit
- Configure fallback settings (block or log and permit)
for this profile. [edit security utm feature-profile web-filtering]user@host# set juniper–local profile localprofile1 fallback-settings default blockuser@host# set juniper–local profile localprofile1 fallback-settings too-many-requests block
Results
From configuration mode, confirm your configuration by entering the show security utm feature-profile command. If the output does not display the intended configuration, repeat the configuration instructions in this example to correct it.
If you are done configuring the device, enter commit from configuration mode.
Attaching Web Filtering UTM Policies to Security Policies
CLI Quick Configuration
To quickly configure this section of the example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.
Step-by-Step Procedure
To configure a UTM policy:
- Create the UTM policy referencing a profile. Apply the
Web filtering profile to the UTM policy.[edit]user@host# set security utm utm-policy utmp5 web-filtering http-profile localprofile1
Results
From configuration mode, confirm your configuration by entering the show security utm command. If the output does not display the intended configuration, repeat the configuration instructions in this example to correct it.
For brevity, this show command output includes only the configuration that is relevant to this example. Any other configuration on the system has been replaced with ellipses (...).
If you are done configuring the device, enter commit from configuration mode.
Attaching Local Web Filtering UTM Policies to Security Policies
CLI Quick Configuration
To quickly configure this section of the example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.
Step-by-Step Procedure
To attach a UTM policy to a security policy:
- Create and configure the security policy.[edit security policies from-zone trust to-zone untrust policy p5]user@host# set match source-address anyuser@host# set match destination-address anyuser@host# set match application junos-http
- Apply the UTM policy to the security policy.[edit security policies from-zone trust to-zone untrust policy p5]user@host# set then permit application-services utm-policy utmp5
Results
From configuration mode, confirm your configuration by entering the show security policies command. If the output does not display the intended configuration, repeat the configuration instructions in this example to correct it.
If you are done configuring the device, enter commit from configuration mode.
Verification
To confirm that the configuration is working properly, perform the following task:
Verifying the Statistics of UTM Web Filtering
Purpose
Verify the Web filtering statistics for connections including whitelist and blacklist hits and custom category hits.
Action
From operational mode, enter the show security utm web-filtering statistics command.
Sample Output
user@host>show security utm web-filtering statisticsUTM web-filtering statistics:
Total requests: 0
white list hit: 0
Black list hit: 0
Custom category permit: 0
Custom category block: 0
Custom category quarantine: 0
Custom category qurantine block: 0
Custom category quarantine permit: 0
Web-filtering sessions in total: 0
Web-filtering sessions in use: 0
Fallback: log-and-permit block
Default 0 0
Timeout 0 0
Connectivity 0 0
Too-many-requests 0 0