Intelligent Inspection
IDP (Intrusion Detection and Prevention) intelligent inspection is designed to ensure that the firewall can continue to operate efficiently even when it is under high load conditions.
Understand Intelligent Inspection
You can enable IDP intelligent inspection and tune it dynamically to reduce the load of full IDP inspection. IDP does not reject or ignore the session by tuning the IDP inspection when the resource limits reach the configured CPU and memory threshold values.
To enable IDP intelligent inspection and the bypass feature, use the set
security idp sensor-configuration flow intel-inspect-enable
command.
- Benefits of Intelligent Inspection
- Security Mechanisms for Tuning IDP Intelligent Inspection
- CPU Utilization
- Memory Utilization
- Limitation
Benefits of Intelligent Inspection
-
Gives importance to critical IDP inspection.
-
Avoids low-priority IDP inspection.
-
Reduces high system resource usage.
Security Mechanisms for Tuning IDP Intelligent Inspection
The following configurations allow for fine-tuning of the IDP system to optimize performance and focus on critical threats:
-
Dynamic policy—Critical, major, and minor are the three important signature severities. You can tune the policy dynamically to include only the signatures of desired severity level. To include signatures of only critical severity, use the command
set security idp sensor-configuration flow intel-inspect-signature-severity critical. To include signatures of critical and major severity, use the commandset security idp sensor-configuration flow intel-inspect-signature-severity major. To include signatures of both critical, major and minor severity, use the commandset security idp sensor-configuration flow intel-inspect-signature-severity minor. By default, attacks with severity as critical are included. -
Content decompression—The content decompression can be avoided only when Intel inspect is enabled and thresholds are reached. The protocol decoder decompresses the protocol content if the content is in a compressed state. You can avoid decompression of the protocol content by configuring the
set security idp sensor-configuration flow intel-inspect-disable-content-decompresscommand. -
Selective protocols—By default, IDP inspects all critical protocols. You can specify the list of critical protocols for IDP processing. To specify the list of protocols, use the
set security idp sensor-configuration flow intel-inspect-protocols protocolcommand. IDP does not inspect noncritical protocols. -
Inspection depth—For each session, by default, IDP inspects all the bytes of the session. By specifying inspection depth, IDP limits inspection to only specified number of bytes. To enable the inspection depth, use the command
set security idp sensor-configuration flow intel-inspect-session-bytes-depth value. By default, the IDP intelligent inspection disables the inspection depth, which means all bytes are inspected.
CPU Utilization
You can configure the threshold limits for IDP inspection. When the CPU usage reaches the configured threshold, IDP intelligent inspection is activated.
To configure the threshold limits, use the following commands:
-
set security idp sensor-configuration flow intel-inspect-cpu-usg-threshold value -
set security idp sensor-configuration flow intel-inspect-cpu-usg-tolerance value
CPU utilization behaves as follows:
-
IDP stops full IDP processing on the new session when the CPU utilization reaches the configured intelligent inspection threshold. IDP processes only the tuned security inspection. This behavior triggers a syslog message to activate the IDP intelligent inspection.
-
IDP continues to function in intelligent inspection mode when the CPU utilization exceeds the intelligent inspection threshold and it lies between the IDP bypass threshold and intelligent inspection lower threshold.
-
IDP starts the full IDP inspection on the new session and triggers a syslog to deactivate the IDP intelligent inspection when the CPU utilization drops below the lower threshold of intelligent inspection.
-
The IDP intelligent bypass feature activates when the CPU utilization reaches the IDP bypass threshold.
Memory Utilization
You can configure the memory limits for the IDP inspection. When the memory usage reaches the configured limit, it activates the IDP intelligent inspection.
To configure the available memory limits, use the following commands:
-
set security idp sensor-configuration flow intel-inspect-free-mem-threshold value -
set security idp sensor-configuration flow intel-inspect-mem-tolerance value
Memory utilization behaves as follows:
-
IDP activates the IDP intelligent inspection mode when the memory utilization reaches the intelligent inspection available memory lower threshold.
-
IDP continues to function in intelligent inspection mode when the memory utilization is in between intelligent inspection memory upper threshold and memory lower threshold.
-
IDP activates the IDP bypass feature when the memory utilization reaches the available memory lower threshold.
-
IDP activates to normal mode when the memory utilization drops and exceeds the intelligent inspection available memory upper threshold.
Limitation
IDP intelligent inspection is supported only at the primary logical system level.
Example: Configure IDP Intelligent Inspection
The IDP intelligent inspection helps the device to recover from the overload state when the device exceeds the configured CPU and memory threshold limit.
This example shows how to enable the IDP intelligent inspection and tune the IDP inspection dynamically to reduce the load of full IDP inspection.
Requirements
Read IDP Sensor Configuration to understand when and how the IDP intelligent inspection and IDP bypass feature works.
Overview
Earlier, when the device reached the configured CPU and memory threshold values, IDP ignored or rejected new sessions. Also, when the device crossed the upper threshold, IDP discarded packets of existing and new session.
Tuning the IDP inspection helps the device gradually increase the CPU and memory utilization and gives importance to critical inspection. This example shows how to tune the IDP inspection after enabling the IDP intelligent inspection.
Configuration
CLI Quick Configuration
To quickly configure this example, copy the following commands, paste them
into a TXT file, remove any line breaks, and change any details necessary to
match your network configuration. Copy and paste the commands into the CLI
at the [edit] hierarchy level. Enter
commit from configuration mode.
set security idp sensor-configuration flow intel-inspect-enable set security idp sensor-configuration flow intel-inspect-cpu-usg-threshold 60 set security idp sensor-configuration flow intel-inspect-cpu-usg-tolerance 15 set security idp sensor-configuration flow intel-inspect-mem-tolerance 5 set security idp sensor-configuration flow intel-inspect-free-mem-threshold 30 set security idp sensor-configuration flow intel-inspect-signature-severity critical set security idp sensor-configuration flow intel-inspect-disable-content-decompress set security idp sensor-configuration flow intel-inspect-session-bytes-depth 2 set security idp sensor-configuration flow intel-inspect-protocols HTTP set security idp sensor-configuration flow intel-inspect-protocols FTP
Procedure
Step-by-Step Procedure
To configure the IDP intelligent inspection:
Enable the IDP intelligent inspection.
[edit security idp sensor-configuration] user@host# set flow intel-inspect-enable
Configure the CPU threshold limit.
[edit security idp sensor-configuration] user@host# set flow intel-inspect-cpu-usg-threshold 60
Configure the CPU tolerance.
[edit security idp sensor-configuration] user@host# set flow intel-inspect-cpu-usg-tolerance 15
Configure the memory tolerance.
[edit security idp sensor-configuration] user@host# set security idp sensor-configuration flow intel-inspect-mem-tolerance 5
Configure the memory limit.
[edit security idp sensor-configuration] user@host# set security idp sensor-configuration flow intel-inspect-memory-limit-lt 30
Specify the severity level.
[edit security idp sensor-configuration] user@host# set flow intel-inspect-signature-severity critical
Disable content decompression.
[edit security idp sensor-configuration] user@host# set flow intel-inspect-disable-content-decompress
Configure the packet inspection depth.
[edit security idp sensor-configuration] user@host# set flow intel-inspect-session-bytes-depth 2
Configure the protocol for inspection.
[edit security idp sensor-configuration] user@host# set flow intel-inspect-protocols HTTP user@host# set flow intel-inspect-protocols FTP
Results
From configuration mode, confirm your configuration by entering the
show security idp sensor-configuration command. If the
output does not display the intended configuration, repeat the instructions
in this example to correct the configuration.
[edit]
user@host# show security idp sensor-configuration
flow {
intel-inspect-enable;
intel-inspect-cpu-usg-threshold 60;
intel-inspect-cpu-usg-tolerance 15;
intel-inspect-free-mem-threshold 30;
intel-inspect-mem-tolerance 5;
intel-inspect-disable-content-decompress;
intel-inspect-session-bytes-depth 2;
intel-inspect-protocols [ HTTP FTP ];
intel-inspect-signature-severity critical;
}
If you are done configuring the devices, enter commit from
configuration mode.
Verification
Confirm that the configuration is working properly.
- Verify the Status of All IDP Flow Counter Values
- Verify the Status of IDP Current Policy
- Protocol-Specific Intelligent-Offload
- Configure Protocol-Specific Offload Limits
Verify the Status of All IDP Flow Counter Values
Purpose
Verify that the IDP intelligent inspection captures counter values.
Action
user@host> show security idp counters flow IDP counters: IDP counter type Value Fast-path packets 580 Slow-path packets 61 Session construction failed 0 Session limit reached 0 Session inspection depth reached 0 Memory limit reached 0 Not a new session 0 Invalid index at ageout 0 Packet logging 0 Policy cache hits 58 Policy cache misses 3 Maximum flow hash collisions 0 Flow hash collisions 0 Gates added 0 Gate matches 0 Sessions deleted 62 Sessions aged-out 0 Sessions in-use while aged-out 0 TCP flows marked dead on RST/FIN 47 Policy init failed 0 Policy reinit failed 0 Number of times Sessions exceed high mark 0 Number of times Sessions drop below low mark 0 Memory of Sessions exceeds high mark 0 Memory of Sessions drops below low mark 0 SM Sessions encountered memory failures 0 SM Packets on sessions with memory failures 0 Number of times Sessions crossed the CPU threshold value that is set 0 Number of times Sessions crossed the CPU upper threshold 0 Sessions constructed 61 SM Sessions ignored 3 SM Sessions dropped 0 SM Sessions interested 61 SM Sessions not interested 101612 SM Sessions interest error 0 Sessions destructed 62 SM Session Create 58 SM Packet Process 580 SM ftp data session ignored by idp 0 SM Session close 59 SM Client-to-server packets 312 SM Server-to-client packets 268 SM Client-to-server L7 bytes 8468 SM Server-to-client L7 bytes 19952 Client-to-server flows ignored 0 Server-to-client flows ignored 0 Server-to-client flows tcp optimized 0 Client-to-server flows tcp optimized 0 Both directions flows ignored 47 Fail-over sessions dropped 0 Sessions dropped due to no policy 0 IDP Stream Sessions dropped due to memory failure 0 IDP Stream Sessions ignored due to memory failure 0 IDP Stream Sessions closed due to memory failure 0 IDP Stream Sessions accepted 0 IDP Stream Sessions constructed 0 IDP Stream Sessions destructed 0 IDP Stream Move Data 0 IDP Stream Sessions ignored on JSF SSL Event 0 IDP Stream Sessions not processed for no matching rules 0 IDP Stream stbuf dropped 0 IDP Stream stbuf reinjected 0 Busy pkts from stream plugin 0 Busy pkts from pkt plugin 0 bad kpp 0 Lsys policy id lookup failed sessions 0 NGAppID Events with no L7 App 0 NGAppID Events with no active-policy 0 NGAppID Detector failed from event handler 0 NGAppID Detector failed from API 0 Busy packets 0 Busy packet Errors 0 Dropped queued packets (async mode) 0 Dropped queued packets failed(async mode) 0 Reinjected packets (async mode) 0 Reinjected packets failed(async mode) 0 AI saved processed packet 0 busy packet count incremented 0 busy packet count decremented 0 session destructed in pme 0 session destruct set in pme 0 kq op hold 0 kq op drop 11 kq op route 47 kq op continue 522 kq op error 0 kq op stop 0 PME wait not set 0 PME wait set 0 PME KQ run not called 0 IDP sessions ignored for content decompression in intel inspect mode 47 IDP sessions ignored for bytes depth limit in intel inspect mode 0 IDP sessions ignored for protocol decoding in intel inspect mode 0 IDP sessions detected CPU usage crossed intel inspect CPU threshold 43 IDP sessions detected mem drop below intel inspect low mem threshold 0
Meaning
The show command displays counters for the IDP intelligent inspection.
Verify the Status of IDP Current Policy
Purpose
Verify that the IDP intelligent inspection captures current policy.
Action
user@host>show security idp status Intelligent Inspection State Details: State: Active State of IDP: Default, Up since: 2018-07-03 14:16:03 PDT (132w4d 09:19 ago) Packets/second: 6 Peak: 12 @ 2019-01-17 22:25:26 PST KBits/second : 249 Peak: 490 @ 2019-01-17 22:25:26 PST Latency (microseconds): [min: 0] [max: 0] [avg: 0] Packet Statistics: [ICMP: 0] [TCP: 127] [UDP: 7] [Other: 0] Flow Statistics: ICMP: [Current: 0] [Max: 6 @ 2019-01-16 20:36:17 PST] TCP: [Current: 4] [Max: 4 @ 2019-01-17 22:34:33 PST] UDP: [Current: 2] [Max: 6 @ 2019-01-17 20:03:55 PST] Other: [Current: 0] [Max: 0 @ 2016-07-03 14:16:03 PDT] Session Statistics: [ICMP: 0] [TCP: 2] [UDP: 1] [Other: 0] Number of SSL Sessions : 0 Policy Name : idp-policy-unified Running Detector Version : 12.6.130180509
Meaning
The show security idp status command displays IDP
current policy. Though you have enabled IDP intelligent inspection, the
state of IDP intelligent inspection can be inactive when you execute
show security idp status operational command. The
reason is the configured CPU and memory threshold values don’t exceed
the resource limit. When the CPU usage reaches the configured threshold,
the state of IDP intelligent inspection becomes active.
Protocol-Specific Intelligent-Offload
The existing Intelligent offload feature in IDP offloads a session when the limit for the examined bytes is reached. In addition, the inspection limit is not granular, and it is applied to all the sessions irrespective of the protocol or service.
With the ability to enable or disable IDP intelligent offloading on a per protocol basis, the administrators can use the flexibility to decide which protocols should leverage the offloading capability. The administrators can also configure the offload limit per protocol.
The Protocol-Specific Intelligent-Offload Configuration feature in the IDP system allows you to tailor inspection depth limits for different protocols, enhancing both performance and security. By configuring separate offload limits for protocols such as SSH and FTP, you can optimize resource usage and ensure more efficient session inspections.
This feature simplifies configuration and management with clear CLI commands, making it easier for administrators to implement and adjust offload settings based on specific network requirements.
Configure Protocol-Specific Offload Limits
You can use the new options to configure the offload limit per protocol by specifying the protocol and setting the offload limit.
[edit]user@host# set security idp
sensor-configuration global intelligent-offload-tunable
?
The range for the offloads limit is the same for all the protocols, that is, 0 to 4294967295 bytes. Offload limit range is same for all protocols and range is 0 to 4294967295 in bytes and 0 means unlimited inspection.
You can configure the limits in KB, MB, and GB, in which case you must append the end of the limit value with k, m, and g respectively.
The set security idp sensor-configuration global
intelligent-offload disable command disables intelligent
offload globally. You cannot configure set security idp
sensor-configuration global intelligent-offload disable as well
as the per protocol custom offload limit.
-
If an offload limit is configured for a protocol, then that offload limit has the highest precedence for that protocol. For example, if the limit is configured for the protocol MYSQL, the session offload limit is taken from the configuration and not from the detector-capabilities.xml file.
-
If no offload limit is configured for a protocol, but limits exist in the detector-capablities.xml file, the limit from the file is applied for that protocol.
-
If the detector-capabilities.xml file lacks offload limits for a protocol, the default limit is 256 KB. In conservative mode, intelligent offload sets the limit to 1 MB. If the intelligent offload is disabled, no offloading occurs.
If you use the set security idp sensor-configuration global
intelligent-offload disable option, the offloading feature is
disabled, and it will work for the entire data inspection until the session
gets closed.