Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Navigation
Guide That Contains This Content
[+] Expand All
[-] Collapse All

    AI-Scripts Test Plan

    Advanced Insight scripts (AI-Scripts) are specialized operational and event scripts from Juniper Networks that detect events on devices running Junos OS and provide data to analyze the events.

    AI-Scripts package data in a structured format called Juniper Message Bundle (JMB) which is retrieved by Junos Space Service Now for display.

    Note: AI-Scripts can be installed on devices by using Service Now. For installing AI scripts on multiple devices, we recommend that you use Service Now as Service Now can install AI-Scripts on multiple devices in a single operation.

    For information about installing AI-Scripts on devices running Junos OS, see Installing an Event Profile on a Device by Using Service Now.

    Testing AI-Scripts Installation

    To check if AI-Scripts is already installed on a device running Junos OS, execute the show version command on the device. The following is a sample of the output of the show version command.

    show version
    Hostname: mx-80-sn
    Model: mx80-48t
    Junos: 13.3R9
    JUNOS Base OS boot [13.3R9]
    JUNOS Base OS Software Suite [13.3R9]
    JUNOS Kernel Software Suite [13.3R9]
    JUNOS Crypto Software Suite [13.3R9]
    JUNOS Packet Forwarding Engine Support (MX80) [13.3R9]
    JUNOS Online Documentation [13.3R9]
    JUNOS Services Application Level Gateways [13.3R9]
    JUNOS Services Jflow Container package [13.3R9]
    JUNOS Services Stateful Firewall [13.3R9]
    JUNOS Services NAT [13.3R9]
    JUNOS Services RPM [13.3R9]
    JUNOS Services Crypto [13.3R9]
    JUNOS Services SSL [13.3R9]
    JUNOS Services IPSec [13.3R9]
    JUNOS Routing Software Suite [13.3R9]
    AI-Scripts [6.0R3.2]
    

    AI-Scripts [6.0R3.2] in the output indicates that AI-Scripts Release 6.0R3.2 is installed on the device.

    Once AI-Scripts is installed on a device successfully, AI-Scripts creates an intelligence JMB (iJMB) automatically. The event activation script that monitors the event daemon (eventd) to detect occurrence of any event on the device runs along when the iJMB is being generated. The renice feature in the newer builds does not let AI-Scripts to take a value of less than 20, that is, scripts in the AI-Scripts bundle of newer releases do not take priority over the current Junos OS processes running on the device.

    % ps -alx | grep ais
        0 56330     1   9   8 20  2760  1468 wait   IN    ??    0:00.01 /bin/sh /var/db/scripts/op/ais_shell2.sh
        0 56414     1  16   8 20  2764  1476 wait   IN    ??    0:00.01 /bin/sh /var/db/scripts/op/ais_shell.sh
        0 56422 56414  20   4 20  1972   772 fifoor IN    ??    0:00.01 cat /var/db/scripts/op/aisfifo
        0 56423 56414  23  -8 20  2456  1104 piperd IN    ??    0:00.01 grep -e /var/db/scripts/op/ais_
    

    Generating a Test JMB

    After you install event scripts on a device, you can find out the events activated on the device by executing the op ais-param-set event-list text command. The following is a sample output of the command:

    regress@mx-480-sn> op ais-param-set event-list text 
       1 RPD_ISIS_OVERLOAD.slax                                   RPD_ISIS_OVERLOAD
       5 RPD_ABORT.slax                                             	RPD_ABORT
       7 RPD_ACTIVE_TERMINATE.slax                           RPD_ACTIVE_TERMINATE
       8 RPD_ASSERT.slax                                            	RPD_ASSERT
       9 RPD_ASSERT_SOFT.slax                                       RPD_ASSERT_SOFT
      10 RPD_ISIS_LSPCKSUM.slax                                   RPD_ISIS_LSPCKSUM
      13 CHASSISD_CLOCK_FAILURE.slax                        CHASSISD_CLOCK_FAILURE
      14 CHASSISD_CMB_READBACK_ERROR.slax        CHASSISD_CMB_READBACK_ERROR
      15 CHASSISD_FAN_FAILURE.slax                            CHASSISD_FAN_FAILURE
      37 CHASSISD_TEMP_SENSOR_FAILURE.slax        CHASSISD_TEMP_SENSOR_FAILURE
      39 CHASSISD_UNEXPECTED_EXIT.slax                  CHASSISD_UNEXPECTED_EXIT
      40 EVENTD_EVENT_SEND_FAILED.slax                 EVENTD_EVENT_SEND_FAILED
      41 FUD_MEMORY_ALLOCATION_FAILURE.slax   FUD_MEMORY_ALLOCATION_FAILURE
      42 L2CPD_ASSERT.slax                                             L2CPD_ASSERT
      43 L2CPD_ASSERT_SOFT.slax                                  L2CPD_ASSERT_SOFT
      44 L2TPD_SERVER_START_FAILED.slax                 L2TPD_SERVER_START_FAILED
      45 PPMD_ASSERT_SOFT.slax                                  PPMD_ASSERT_SOFT
      46 RPD_RT_ERROR.slax                                           RPD_RT_ERROR
    …..
    

    For testing, you can trigger a JMB to be generated for any of the activated events by executing the shell script located in /var/db/scripts/op. All the events are divided into categories based on the type of the event. To trigger a JMB to be generated for an event:

    1. Navigate to the /var/db/scripts/op folder.
      % cd /var/db/scripts/op
    2. Execute the AIS_event_sim.sh script.
      9% ./AIS_event_sim.sh

      The following is a sample of the command output:

      AI-Scripts PROBLEM SIMULATION MENU (AI-Scripts 5.0R5/6.0R3 release)
       ====================================================== 
      1. Hardware Failure
      2. Software Failure
      3. Resource Exhaustion Failure
      4. Daemon crash [WARNING: This test could be service affecting as it kills the daemon]
      5. Unstructured Events
      6. Events that may be triggered on a Backup Routing Engine (platform dependent)
      7. Events added/modified for 4.1R9
      8. Events added/modified for 5.0R3
      9. Events added/modified for 5.0R4/6.0R1
      10. Events added/modified for 5.0R5/6.0R1
      11. Events added/modified for 5.0R6/6.0R3
      Q. Quit
      
      Please enter option [1 - 11,Q]: 3 (Select the type of the event to be triggered)
    3. Enter a number from 1 to 11 to simulate an event as indicated against the number. For example, enter 3 to simulate an event caused by resource exhaustion on the device.

      When you enter 3, the events related to resource exhaustion are listed as follows:

      Calling rex-failure . . .
       RESOURCE EXHAUSTION FAILURE MENU 
       ================================ 
      1. ACCT_MALLOC_FAILURE
      2. ASP_L2TP_NO_MEM
      3. AUTOCONFD_AUTH_NO_MEM
      4. CHASSISD_IPC_MSG_DROPPED
      5. L2CPD_SCHED_SLIP
      6. L2CPD_SYSTEM_CALL_FAILED
      7. RPD_ISIS_OVERLOAD
      8. RPD_OS_MEMHIGH
      9. RT_SCREEN_TCP
      10. RT_SCREEN_UDP
      11. RTPERF_CPU_THRESHOLD_EXCEEDED
      12. SNMPD_SUBAGENT_NO_RESOURCES
      13. TASK_OS_MEMHIGH
      14. VCCPD_PROTOCOL_OVERLOAD
      15. CHASSISD_IFDEV_GET_BY_NAME_FAIL
      
      M. Return to Main Menu
      Q. Exit
      Please enter option [1 - 14,M,Q]: 9   ( Select the event to be triggered)
      RT_SCREEN_TCP . . .
      
    4. Enter a number to generate a JMB for the corresponding event.
      RT_SCREEN_TCP . . .
      Enter a number to generate a JMB for the corresponding event. For example, enter 9 to generate a JMB for RT_SCREEN_TCP event. 
      

      When you enter a number corresponding to an event, the event is triggered with a renice value of 20 and hence the JMB is generated at a lower priority.

      % ps -alx | grep ais
          0  8550     1 119   8 20  2252  1548 wait   IN    ??    0:00.00 sh /var/db/scripts/op/ais_shell2.sh
          0 38837     1 160   8 20  2392  1764 wait   IN    ??    0:00.30 sh /var/db/scripts/op/ais_jmb_chm.sh 239
          0 42681     1   2   8 20  2252  1548 wait   IN    ??    0:00.01 /bin/sh /var/db/scripts/op/ais_shell.sh
          0 42690 42681   0   4 20  1748   924 fifoor IN    ??    0:00.00 cat /var/db/scripts/op/aisfifo
          0 42691 42681   2  -8 20  2076  1260 piperd IN    ??    0:00.00 grep -e /var/db/scripts/op/ais_
        928 43038 35581   0  -8  0  2068  1260 piperd S+    p0    0:00.00 grep ais
      
      

      On a device, when free disk space available in /var/tmpis between 50% to 75%, AI-Scripts generates the JMB and attachments with an additional message stating that the space has reached the warning limit.

      Jan 13 05:07:34  mx-480-sn root: AIS_PROCESSING: Background process (PID=75938) started for eventID 279
      Jan 13 05:07:37  mx-480-sn cscript: AIS_DATA_AVAILABLE: WARNING: Disk at 14 percent Full
      

      Once the disk space reaches the threshold limit, the JMB generation and the attachments are both dampened.

      Jan 13 05:12:24  mx-480-sn root: AIS_PROCESSING: Background process (PID=75938) completed for eventID 279
      Jan 13 05:14:10  mx-480-sn root: AIS_PROCESSING: Background process (PID=81322) started for eventID 310
      Jan 13 05:14:15  mx-480-sn cscript: AIS_DATA_AVAILABLE: ERROR: Disk Full: Dampening JMB attachment generation
      Jan 13 05:14:17  mx-480-sn cscript: AIS_PROCESSING: Deleted the file /var/tmp/mx-480-sn_20170113_051415_310_jmb_ais_prob
      
    5. Log in to the Junos Space GUI and navigate to Service Now > Service Central > Incidents.

      An incident is generated in Service Now for the event that you generated using the event simulator. The incident is usually generated in Service Now within a few minutes after the event is generated on the device.

    Modified: 2017-10-25