Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Navigation
Guide That Contains This Content
[+] Expand All
[-] Collapse All

    General Procedures for Troubleshooting the IP/MPLSView Installation

    This topic outlines general procedures for troubleshooting issues with IP/MPLSView, such as unexpected behavior, hanging, Java exceptions, or error messages. The first step is to check the status of the application server. The second step is to identify any conflicting, missing, or lingering processes. The third step is to gracefully shut down and restart those problematic processes. Specific troubleshooting questions are answered in other topics in the Troubleshooting the IP/MPLSView Installation chapter.

    Checking the Status of the Application Server

    • Log in to the application server as the wandl user, or as the administrative user that installed IP/MPLSView, and change directory to /u/wandl/bin. Use the ./status_mplsview command to see the status of the application server.
      # ./status_mplsview
      NPAT Server (pid=21896) detected on port 7000
      MySQL detected on port 3333, pid=21933
      JMS (pid=21956) detected, Memory usage: 122M/256M, CPU usage: 0.1%
      Web server (pid=21975) detected, Memory usage: 249M/256M, CPU usage: 0.5%
      Web server started up successfully!
      Task Server (pid=21992) detected, Memory usage: 72M/512M, CPU usage: 0.7%
      Event Server (pid=22007) detected, Memory usage: 90M/256M, CPU usage: 0.1%
      Warning! : Threshold Server not detected
      DGS (pid=22032) detected, Memory usage: 108M/256M, CPU usage: 0.1%
      Aggregation crontask scheduled at 0:30
      Bulk stat interface traffic generation crontask scheduled
      LDAP detected on port 3389, pid=22067
      IP/MPLSView Application Monitor (pid=12529) detected
      SNMP Trap Server Process detected, pid=22083
      Errors : 0
      Warnings : 1
      Active Process for ROUTER-module: Process id=13175, User name=wandl.
      Wed Sep 08 12:24:06 2010 IP: 172.17.7.7 Process ID: 13157 User wandl
      Wed Sep 08 14:22:20 2010 IP: 172.18.8.8 Process ID: 21996 User tester

      This command displays various processes used by the IP/MPLSView. Note some of your processes might be different from the example, depending on your license.

    • The Error message occurs when processes required to run IP/MPLSView are not detected.
    • The Warning message occurs when expected processes are not detected; however, these processes are not required to run IP/MPLSView.
    • The Active Process for ROUTER-module shows open clients. It displays the timestamp when the client started, the IP that started the client, the process ID, and user ID.

    Identifying Processes to Fix

    1. Using the ./status_mplsview command in combination with the Linux ps command can help identify the processes to fix.
    2. Use the ps -ef | grep java command to report all the Java processes used by IP/MPLSView. The Java processes reported should be the same as the pid’s detected using the ./status_mplsview command. If there are duplicate or missing Java processes reported by the ps command, then this could lead to conflicts.
    3. Use the ps -ef | grep wandl command to report all the wandl processes used by IP/MPLSView, or replace wandl with the administrative user that installed IP/MPLSview. The wandl processes reported should be the same as the pid’s detected using the ./status_mplsview command. If there are duplicate or missing wandl processes reported by the ps command, then this could lead to conflicts.
    4. Use the ps -ef | grep <user> command, where <user> is the user detected by the ./status_mplsview command, to report user processes used by IP/MPSLView, Verify the user is not running lingering or outdated processes.

      The LDAP pid can be verified using the ps -ef | grep ldap command.

    5. The following is an example ps -ef | grep java command output of when the application server is running properly:
      # ps -ef | grep java
      root 25995 25991    0 00:43:01 ?      0:09   /usr/jdk/latest/bin/java
      -version:1.5+ -jar /usr/lib/patch/swupa.jar -autoAnaly
      noaccess    641  1  0   Aug 12 ?      18:21 /usr/java/bin/java -server
      -Xmx128m      -XX:+UseParallelGC -XX:ParallelGCThreads=4
         wandl   12529   1   0  12:19:18    pts/3 0:46
      /home/wandl/ipmplsview/java/bin/java -Dprogram.name=appmonitor -Xmx128M -D
         wandl   22032  1    0  16:44:59 ?        0:36
      /home/wandl/ipmplsview/java/bin/java -Dprogram.name=DGS -Xmx256M -cp /u/wa
         wandl   22007  1    0  16:44:46 ?        1:00
      /home/wandl/ipmplsview/java/bin/java -Dprogram.name=eventserver -Xmx256M -
         wandl   21956  1    0  16:44:20 ?        2:03
      /home/wandl/ipmplsview/java/bin/java -Dprogram.name=HornetQjms.sh -server -X
         wandl   21992  1    1 16:44:41 ?         16:21 /u/wandl/java/bin/java -Xmx5
      12M -server -Djava.security.policy=taskserver.polic
         wandl   13686  1    0 12:25:42     pts/3    0:09
      /home/wandl/ipmplsview/java/bin/java -Dprogram.name=threshold -Xmx256M -Dj
         root    26035 26026 49 00:43:02 ?        1637:00 /usr/jdk/latest/bin/java
      -version:1.5+ -Djava.library.path=/usr/lib/cc-ccr/lib
         root    22083  1    0 16:45:11     pts/3    0:53
      /home/wandl/ipmplsview/java/bin/java -Djava.util.logging.config.file=/u/wa
         wandl   21975  1    0 16:44:31 ?          6:27
      /home/wandl/ipmplsview/java/bin/java -Dprogram.name=webserver.sh -server -

    Restarting the Application Server

    1. Once the problematic processes are identified, some processes can be stopped and started individually although we recommend that you stop and start the entire application server. Close all clients running IP/MPLSView by exiting or closing the window. Be sure to save your work. If you do not have direct access to close a client, these are closed automatically when the stop_mplsview command is used.

      Change directory to the /u/wandl/bin directory and execute the ./stop_mplsview command. This command attempts a graceful shutdown of the npatserver, rtserver, and filemanager processes and processes used by IP/MPLSView. Open clients receive a pop-up message warning the user that the server will be shut down in 1 minute. Users should save their work and close the client. After the stop_mplsview command completes the shutdown messages, wait at least 2 minutes to allow the processes to shut down.

      # ./stop_mplsview
      Shutdown IP/MPLSView Application Monitor(pid=12529) ...
      Would you like to stop the SNMP Trap Server (default=no)? [y/n] y
      Shutdown SNMP Trap Server(pid=22083) ...
      Shutdown LDAP Server(pid=22067) ...
      Removing Aggregation crontask...
      Aggregation crontask removed
      Removing Bulk stat interface traffic generation crontask...
      Bulk stat interface traffic generation crontask removed
      Shutdown DGS(pid=22032) ...
      Shutdown Event Server(pid=22007) ...
      Shutdown Threshold Server(pid=13686) ...
      Shutdown Task Manager(pid=21992) ...
      Shutdown Web server(pid=21975) ...
      Shutdown MySQL(pid=21933) ...
      Shutdown NPAT Server(pid=21896) on port 7000 ...
    2. Execute the ps -ef | grep java command and ps -ef | grep wandl command to verify there are no lingering Java and wandl processes related to IP/MPLSView. If there are processes still running, use the Linux kill command to force a shutdown. If lingering processes cannot be shut down, then the server should be rebooted as the last option.

      The following is an example ps -ef | grep java command of when the application server is shut down properly with no lingering processes:

      # ps -ef | grep java
           root  25995  25991  0   Sep 08 ?    0:20 /usr/jdk/latest/bin/java
      -version:1.5+ -jar /usr/lib/patch/swupa.jar -autoAnaly
      noaccess    641       1  0   Aug 12 ?    18:42 /usr/java/bin/java -server
      -Xmx128m -XX:+UseParallelGC -XX:ParallelGCThreads=4
         wandl  25845   28108     0 09:29:37 pts/3     0:00  grep java
          root  26035   26026 50   Sep 08 ?    3874:45 /usr/jdk/latest/bin/java
      -version:1.5+ -Djava.library.path=/usr/lib/cc-ccr/lib

      After verifying the processes were properly shut down, use the ./startup_mplsview command as the wandl user to restart the application server. After the startup_mplsview command completes the startup messages, wait at least 3 minutes to allow processes to start. Use the ./status_mplsview command to check the status of the application server and verify there are no warning or error messages.

      Launch the client to confirm the issues are resolved.

    Restarting Individual Processes

    The following table lists the commands to stop and start individual processes. This should be done as the wandl user or the user that installed IP/MPLSView, in the /u/wandl/bin directory. Do not run these commands as root because this might lead to ownership and process conflicts. Some commands are hidden in the directory and can be viewed with ls -a command. Note that if Application Monitor is running, it automatically restarts processes when it no longer detects a heartbeat from that process. To prevent a process from automatically restarting and keeping it stopped, Application Monitor should be stopped first.

    Process

    Effected Function

    Command

    Application Monitor

    System Monitor

    ./.appmonitor start or stop

    Event Server

    Event Browser

    ./.eventserver start or stop

    Threshold Server

    Threshold Editor

    ./.threshold start or stop

    LDAP Server

    User Authentication

    ./.ldap start or stop

    SNMP Trap Server

    Event Browser

    ./.snmptrap start or stop

    Task Server

    Task Manager

    ./.tmng start or stop

    DGS

    Traffic Collection

    ./.dgs start or stop

    MariaDB

    MariaDB Database

    ./.mysql start or stop

    Webserver

    Web

    ./webserver.sh

    Application Server

    All

    ./startup_mplsview or ./stop_mplsview

    Modified: 2016-11-08