Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Guide That Contains This Content
[+] Expand All
[-] Collapse All

    Related Documentation


    Troubleshooting GTP

    This topic explains how to troubleshoot GTP connectivity problems.

    Since the mobility environment is a distributed system, troubleshooting this environment means you need to know details about subscribers, the SPIC/PFE which hosts the subscriber, and which SPIC is the designated SPIC for the GTP peer.

    The show commands listed in here are the primary troubleshooting tools you can use to identify problems in your mobility environment.

    In this scenario, the SGW is sending GTP packets, but the PGW is silently dropping them somewhere. The packets are reaching the SPIC, but they are not reaching GTP.

    1. Check if packets are being dropped in the PFE. You can determine that by seeing if there are packet statistics on the PFE. If not, then packets are not getting this far. You also need to examine the packet steering filters on the PFE.
    2. Check if packets have reached the SPIC. To do this, look at the packet counters on the ms0 interface.
    3. Check if packets have reached the GTP stack. To do this, use:
      user@host> show unifed-edge ggsn-pgw gtp statistics detail

      If the packet has been received on the GTP stack, it should show up in the Rx counters.

      The GTP stack always counts GTP requests and sends a response. So if a packet is received, it will be counted.

    4. Verify if GTP is generating response packets and they are being acknowledged. To do this, use:
      user@host> show unifed-edge ggsn-pgw gtp statistics detail

      Check if the Tx counters are increasing. If they are, check the ms0 packet counters and verify that they are also increasing.

    5. If the ms0 packet statistics are good, then examine the packet steering filter counters. You should see response packet counters incrementing.

    In this scenario, the SGW is sending GTP packets but the PGW is returning errors.

    You should check the GTP traces, because the GTP stack:

    • Handles all the receiving, decoding, and encoding of GTP packets
    • Performs syntax checks on IEs
    • Checks for objects like APN configurations
    • Manages conflicts between GTP peer versions and GTP packets

    In this scenario, sessions are set up successfully, but they are being deleted over time. The likely cause is GTP path management clearing sessions. GTP path management is likely:

    • Discovering that the peer is dead because of an echo request timeout.
    • Restart count in recovery. IE is different from earlier messages.
    1. To examine the problem, enter:
      user@host> show unifed-edge ggsn-pgw gtp statistics detail

      From the data returned, you should be able to see the number of echo requests (Tx) and echo responses (Rx). If Rx is smaller than Tx, then you can conclude that echo responses are not getting to the PGW. To troubleshoot this problem, do the following:

      • Check the statistics on the GTP peer to see if it is receiving echo requests. If not, debug why packets are not being sent out or are not being received by the GTP peer.
      • If the GTP peer is receiving but not responding, check the statistics/traces on the GTP peer.
      • If the GTP peer is sending back an echo response, check if it is being sent from the PFE to the SPIC.
      • If echo responses are going to the SPIC, check the packet steering filters.
      • Check the GTP traces to see if the GTP stack is receiving echo responses but not decoding or accepting them.
    2. Enable GTP history so peer statistics are not deleted after sessions are deleted. To do this, user:
      user@host> show unifed-edge ggsn-pgw gtp peer history
    3. Check how often the peer has been declared down or dead and for what reason. If the peer was down because of a mismatching restart court, this will appear in the stats.
    4. Find out why there was a mismatch in the restart count.
    5. You can workaround a mismatch in the restart count by disabling path management on the PGW. This isolates the real issue behind this behavior. Once the issue is resolved, you can enable path management.

    In this scenario, you experience a random set up failure.

    1. To dump traces for the next n calls to help isolate this problem, use this command:
      user@host> monitor unifed-edge ggsn-pgw call-trace start next-call

    In this scenario, you have subscriber failures.

    1. To troubleshoot this problem, turn on tracing for the subscriber who is having problems. To do this, enter:
      user@host> monitor unifed-edge ggsn-pgw call-trace start IMSI

      This will output traces for this IMSI/MSISDN to help isolate the problem.

    In this scenario, the system is behaving erratically when subscriber counts are high. To troubleshoot this type of problem, you need to try to isolate the problem to an APN, a SPIC, the IMSI, MSISDN, or the GTP SGW peer.

    1. When you have isolated the problem, look at the GTP global statistics to determine which counter is increasing. To do this, use:
      user@host> show unifed-edge ggsn-pgw statistics ?

      This presents a wealth of options for looking at the internals of the GTP stack.

      Also look at the App-fw stats/traces to see what is causing session setup failures.

    In general, troubleshooting GTP makes use of the following show and trace commands:

    • show subscribers gateway
      user@host> show unifed-edge ggsn-pgw subscribers gateway gateway-name ?
    • show gtp peer
      user@host> show unifed-edge ggsn-pgw gtp peer ?
    • show gtp statistics
      user@host> show unifed-edge ggsn-pgw gtp statistics ?
    • trace gtp
      user@host> edit unifed-edge gateway ggsn-pgw pgw-name gtp traceoptions
      file gtp_log size 100m;
      level all
      flag all

    The show output will reveal the counters being incriminated which helps with isolating problems. The trace output shows the GTP encoding/decoding details.


    Related Documentation


    Published: 2011-11-16