Performing Loopback Testing for 10, 40, and 100 Gigabit Ethernet Interfaces
Checklist for Using Loopback Testing for 10, 40, and 100 Gigabit Ethernet Interfaces
Purpose
To use loopback testing to isolate 10, 40, and 100 Gigabit Ethernet interface problems.
Action
Table 1 provides links and commands for using loopback testing for 10, 40, and 100 Gigabit Ethernet interfaces.
Tasks |
Command or Action |
---|---|
Diagnose a Suspected Hardware Problem with a Gigabit Ethernet Interface | |
Connect the transmit port to the receive port. |
|
|
|
|
|
|
|
|
|
|
|
|
|
Diagnose a Suspected Circuit Problem | Perform Steps 2 through 8 from Diagnose a Suspected Hardware Problem with a Fast Ethernet or Gigabit Ethernet Interface. |
Diagnose a Suspected Hardware Problem with a Gigabit Ethernet Interface
Problem
Description
When you suspect a hardware problem, take the following steps to help verify if there is a problem.
Solution
To diagnose a suspected hardware problem with the Ethernet interface, follow these steps:
Create a Loopback
You can create a physical loopback or configure a local loopback to help diagnose a suspected hardware problem. Creating a physical loopback is recommended because it allows you to test and verify the transmit and receive ports. If a field engineer is not available to create the physical loopback, you can configure a local loopback for the interface. The local loopback creates a loopback internally in the Physical Interface Card (PIC).
Create a Physical Loopback for a Fiber-Optic Interface
Action
To create a physical loopback at the port, connect the transmit port to the receive port using a known good fiber cable.
Make sure you use single-mode fiber for a single-mode port and multimode fiber for a multimode port.
Meaning
When you create and then test a physical loopback, you are testing the transmit and receive ports of the PIC. This action is recommended if a field engineer is available to create the physical loop as it provides a more complete test of the PIC.
See Also
Configure a Local Loopback
Action
To configure a local loopback without physically connecting the transmit port to the receive port, follow these steps:
In configuration mode, go to the following hierarchy level:
[edit] user@router# edit interfaces
interface-name
gigether-optionsConfigure the local loopback:
[edit interfaces
interface-name
gigether-options user@router# set loopbackVerify the configuration:
user@router# show
For example:
[edit interfaces xe-2/0/0 gigether-options] user@router# show loopback;
Commit the change:
user@router# commit
For example:
[edit interfaces xe-2/0/0 gigether-options] user@router# commit commit complete
When you create a local loopback, you create an internal loop
on the interface being tested. A local loopback loops the traffic
internally on that PIC. A local loopback tests the interconnection
of the PIC but does not test the transmit and receive ports. On an
Ethernet interface, you cannot create a remote loopback, therefore
there is no option to use a local
or remote
statement.
Simply including the loopback
statement at the [edit
interfaces interface-name
gigether-options
hierarchy level, places the interface into local loopback mode.
Remember to delete the loopback statement after completing the test.
See Also
Verify That the Gigabit Ethernet Interface Is Up
Purpose
Display the status of the Gigabit Ethernet interface to provide the information you need to determine whether the physical link is up or down.
Action
To verify that the status of the Gigabit Ethernet interface is up, use the following Junos OS command-line interface (CLI) operational mode command:
user@router> show interfaces (xe-fpc
/pic
/port
| et-fpc
/pic
/port
)
Sample Output
user@router# show interfaces xe-2/0/0 extensive Physical interface: xe-2/0/0, Enabled, Physical link is Up Interface index: 187, SNMP ifIndex: 591, Generation: 190 Link-level type: Ethernet, MTU: 1514, MRU: 1522, LAN-PHY mode, Speed: 10Gbps, BPDU Error: None, Loop Detect PDU Error: None, MAC-REWRITE Error: None, Loopback: Local, Source filtering: Disabled, Flow control: Enabled, Speed Configuration: Auto Pad to minimum frame size: Disabled Device flags : Present Running Loop-Detected Interface flags: SNMP-Traps Internal: 0x4000 Link flags : None CoS queues : 8 supported, 8 maximum usable queues Schedulers : 0 Hold-times : Up 4000 ms, Down 0 ms Damping : half-life: 0 sec, max-suppress: 0 sec, reuse: 0, suppress: 0, state: unsuppressed Current address: d8:18:d3:b3:6d:ea, Hardware address: d8:18:d3:b3:6d:ea Last flapped : 2019-04-20 17:13:55 PDT (13w4d 21:42 ago) Statistics last cleared: 2019-07-25 14:55:21 PDT (00:01:01 ago) Traffic statistics: Input bytes : 537600 0 bps Output bytes : 539600 0 bps Input packets: 6400 0 pps Output packets: 6400 0 pps IPv6 transit statistics: Input bytes : 0 Output bytes : 0 Input packets: 0 Output packets: 0 Dropped traffic statistics due to STP State: Input bytes : 0 Output bytes : 0 Input packets: 0 Output packets: 0 Input errors: Errors: 0, Drops: 0, Framing errors: 0, Runts: 0, Policed discards: 0, L3 incompletes: 0, L2 channel errors: 0, L2 mismatch timeouts: 0, FIFO errors: 0, Resource errors: 0 Output errors: Carrier transitions: 0, Errors: 0, Drops: 0, Collisions: 0, Aged packets: 0, FIFO errors: 0, HS link CRC errors: 0, MTU errors: 0, Resource errors: 0 Egress queues: 8 supported, 4 in use Queue counters: Queued packets Transmitted packets Dropped packets 0 6400 6400 0 1 0 0 0 2 0 0 0 3 0 0 0 Queue number: Mapped forwarding classes 0 best-effort 1 expedited-forwarding 2 assured-forwarding 3 network-control Active alarms : None Active defects : None PCS statistics Seconds Bit errors 0 Errored blocks 0 MAC statistics: Receive Transmit Total octets 652800 652800 Total packets 6400 6400 Unicast packets 6400 6400 Broadcast packets 0 0 Multicast packets 0 0 CRC/Align errors 0 0 FIFO errors 0 0 MAC control frames 0 0 MAC pause frames 0 0 Oversized frames 0 Jabber frames 0 Fragment frames 0 VLAN tagged frames 0 Code violations 0 Total errors 0 0 Filter statistics: Input packet count 6400 Input packet rejects 0 Input DA rejects 0 Input SA rejects 0 Output packet count 6400 Output packet pad count 0 Output packet error count 0 CAM destination filters: 0, CAM source filters: 0 Packet Forwarding Engine configuration: Destination slot: 0 (0x00) CoS information: Direction : Output CoS transmit queue Bandwidth Buffer Priority Limit % bps % usec 0 best-effort 95 9500000000 95 0 low none 3 network-control 5 500000000 5 0 low none Preclassifier statistics: Traffic Class Received Packets Transmitted Packets Dropped Packets real-time 0 0 0 network-control 6400 6400 0 best-effort 0 0 0 Link Degrade : Link Monitoring : Disable Interface transmit statistics: Disabled Logical interface xe-2/0/0.0 (Index 353) (SNMP ifIndex 599) (Generation 175) Flags: Up SNMP-Traps 0x4004000 Encapsulation: ENET2 Traffic statistics: Input bytes : 537600 Output bytes : 539000 Input packets: 6400 Output packets: 6400 Local statistics: Input bytes : 0 Output bytes : 9800 Input packets: 0 Output packets: 100 Transit statistics: Input bytes : 537600 0 bps Output bytes : 529200 0 bps Input packets: 6400 0 pps Output packets: 6300 0 pps Protocol inet, MTU: 1500 Max nh cache: 75000, New hold nh limit: 75000, Curr nh cnt: 1, Curr new hold cnt: 0, NH drop cnt: 0 Generation: 206, Route table: 0 Flags: Sendbcast-pkt-to-re Addresses, Flags: Is-Preferred Is-Primary Destination: 10.108.120.0/30, Local: 10.108.120.1, Broadcast: 10.108.120.3, Generation: 146 Protocol multiservice, MTU: Unlimited, Generation: 207, Route table: 0 Policer: Input: __default_arp_policer__
Meaning
The sample output shows that the link is up and there are no alarms in this loopback configuration. When an internal loopback is configured, the physical loopback should come up without an alarm.
Sample Output
When you see that the physical link is down, there may be a problem with the port. The following output is an example of the show interfaces et-fpc/pic/port command when the physical link is down:
user@router> show interfaces et-3/0/1 Physical interface: et-3/0/1, Enabled, Physical link is Down Interface index: 620, SNMP ifIndex: 564 Link-level type: Ethernet, MTU: 1514, MRU: 1522, Speed: 40Gbps, BPDU Error: None, Loop Detect PDU Error: None, Loopback: Disabled, Source filtering: Disabled, Flow control: Enabled Pad to minimum frame size: Disabled Device flags : Present Running Down Interface flags: Hardware-Down SNMP-Traps Internal: 0x4000 Link flags : None CoS queues : 8 supported, 8 maximum usable queues Schedulers : 0 Current address: 54:e0:32:71:e3:e2, Hardware address: 54:e0:32:71:e3:e2 Last flapped : 2019-07-05 09:10:02 PDT (3d 14:46 ago) Input rate : 0 bps (0 pps) Output rate : 0 bps (0 pps) Active alarms : LINK Active defects : LINK, LOCAL-FAULT PCS statistics Seconds Bit errors 2 Errored blocks 6 Ethernet FEC Mode : NONE Ethernet FEC statistics Errors FEC Corrected Errors 0 FEC Uncorrected Errors 0 FEC Corrected Errors Rate 0 FEC Uncorrected Errors Rate 0 Interface transmit statistics: Disabled
Meaning
The sample output shows that the physical link is down and there are active alarms and defects.
Table 2 presents problem situations and actions for a physical link that is down.
Problem |
Action |
---|---|
Cable mismatch |
Verify that the fiber connection is correct. |
Damaged and/or dirty cable |
Verify that the fiber can successfully loop a known good port of the same type. |
Too much or too little optical attenuation |
Verify that the attenuation is correct per the PIC optical specifications. |
The transmit port is not transmitting within the dBm optical range per the specifications |
Verify that the Tx power of the optics is within range of the PIC optical specification. |
Mismatch between the cable type and the port |
Verify that a single-mode fiber cable is connected to a single-mode interface and that a multimode fiber cable is connected to a multimode interface. (This problem does not always cause the physical link to go down; errors and dropped packets are sometimes the result.) |
Configure a Static Address Resolution Protocol Table Entry
Purpose
Configure a static Address Resolution Protocol (ARP) entry to allow a packet to be sent out of a looped Ethernet interface.
Remove the static ARP entry at the end of the loop test after you have completed the tests and monitored interface traffic.
Action
To configure a static ARP table entry for a Gigabit Ethernet interface, follow these steps:
Find the Media Access Control (MAC) address for the Gigabit Ethernet interface:
user@router#
run show interfaces xe-2/0/0 extensive | match "Current address"
Current address: d8:18:d3:b3:6d:ea, Hardware address: d8:18:d3:b3:6d:ea
In configuration mode, go to the following hierarchy level:
[edit] user@router# edit interfaces
interface-name
unitlogical-unit-number
family inet addressaddress
Configure the static ARP entry:
user@router# set arp
ip-address
macmac-address
Commit the configuration:
user@router# commit
Verify that the static ARP entry is installed:
[edit interfaces xe-2/0/0.0 unit 0 family inet address 10.108.120.1/30] user@router#run show arp no-resolve
MAC Address Address Interface Flags 02:01:00:00:00:05 10.0.0.5 em1.0 none 00:00:5e:00:01:01 10.85.175.1 fxp0.0 none d8:b1:22:0a:6e:00 10.85.175.2 fxp0.0 none d0:07:ca:57:d7:a0 10.85.175.3 fxp0.0 none 00:a0:a5:c2:06:e2 10.85.175.4 fxp0.0 none d8:18:d3:b3:6d:ea 10.108.120.2 xe-2/0/0.0 permanent 02:01:00:00:00:05 128.0.0.5 em1.0 none 02:01:00:00:00:05 128.0.0.6 em1.0 none 02:00:00:00:00:10 128.0.0.16 em0.0 none 02:00:00:00:00:12 128.0.0.18 em0.0 none 02:00:00:00:00:17 128.0.0.23 em0.0 none 02:00:00:00:00:1a 128.0.0.26 em0.0 none Total entries: 12
Meaning
The sample output is for Step 1 through Step 6 and shows that
a static ARP entry was configured on Gigabit Ethernet interface xe-2/0/0.0
.
Clear Gigabit Ethernet Interface Statistics
Purpose
You can reset the Gigabit Ethernet interface statistics. Resetting the statistics provides a clean start so that previous input/output errors and packet statistics do not interfere with the current diagnostics.
Action
To clear all statistics for the interface, use the following
Junos OS CLI operational mode command:
user@router> clear interfaces statistics (xe-fpc
/pic
/port
| et-fpc
/pic
/port
)
Sample Output
user@router> clear interfaces statistics xe-2/0/0 user@router>
Meaning
This command clears the interface statistics counters for the Gigabit Ethernet interface only.
Ping the Gigabit Ethernet Interface
Purpose
Use the ping command to verify the loopback connection.
Action
To send ping packets from the Ethernet interface, use the following Junos OS CLI operational mode command:
user@router> pingremote-IP-address
bypass-routing interface (xe-fpc
/pic
/port
| et-fpc
/pic
/port
) count 100 rapid
Sample Output
user@router> ping 10.108.120.2 bypass-routing interface xe-2/0/0 count 100 rapid ping 10.108.120.2 bypass-routing interface xe-2/0/0 count 100 rapid (snip) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 6a14 0 0000 36 01 15ba 10.108.120.1 10.108.120.2 36 bytes from 10.108.120.1: Redirect Host(New addr: 10.108.120.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 6a14 0 0000 35 01 16ba 10.108.120.1 10.108.120.2 36 bytes from 10.108.120.1: Time to live exceeded Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 6a14 0 0000 01 01 4aba 10.108.120.1 10.108.120.2 .36 bytes from 10.108.120.1: Time to live exceeded Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 6a76 0 0000 01 01 4a58 10.108.120.1 10.108.120.2 .36 bytes from 10.108.120.1: Time to live exceeded Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 6aa7 0 0000 01 01 4a27 10.108.120.1 10.108.120.2 .36 bytes from 10.108.120.1: Time to live exceeded Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 6ae3 0 0000 01 01 49eb 10.108.120.1 10.108.120.2 .36 bytes from 10.108.120.1: Time to live exceeded Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 6b0a 0 0000 01 01 49c4 10.108.120.1 10.108.120.2 .36 bytes from 10.108.120.1: Time to live exceeded
Meaning
The sample output shows that the time to live (TTL) expired, indicating that the link is receiving the frames from the ping test. The MAC address used is the same as the physical address of the port being tested because this allows the port to accept the frames from the ping test. As the packet is looped over the link, you expect to receive a TLL exceeded message for each ping sent. These messages are generated because the ping packets are repeatedly looped between the router and the physical loopback. When the packet is sent to the other end of the link, which does not exist, the loopback returns the packet back to the same interface, where it is again subjected to the Packet Forwarding Engine fabric for routing. After the route lookup, the TTL is decremented, and the packet is again sent out of the looped interface. This process repeats until the packed is either lost, or the TLL expires with subsequent TTL expired message displayed. Should any errors occur, the packet is discarded and a time-out error is displayed, rather than the expected TTL expired message. Note that the default TTL for ICMP echo packets in Junos OS is 64. This means a given test packet must be successfully sent and received 63 times before a TTL expired message can be generated. You can alter the TTL value to adjust the tolerance for loss, for example, a value of 255 is the most demanding test because now the packet must be sent and received error free 254 times.
Check for Gigabit Ethernet Interface Error Statistics
Purpose
Persistent interface error statistics indicate that you need to open a case with the Juniper Networks Technical Assistance Center (JTAC).
Action
To check the local interface for error statistics, use the following Junos OS CLI operational mode command:
user@router> (xe-fpc
/pic
/port
| et-fpc
/pic
/port
) extensive
Sample Output
user@router> show interfaces xe-2/0/0 extensive Physical interface: xe-2/0/0, Enabled, Physical link is Up Interface index: 187, SNMP ifIndex: 591, Generation: 190 Link-level type: Ethernet, MTU: 1514, MRU: 1522, LAN-PHY mode, Speed: 10Gbps, BPDU Error: None, Loop Detect PDU Error: None, MAC-REWRITE Error: None, Loopback: Local, Source filtering: Disabled, Flow control: Enabled, Speed Configuration: Auto Pad to minimum frame size: Disabled Device flags : Present Running Loop-Detected Interface flags: SNMP-Traps Internal: 0x4000 Link flags : None CoS queues : 8 supported, 8 maximum usable queues Schedulers : 0 Hold-times : Up 4000 ms, Down 0 ms Damping : half-life: 0 sec, max-suppress: 0 sec, reuse: 0, suppress: 0, state: unsuppressed Current address: d8:18:d3:b3:6d:ea, Hardware address: d8:18:d3:b3:6d:ea Last flapped : 2019-04-20 17:13:55 PDT (13w4d 21:40 ago) Statistics last cleared: 2019-07-25 14:49:21 PDT (00:04:41 ago) Traffic statistics: Input bytes : 537600 0 bps Output bytes : 539600 0 bps Input packets: 6400 0 pps Output packets: 6400 0 pps IPv6 transit statistics: Input bytes : 0 Output bytes : 0 Input packets: 0 Output packets: 0 Dropped traffic statistics due to STP State: Input bytes : 0 Output bytes : 0 Input packets: 0 Output packets: 0 Input errors: Errors: 0, Drops: 0, Framing errors: 0, Runts: 0, Policed discards: 0, L3 incompletes: 0, L2 channel errors: 0, L2 mismatch timeouts: 0, FIFO errors: 0, Resource errors: 0 Output errors: Carrier transitions: 0, Errors: 0, Drops: 0, Collisions: 0, Aged packets: 0, FIFO errors: 0, HS link CRC errors: 0, MTU errors: 0, Resource errors: 0 Egress queues: 8 supported, 4 in use Queue counters: Queued packets Transmitted packets Dropped packets 0 6400 6400 0 1 0 0 0 2 0 0 0 3 0 0 0 Queue number: Mapped forwarding classes 0 best-effort 1 expedited-forwarding 2 assured-forwarding 3 network-control Active alarms : None Active defects : None PCS statistics Seconds Bit errors 0 Errored blocks 0 MAC statistics: Receive Transmit Total octets 652800 652800 Total packets 6400 6400 Unicast packets 6400 6400 Broadcast packets 0 0 Multicast packets 0 0 CRC/Align errors 0 0 FIFO errors 0 0 MAC control frames 0 0 MAC pause frames 0 0 Oversized frames 0 Jabber frames 0 Fragment frames 0 VLAN tagged frames 0 Code violations 0 Total errors 0 0 Filter statistics: Input packet count 6400 Input packet rejects 0 Input DA rejects 0 Input SA rejects 0 Output packet count 6400 Output packet pad count 0 Output packet error count 0 CAM destination filters: 0, CAM source filters: 0 Packet Forwarding Engine configuration: Destination slot: 0 (0x00) CoS information: Direction : Output CoS transmit queue Bandwidth Buffer Priority Limit % bps % usec 0 best-effort 95 9500000000 95 0 low none 3 network-control 5 500000000 5 0 low none Preclassifier statistics: Traffic Class Received Packets Transmitted Packets Dropped Packets real-time 0 0 0 network-control 6400 6400 0 best-effort 0 0 0 Link Degrade : Link Monitoring : Disable Interface transmit statistics: Disabled Logical interface xe-2/0/0.0 (Index 353) (SNMP ifIndex 599) (Generation 175) Flags: Up SNMP-Traps 0x4004000 Encapsulation: ENET2 Traffic statistics: Input bytes : 537600 Output bytes : 539000 Input packets: 6400 Output packets: 6400 Local statistics: Input bytes : 0 Output bytes : 9800 Input packets: 0 Output packets: 100 Transit statistics: Input bytes : 537600 0 bps Output bytes : 529200 0 bps Input packets: 6400 0 pps Output packets: 6300 0 pps Protocol inet, MTU: 1500 Max nh cache: 75000, New hold nh limit: 75000, Curr nh cnt: 1, Curr new hold cnt: 0, NH drop cnt: 0 Generation: 206, Route table: 0 Flags: Sendbcast-pkt-to-re Addresses, Flags: Is-Preferred Is-Primary Destination: 10.108.120.0/30, Local: 10.108.120.1, Broadcast: 10.108.120.3, Generation: 146 Protocol multiservice, MTU: Unlimited, Generation: 207, Route table: 0 Policer: Input: __default_arp_policer__
Meaning
Check for any error statistics. There should not be
any input or output errors. If there are any persistent input or output
errors, open a case with the Juniper Networks Technical Assistance
Center (JTAC) at support@juniper.net
, or at 1-888-314-JTAC
(within the United States) or 1-408-745-9500 (from outside the United
States).
Diagnose a Suspected Circuit Problem
Purpose
When you suspect a circuit problem, it is important to work with the transport-layer engineer to resolve the problem. The transport-layer engineer may create a loop to the router from various points in the network. You can then perform tests to verify the connection from the router to that loopback in the network.
Action
After the transport-layer engineer has created the loop to the router from the network, you must verify the connection from the router to the loopback in the network. Follow Step 2 through Step 8 in Diagnose a Suspected Hardware Problem with a Gigabit Ethernet Interface. Keep in mind that any problems encountered in the test indicate a problem with the connection from the router to the loopback in the network.
By performing tests to loopbacks at various points in the network, you can isolate the source of the problem.
This document is applicable for 1Gb, 10Gb, 40Gb, and 100Gb interfaces.