Validate BGP Unnumbered EVPN Fabric
SUMMARY Follow these steps to verify the unnumbered underlay with automatic BGP peering is working properly.
Use these commands and sample output to confirm the proper operation of your unnumbered underlay. The configuration and operation of all nodes are similar. Below we only show commands and output for leaf 1. The same commands apply to all nodes, and similar output is expected for all devices.
Overall, the main verification task is to confirm all nodes have expected BGP sessions established using link-local addresses, and that the fabric devices are properly exchanging loopback routes. We take a structured bottom-up approach that confirms each aspect needed to successfully establish BGP sessions.
Validate Fabric Interfaces
Purpose
Confirm fabric interfaces are up and operational.
Action
user@leaf1> show interfaces xe-0/0/0Physical interface: xe-0/0/0, Enabled, Physical link is Up
Interface index: 650, SNMP ifIndex: 516
Link-level type: Ethernet, MTU: 1514, LAN-PHY mode, Speed: 10Gbps, Duplex: Full-Duplex, BPDU Error: None,
Loop Detect PDU Error: None, Ethernet-Switching Error: None, MAC-REWRITE Error: None, Loopback: Disabled,
Source filtering: Disabled, Flow control: Disabled, Media type: Fiber
Device flags : Present Running
Interface flags: SNMP-Traps Internal: 0x4000
Link flags : None
CoS queues : 8 supported, 8 maximum usable queues
Current address: 02:05:86:24:88:03, Hardware address: 02:05:86:24:88:03
Last flapped : 2022-03-14 10:56:00 PDT (6d 23:47 ago)
Input rate : 0 bps (0 pps)
Output rate : 0 bps (0 pps)
Active alarms : None
Active defects : None
PCS statistics Seconds
Bit errors 0
Errored blocks 0
Ethernet FEC statistics Errors
FEC Corrected Errors 0
FEC Uncorrected Errors 0
FEC Corrected Errors Rate 0
FEC Uncorrected Errors Rate 0
Interface transmit statistics: Disabled
Logical interface xe-0/0/0.0 (Index 555) (SNMP ifIndex 540)
Flags: Up SNMP-Traps 0x4004000 Encapsulation: ENET2
Input packets : 47384
Output packets: 48579
Protocol inet, MTU: 1500
Max nh cache: 75000, New hold nh limit: 75000, Curr nh cnt: 0, Curr new hold cnt: 0, NH drop cnt: 0
Flags: Sendbcast-pkt-to-re
Protocol inet6, MTU: 1500
Max nh cache: 75000, New hold nh limit: 75000, Curr nh cnt: 1, Curr new hold cnt: 0, NH drop cnt: 0
Flags: Is-Primary
Addresses, Flags: Is-Preferred
Destination: fe80::/64, Local: fe80::205:86ff:fe24:8803user@leaf1> show interfaces terse
Interface Admin Link Proto Local Remote
gr-0/0/0 up up
pfe-0/0/0 up up
pfe-0/0/0.16383 up up inet
inet6
pfh-0/0/0 up up
pfh-0/0/0.16383 up up inet
pfh-0/0/0.16384 up up inet
xe-0/0/0.0 up up inet
inet6 fe80::205:86ff:fe24:8803/64
xe-0/0/1 up up
xe-0/0/1.0 up up inet
inet6 fe80::205:86ff:fe24:8807/64
xe-0/0/2 up up
xe-0/0/2.16386 up up
. . .
jsrv up up
jsrv.1 up up inet 128.0.0.127/2
lo0 up up
lo0.0 up up inet
inet6 2001:db8:100::3
fe80::205:860f:fcc1:6f00
lo0.16385 up up inet
lsi up up
mtun up up
pimd up up
pime up up
pip0 up up
tap up up
vme up down
vtep up up
Meaning
The output shows that the leaf 1's fabric interfaces are operational. Also
seen is the lack of an explicit IPv4 or IPv6 address. Only the
inet and inet6 families are configured
on the interface. As a result, only IPv6 link-local IPv6 addresses are
present as part of IPv6 stateless address configuration. We also note the
loopback address has the expected IPv6 address assignment.
The output of
the
show
interfaces terse
CLI
command makes it easy to verify the state and
configuration of all interfaces, including the loopback address.
Validate Router Advertisements
Purpose
Confirm all fabric interfaces are sending and receiving IPv6 router advertisements (RAs).
Action
user@leaf1> show ipv6 router-advertisement
Interface: xe-0/0/0.0
Advertisements sent: 1512, last sent 00:01:05 ago
Solicits sent: 1, last sent 6d 23:49:07 ago
Solicits received: 3, last received 4d 18:18:57 ago
Advertisements received: 503
Solicited router advertisement unicast: Disable
IPv6 RA Preference: DEFAULT/MEDIUM
Advertisement from fe80::205:86ff:fe0c:dd03, heard 4d 18:15:17 ago
Managed: 0
Other configuration: 0
Reachable time: 0 ms
Default lifetime: 0 sec
Retransmit timer: 0 ms
Current hop limit: 64
Interface: xe-0/0/1.0
Advertisements sent: 1523, last sent 00:02:05 ago
Solicits sent: 1, last sent 6d 23:49:07 ago
Solicits received: 0
Advertisements received: 1515
Solicited router advertisement unicast: Disable
IPv6 RA Preference: DEFAULT/MEDIUM
Advertisement from fe80::205:86ff:fec6:b503, heard 00:03:47 ago
Managed: 0
Other configuration: 0
Reachable time: 0 ms
Default lifetime: 1800 sec
Retransmit timer: 0 ms
Current hop limit: 64
Meaning
The output confirms that leaf 1 is sending, and receiving RAs, to and from both spine devices. The RAs correctly report the sending end's link-local address.
Verify IPv6 Neighbor Discovery
Purpose
Confirm the fabric devices have learned the MAC to link-local address bindings of all directly attached IPv6 neighbors using IPv6 ND.
Action
user@leaf1> show ipv6 neighbors IPv6 Address Linklayer Address State Exp Rtr Secure Interface fe80::205:86ff:fe0c:dd03 02:05:86:0c:dd:03 reachable 19 yes no xe-0/0/0.0 fe80::205:86ff:fec6:b503 02:05:86:c6:b5:03 reachable 34 yes no xe-0/0/1.0 Total entries: 2
Meaning
The output confirms that leaf 1 has successfully resolved the MAC to link-local address of both its neighbors, that is, the two spine devices.
Verify Link-Local Connectivity
Purpose
Confirm that you can ping a neighbor using its link-local address.
Because all IPv6 link-locals share the same fe80::/64 prefix you must disambiguate a link-local ping by qualifying it with the corresponding egress interface.
Action
user@leaf1> ping fe80::205:86ff:fe0c:dd03 interface xe-0/0/0 PING6(56=40+8+8 bytes) fe80::205:86ff:fec1:6f03 --> fe80::205:86ff:fe0c:dd03 16 bytes from fe80::205:86ff:fe0c:dd03, icmp_seq=0 hlim=64 time=117.229 ms 16 bytes from fe80::205:86ff:fe0c:dd03, icmp_seq=1 hlim=64 time=114.074 ms ^C --- fe80::205:86ff:fe0c:dd03 ping6 statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max/std-dev = 114.074/115.651/117.229/1.577 ms
Meaning
The ping from leaf 1 to spine 1 succeeds. This result confirms IPv6 connectivity using link-local addresses.
Verify BGP Peering
Purpose
Confirm that all fabric devices have established BGP peering sessions to the directly connected neighbors.
Action
user@leaf1> show bgp summary
Threading mode: BGP I/O
Default eBGP mode: advertise - accept, receive - accept
Groups: 1 Peers: 2 Down peers: 0
Auto-discovered peers: 2
Table Tot Paths Act Paths Suppressed History Damp State Pending
inet6.0
4 4 0 0 0 0
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
fe80::205:86ff:fe0c:dd03%xe-0/0/0.0 65001 6 5 0 1 50 Establ
inet6.0: 2/2/2/0
fe80::205:86ff:fec6:b503%xe-0/0/1.0 65002 6 6 0 1 51 Establ
inet6.0: 2/2/2/0Meaning
As expected, the output shows that the leaf 1 device has two BGP sessions established. The display confirms that IPv6 routes are exchanged. More on that later. If desired, show details about a BGP neighbor:
user@leaf1> show bgp neighbor
Peer: fe80::205:86ff:fe0c:dd03%xe-0/0/0.0+56258 AS 65001 Local: fe80::205:86ff:fec1:6f03%xe-0/0/0.0+179 AS 65003
Group: auto-disc Routing-Instance: master
Forwarding routing-instance: master
Type: External State: Established Flags: <Sync PeerAsList AutoDiscoveredNdp>
Last State: OpenConfirm Last Event: RecvKeepAlive
Last Error: None
Export: [ DIRECT-RTS ]
Options: <AddressFamily Multipath LocalAS Refresh>
Options: <MultipathAs>
Options: <GracefulShutdownRcv>
Address families configured: inet6-unicast
Holdtime: 90 Preference: 170
Graceful Shutdown Receiver local-preference: 0
Local AS: 65003 Local System AS: 0
Number of flaps: 1
Last flap event: TransportError
Peer ID: 10.0.0.1 Local ID: 10.0.0.3 Active Holdtime: 90
Keepalive Interval: 30 Group index: 0 Peer index: 1 SNMP index: 2
I/O Session Thread: bgpio-0 State: Enabled
BFD: disabled, down
Local Interface: xe-0/0/0.0
NLRI for restart configured on peer: inet6-unicast
NLRI advertised by peer: inet6-unicast
NLRI for this session: inet6-unicast
Peer supports Refresh capability (2)
Stale routes from peer are kept for: 300
Peer does not support Restarter functionality
Restart flag received from the peer: Notification
NLRI that restart is negotiated for: inet6-unicast
NLRI of received end-of-rib markers: inet6-unicast
NLRI of all end-of-rib markers sent: inet6-unicast
Peer does not support LLGR Restarter functionality
Peer supports 4 byte AS extension (peer-as 65001)
Peer does not support Addpath
NLRI(s) enabled for color nexthop resolution: inet6-unicast
Table inet6.0 Bit: 20000
RIB State: BGP restart is complete
Send state: in sync
Active prefixes: 2
Received prefixes: 2
Accepted prefixes: 2
Suppressed due to damping: 0
Advertised prefixes: 2
Last traffic (seconds): Received 1 Sent 27 Checked 56
Input messages: Total 7 Updates 3 Refreshes 0 Octets 342
Output messages: Total 5 Updates 2 Refreshes 0 Octets 260
Output Queue[1]: 0 (inet6.0, inet6-unicast)
. . .
Verify BGP Route Exchange
Purpose
Confirm that all nodes are advertising their loopback address while learning the loopback address of other nodes.
Action
user@leaf1> show route protocol bgp
inet.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
inet6.0: 8 destinations, 11 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
2001:db8:100::1/128*[BGP/170] 00:33:43, localpref 100
AS path: 65001 I, validation-state: unverified
> to fe80::205:86ff:fe0c:dd03 via xe-0/0/0.0
[BGP/170] 00:33:46, localpref 100
AS path: 65002 65004 65001 I, validation-state: unverified
> to fe80::205:86ff:fec6:b503 via xe-0/0/1.0
2001:db8:100::2/128*[BGP/170] 00:33:46, localpref 100
AS path: 65002 I, validation-state: unverified
> to fe80::205:86ff:fec6:b503 via xe-0/0/1.0
[BGP/170] 00:33:43, localpref 100
AS path: 65001 65004 65002 I, validation-state: unverified
> to fe80::205:86ff:fe0c:dd03 via xe-0/0/0.0
2001:db8:100::4/128*[BGP/170] 00:33:46, localpref 100
AS path: 65002 65004 I, validation-state: unverified
> to fe80::205:86ff:fec6:b503 via xe-0/0/1.0
[BGP/170] 00:33:43, localpref 100
AS path: 65001 65004 I, validation-state: unverified
> to fe80::205:86ff:fe0c:dd03 via xe-0/0/0.0
Meaning
As expected, the output confirms that leaf 1 has learned the loopback addresses of all other fabric devices. Note that for leaf 2, it shows two equal costs paths. One path through spine 1, the other through spine 2.
If needed, you can help isolate routing problems by displaying the specific routes advertised to, or received from, a given BGP neighbor. Note that as with a ping to a link-local address, you must qualify the peer's link-local IP with the corresponding egress interface:
user@leaf1> show route advertising-protocol bgp fe80::205:86ff:fe0c:dd03%xe-0/0/0.0 inet6.0: 8 destinations, 10 routes (8 active, 0 holddown, 0 hidden) Prefix Nexthop MED Lclpref AS path * 2001:db8:100::2/128 Self 65002 I * 2001:db8:100::3/128 Self I user@leaf1> show route receive-protocol bgp fe80::205:86ff:fe0c:dd03%xe-0/0/0.0 inet.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden) inet6.0: 8 destinations, 10 routes (8 active, 0 holddown, 0 hidden) Prefix Nexthop MED Lclpref AS path * 2001:db8:100::1/128 fe80::205:86ff:fe0c:dd03 65001 I * 2001:db8:100::4/128 fe80::205:86ff:fe0c:dd03 65001 65004 I
Verify ECMP Load Balancing
Purpose
Confirm that the fabric supports ECMP load balancing. Display details about the routes leaf 1 use to reach leaf 2. You expect two equal costs routes, one through each spine device. In the below section you confirm ECMP in both the routing and forwarding tables (RIB and FIB).
Action
user@leaf1> show route 2001:db8:100::4 detail
inet6.0: 8 destinations, 11 routes (8 active, 0 holddown, 0 hidden)
2001:db8:100::4/128 (2 entries, 1 announced)
*BGP Preference: 170/-101
Next hop type: Router, Next hop index: 0
Address: 0xd057cc8
Next-hop reference count: 2
Source: fe80::205:86ff:fe0c:dd03%xe-0/0/0.0
Next hop: fe80::205:86ff:fe0c:dd03 via xe-0/0/0.0, selected
Session Id: 0
Next hop: fe80::205:86ff:fec6:b503 via xe-0/0/1.0
Session Id: 0
State: <Active Ext>
Peer AS: 65001
Age: 1:39:21
Validation State: unverified
Task: BGP_0_65003.fe80::205:86ff:fe0c:dd03
Announcement bits (3): 0-KRT 1-BGP_Multi_Path 2-BGP_RT_Background
AS path: 65001 65004 I
Accepted Multipath
Localpref: 100
Router ID: 10.0.0.1
Thread: junos-main
BGP Preference: 170/-101
Next hop type: Router, Next hop index: 1731
Address: 0xd22cc50
Next-hop reference count: 5
Source: fe80::205:86ff:fec6:b503%xe-0/0/1.0
Next hop: fe80::205:86ff:fec6:b503 via xe-0/0/1.0, selected
Session Id: 321
State: <Ext>
Inactive reason: Active preferred
Peer AS: 65002
Age: 1:39:21
Validation State: unverified
Task: BGP_0_65003.fe80::205:86ff:fec6:b503
AS path: 65002 65004 I
Accepted MultipathContrib
Localpref: 100
Router ID: 10.0.0.2
Thread: junos-maiuser@leaf1> show route forwarding-table destination 2001:db8:100::4
Routing table: default.inet6
Internet6:
Destination Type RtRef Next hop Type Index NhRef Netif
2001:db8:100::4/128 user 0 ulst 131070 2
fe80::205:86ff:fe0c:dd03
ucst 1730 6 xe-0/0/0.0
fe80::205:86ff:fec6:b503
ucst 1731 6 xe-0/0/1.0Meaning
The output confirms that both paths between leaf 1 and leaf 2 are actively used to forward traffic and are part of a multipath route.
Verify Fabric Forwarding
Purpose
Confirm that leaf 1 and leaf 2 have connectivity over the underlay.
Action
user@leaf1> traceroute no-resolve 2001:db8:100::4 traceroute6 to 2001:db8:100::4 (2001:db8:100::4) from 2001:db8:100::3, 64 hops max, 12 byte packets 1 2001:db8:100::1 220.185 ms 210.200 ms 203.652 ms 2 2001:db8:100::4 213.774 ms 246.773 ms 186.533 ms
Meaning
The output confirms that leaf 1 and leaf 2 have underlay connectivity. The path shows that this flow went through spine 1, as indicated by its loopback address as the first hop.