EVPN-VXLAN Type 5 Unequal Load Balancing
EVPN-VXLAN Type 5 Unequal Load Balancing uses the EVPN Link Bandwidth Extended Community to advertise Layer 3 link bandwidth and dynamically balance traffic across multihomed devices.
The EVPN-VXLAN Type 5 unequal load-balancing feature optimizes traffic distribution across multihomed devices by leveraging the EVPN Link Bandwidth Extended Community to advertise and use bandwidth information from Layer 3 links. This functionality enables bandwidth-based load balancing across multihomed devices by dynamically adjusting traffic distribution based on available bandwidth, ensuring efficient load management, especially during link failures. You can configure OSPF, IS-IS, and BGP protocols to integrate this bandwidth data, enhancing route management and inter-data center communications. The EVPN Link Bandwidth Extended Community enables efficient bandwidth data exchange, using megabits per second (Mbps) as the measurement unit. This mechanism is essential for networks with variable link bandwidth, ensuring efficient and reliable traffic flow across various platforms. In the event of inconsistent bandwidth data, the system defaults to equal-cost multipath (ECMP) routing to maintain network stability.
Benefits of Unequal Load Balancing
-
Improves traffic distribution by utilizing real-time bandwidth information, reducing congestion and enhancing overall network performance.
-
Enables unequal load balancing across multihomed devices, ensuring more efficient use of available network resources.
-
Enhances inter-data center communications through aggregate bandwidth calculations, facilitating smoother and more reliable data exchanges.
-
Provides robust traffic management by integrating bandwidth data from OSPF, IS-IS, and BGP learned routes, offering better route optimization.
-
Alerts users to bandwidth inconsistencies via system logs, allowing for proactive monitoring and troubleshooting to maintain optimal network operations.
Overview of Unequal Load Balancing for EVPN-VXLAN Type 5 Routing
Unequal load balancing in EVPN-VXLAN Type 5 overlays leverages the EVPN Link Bandwidth Extended Community to distribute network traffic based on the actual bandwidth of Layer 3 links. When you configure this feature your network can prioritize paths with higher bandwidth capacity, optimizing traffic flow and reducing congestion. The feature dynamically balances traffic by advertising bandwidth information, which remote PE devices use to calculate precise traffic balance factors for next-hop routes. This capability is crucial for maintaining optimal performance in complex, multihomed network scenarios.
We support OSPF, IS-IS, and BGP protocols in unequal load balancing to ensure that bandwidth-based traffic management is seamless across various network architectures. You can achieve comprehensive bandwidth distribution across data centers by aggregating bandwidth information from multiple multihomed leaf devices and propagating it using Type 5 routes. Syslog notifications alert you to any inconsistencies or missing bandwidth information for timely troubleshooting and to ensure the reliability of your network's traffic management.
You configure unequal load balancing by applying bandwidth policy statements and monitoring the results, as follows:
-
Use policy statements such as
set policy-options policy-statement name then aggregate-bandwidth. -
Export the policies using
set routing-instances name protocols evpn ip-prefix-routes export name. -
Verify the distribution and balance of traffic across multihomed devices using the
show evpn ip-prefix-database extensivecommand.
These policies facilitate advertising the bandwidth details in EVPN Type 5 routes, including the EVPN Link Bandwidth Extended Community, which is essential for the remote PE devices to perform accurate load balancing.
EVPN Link Bandwidth Extended Community
The EVPN Link Bandwidth Extended Community enables unequal-cost multipath (UCMP) or weighted equal-cost multipath (WECMP) load balancing for EVPN Type 5 routes by advertising the per-link capacity associated with each route. With bandwidth information attached to the route, the remote PE device can distribute traffic proportionally across EVPN-VXLAN next hops that have different capacities, improving load sharing and reducing congestion. The EVPN Link Bandwidth Extended Community is transitive, so the bandwidth signal propagates across the EVPN domain.
Each EVPN Type 5 route carries the EVPN Link Bandwidth Extended Community. The
community’s format defines two value indicators: Value-Units and
Value-Weight. The Value-Units field indicates whether
Value-Weight represents Mbps or a generalized weight.
The Value-Units field is encoded as follows:
-
0x00—
Value-Weightis expressed in Mbps (default). -
0x01—
Value-Weightis a generalized weight expressed in units other than Mbps.
The Value-Weight is a 5-octet unsigned integer. The generalized weight
units are intentionally unspecified to allow flexibility across different applications.
Please see EVPN Link Bandwidth Extended Community for more
information.
The EVPN-VXLAN Type 5 unequal load-balancing feature supports the default
Value-Units, which expresses the Value-Weight in
Mbps.
By default, a remote PE device shares traffic equally across multihomed devices. If a Layer 3 (L3) CE-to-device link has lower capacity than the traffic delivered to it, equal sharing can overload that path. To avoid overloading a path, each multihomed device advertises the bandwidth of its CE-facing L3 link using the EVPN Link Bandwidth Extended Community. For Type 5 route-based UCMP, the remote PE device:
-
Associates each destination’s Type 5 route with its advertised bandwidth.
-
Computes a balance factor for each EVPN-VXLAN next hop.
-
Distributes traffic in proportion to the ratio of the CE-to-device link bandwidths.
If an L3 link between a multihomed device and the CE fails or its capacity changes, the remote PE device re-evaluates the advertised bandwidth ratios and adjusts forwarding toward the remaining paths, maintaining proportional distribution as the topology or link capacities evolve.
How EVPN-VXLAN Unequal Load Balancing Works
In this illustration we have a customer edge device CE 1, connected to two multihomed devices, Leaf 1 and Leaf 2.
-
The link between Leaf 1 and CE 1 has a bandwidth capacity of 3 flow units.
-
The link between Leaf 2 and CE 1 has a bandwidth capacity of 5 flow units.
The flow units here represent the aggregated capacity and could be a single or multiple physical links.
The multihomed devices use the EVPN Link Bandwidth Extended Community to advertise the available bandwidth in Type 5 routes to a route reflector (not shown).
The Remote PE router in the EVPN network connects to the multihomed devices via a VXLAN-encapsulated tunnel.
A route reflector (not shown) distributes the EVPN routes between the multihomed devices and the remote PE device.
In Figure 1, Leaf 1 advertises a link bandwidth capacity of 3 flows to the remote PE device, while Leaf 2 advertises a bandwidth capacity of 5 flows. As a result, the remote PE device maintains a unilist of next hops, which includes the VXLAN-encapsulated tunnels to Leaf 1 and Leaf 2, with a traffic distribution balance factor based on the 3:5 ratio of flows. This ratio reflects the relative link bandwidth capacities between the two multihomed devices, ensuring traffic is distributed accordingly.
The remote PE device ensures it receives the Type 5 route with the EVPN Link Bandwidth
Extended Community from all multihomed devices, and verifies the consistency of the
Value-Units fields. If the EVPN Link Bandwidth Extended Community is
missing from the Type 5 route of any multihomed device, or if there is an inconsistency in
the Value-Units, the remote PE device generates a syslog message and
follows regular ECMP forwarding to the multihomed devices.
After the remote PE device validates the consistency of the Value-Units
fields, the remote PE device:
-
Uses the
Value-Weightreceived from each multihomed device to compute a relative weight for each device. -
Uses that relative weight to compute a weighted path-list for load balancing instead of using an ECMP path-list.
Configure and Manage Unequal Load Balancing
To effectively manage and use bandwidth-based load balancing in EVPN-VXLAN Type 5 networks, ensure that the EVPN Link Bandwidth Extended Community is consistently applied across all multihomed devices. If discrepancies occur, the system defaults to standard ECMP forwarding, and generates system log messages to alert you of the issues.
You use CLI commands to set up bandwidth aggregation options, ensuring efficient
configuration and verification of load balancing settings. Begin by defining bandwidth
policies using the policy-statement configuration to aggregate bandwidth
from Layer 3 links and ensure these policies are correctly exported through EVPN Type 5
routes. You can also configure OSPF, IS-IS, and BGP protocols to integrate their bandwidth
data to enhance route management.
You use the show route and show evpn ip-prefix-database
extensive commands to verify the bandwidth distribution and see detailed
information about bandwidth allocation and balance factors. Check the system log messages
for missing or inconsistent bandwidth information that can trigger an ECMP fallback.
- Configure EVPN Link Bandwidth Policies
- Integrate OSPF Learned Bandwidth
- Integrate IS-IS Learned Bandwidth
- Integrate BGP Learned Bandwidth
- Remove BGP Link Bandwidth Extended Community Advertisements
- Link Bandwidth Aggregation Across Data Centers
Configure EVPN Link Bandwidth Policies
As part of the EVPN Type 5 route advertisement, each multihomed device uses an export
policy with the aggregate-bandwidth statement to calculate the available
bandwidth and advertise it using the EVPN Link Bandwidth Extended Community. The
multihomed devices advertise EVPN Type 5 routes to the Route Reflector which forwards
these routes to the remote PE device.
The following example shows a minimal export policy. You can customize this policy using
match conditions to specify protocol or route-filter
criteria.
set policy-options policy-statement policy-name then aggregate-bandwidth set policy-options policy-statement policy-name then accept set routing-instances instance-name protocols evpn ip-prefix-routes export policy-name
Integrate OSPF Learned Bandwidth
Use the OSPF spf-options multipath weighted one-hop configuration to
associate the gateway L3 link bandwidth with OSPF learned routes.
set protocols ospf spf-options multipath weighted one-hop
This configuration enables WECMP for load-sharing data based on the bandwidth for incoming traffic destined to OSPFv2 neighbors that are at a one-hop distance. This associates the bandwidth information with each gateway as shown in the outputs below.
user@device> show route 10.1.255.1 extensive expanded-nh
vrf1.inet.0: 11 destinations, 11 routes (11 active, 0 holddown, 0 hidden)
10.1.255.1/32 (1 entry, 1 announced)
Installed-nexthop:
In-kernel:
List (0x859cc14) Index:1048578
Router (0x7e5cc14) Index:624 10.0.0.1 Session-ID: 320 via ae0.0
Router (0x7e5c514) Index:635 10.0.1.1 Session-ID: 325 via ae1.0
Router (0x7e5c894) Index:636 10.2.2.1 Session-ID: 326 via ae2.0
TSI:
KRT in-kernel 10.1.255.1/32 -> {list:10.0.0.1, 10.0.1.1, 10.2.2.1}
*OSPF Preference: 10
Next hop type: Router, Next hop index: 0
Address: 0x859d314
Next-hop reference count: 2, Next-hop session id: 0
Kernel Table Id: 0
Next hop: 10.0.0.1 via ae0.0 balance 33% ------> Bandwidth value x
Session Id: 0
Next hop: 10.0.1.1 via ae1.0 balance 33% ------> Bandwidth value y
Session Id: 0
Next hop: 10.2.2.1 via ae2.0 balance 33%, selected ------> Bandwidth value z
Session Id: 0
When the above route is advertised as an EVPN Type 5 route, it aggregates the L3 gateway links bandwidth and adds it to the EVPN Link Bandwidth Extended Community as shown in the outputs below:
user@device> show route 10.1.255.1/32 extensive | match "bandwidth"
Communities: 9498:95 9498:100 34111:50112 evpn-bandwidth:0x00:(x+y+z)
Here the Value-Units (0x00) indicates that the Value-Weight (x+y+z) represents the aggregate bandwidth in Mbps [default].
x : Bandwidth of first gateway L3 link in Mbps.
y : Bandwidth of second gateway L3 link in Mbps.
z : Bandwidth of third gateway L3 link in Mbps.
Integrate IS-IS Learned Bandwidth
Use the IS-IS spf-options multipath weighted one-hop configuration to
associate the gateway L3 link bandwidth with IS-IS learned routes.
This configuration enables WECMP for load-sharing data based on the bandwidth for
incoming traffic destined to for IS-IS interfaces like the OSPF version does for OSPF. You
can use the same show route commands to view the IS-IS information as you
would for OSPF.
set protocols isis spf-options multipath weighted one-hop
Integrate BGP Learned Bandwidth
Use the BGP multipath and auto-sense configurations to
associate the gateway L3 link bandwidth with BGP learned routes.
set protocols bgp group group-name external multipath set protocols bgp group group-name external link-bandwidth auto-sense
The BGP multipath configuration enables load sharing among multiple EBGP
paths and multiple IBGP paths.
The auto-sense configuration enables auto-sensing of link bandwidth and
attaching the BGP Link Bandwidth community to a route at import.
You can configure automatic sensing of link bandwidth only on single-hop EBGP neighbors.
When more than one BGP link-bandwidth community is attached to the route, BGP ignores transitivity and installs the lowest link-bandwidth value to the forwarding table (FIB).
With the above configurations, you can view the bandwidth and the balance factor
associated with each gateway using the show route and show route
extensive expanded-nh commands.
user@device> show route 10.0.2.0
inet.0: 24 destinations, 46 routes (24 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
10.0.2.0/30 *[BGP/170] 00:00:58, localpref 100
AS path: 65537 I, validation-state: unverified
to 10.1.0.2 via ge-0/0/3.0
to 10.2.0.2 via ge-0/0/4.0
> to 10.0.1.1 via ge-0/0/1.0
to 10.0.0.2 via ge-0/0/2.0
[BGP/170] 00:00:58, localpref 100
AS path: 65537 I, validation-state: unverified
> to 10.0.0.2 via ge-0/0/2.0
[BGP/170] 00:00:58, localpref 100
AS path: 65537 I, validation-state: unverified
> to 10.2.0.2 via ge-0/0/4.0
[BGP/170] 00:00:58, localpref 100
AS path: 65537 I, validation-state: unverified
> to 10.1.0.2 via ge-0/0/3.0
user@device> show route 10.0.2.0 extensive expanded-nh
inet.0: 24 destinations, 46 routes (24 active, 0 holddown, 0 hidden)
10.0.2.0/30 (4 entries, 1 announced)
Installed-nexthop:
In-kernel:
List (0x859aa14) Index:1048574
Router (0x7e5e414) Index:604 10.1.0.2 Session-ID: 322 via ge-0/0/3.0
Router (0x7e5bb94) Index:605 10.2.0.2 Session-ID: 323 via ge-0/0/4.0
Router (0x7e5e694) Index:593 10.0.1.1 Session-ID: 320 via ge-0/0/1.0
Router (0x7e5bc14) Index:595 10.0.0.2 Session-ID: 321 via ge-0/0/2.0
TSI:
KRT in-kernel 10.0.2.0/30 -> {list:10.1.0.2, 10.2.0.2, 10.0.1.1, 10.0.0.2}
Multipath TSI
Flags: RTargetLBWSet
Lead Route: BGP, 10.0.1.1
Nexthop:
Refcnt: 2
Template: 0x8ce9208(Router, 4 legs)
Object: 0x8599414(Router, 4 legs)
Mode: Multipath
Page 0 idx 1, (group external2 type External) Type 1 val 0x11468fa0 (adv_entry)
Advertised metrics:
Nexthop: Self
AS path: [65536] 65537 I
Communities: bandwidth:65537:500000000
Advertise: 00000001
Path 10.0.2.0
from 10.0.1.1
Vector len 4. Val: 1
*BGP Preference: 170/-101
Next hop type: Router, Next hop index: 0
Address: 0x8599414
Next-hop reference count: 3, Next-hop session id: 0
Kernel Table Id: 0
Source: 10.0.1.1
Next hop: 10.1.0.2 via ge-0/0/3.0 balance 25% ------>Bandwidth value W
Session Id: 0
Next hop: 10.2.0.2 via ge-0/0/4.0 balance 25% ------> Bandwidth value X
Session Id: 0
Next hop: 10.0.1.1 via ge-0/0/1.0 balance 25%, selected ------> Bandwidth value Y
Session Id: 0
Next hop: 10.0.0.2 via ge-0/0/2.0 balance 25% ------> Bandwidth value Z
Session Id: 0
State: <Active Ext>
Local AS: 65536 Peer AS: 65537
Age: 1:36
Validation State: unverified
When the above route is advertised as an EVPN Type 5 route, it aggregates the L3 gateway links bandwidth and adds it to the EVPN Link Bandwidth Extended Community as shown in the outputs below:
user@device> show route 10.0.2.0/30 extensive | match "bandwidth"
Communities: 9498:95 9498:100 34111:50112 evpn-bandwidth:0x00:(W+X+Y+Z)
Here the Value-Units (0x00) indicates that the Value-Weight (W+X+Y+Z) represents the aggregate bandwidth in Mbps [default].
W : Bandwidth of first gateway L3 link in Mbps.
X : Bandwidth of second gateway L3 link in Mbps.
Y : Bandwidth of third gateway L3 link in Mbps.
Z : Bandwidth of fourth gateway L3 link in Mbps.
Remove BGP Link Bandwidth Extended Community Advertisements
The BGP multipath configuration uses the BGP Link Bandwidth extended
community to provide WECMP for unequal load balancing. When you configure an export policy
with the aggregate-bandwidth statement, EVPN adds the BGP Link Bandwidth
extended community to the Type 5 route advertisement. As a result, the Type 5 route
contains both the BGP Link Bandwidth extended community and the EVPN Link Bandwidth
Extended Community by default.
You can configure a policy to remove the BGP Link Bandwidth extended community from the Type 5 route advertisement if you don't want both advertised.
For example, the following policy adds the aggregate-bandwidth statement
to an EVPN type 5 configuration:
set policy-options policy-statement evpn_type5 term 1 from protocol bgp set policy-options policy-statement evpn_type5 term 1 from protocol ospf set policy-options policy-statement evpn_type5 term 1 from route-filter 10.1.0.0/16 orlonger set policy-options policy-statement evpn_type5 term 1 then aggregate-bandwidth set policy-options policy-statement evpn_type5 term 1 then accept set policy-options policy-statement evpn_type5 term 2 from protocol bgp set policy-options policy-statement evpn_type5 term 2 from protocol ospf set policy-options policy-statement evpn_type5 term 2 from route-filter 10.2.0.0/16 orlonger set policy-options policy-statement evpn_type5 term 2 then accept set policy-options policy-statement evpn_type5 term 3 then reject set policy-options community bgp-bw members bandwidth:*:*
In the output from the following show route extensive command, you can
see both the BGP Link Bandwidth extended community and the EVPN Link Bandwidth Extended
Community values were added to the route.
user@device> show route match-prefix 5:*10.1.0.0* extensive
inet.0: 20 destinations, 20 routes (20 active, 0 holddown, 0 hidden)
vrf1.inet.0: 6 destinations, 8 routes (6 active, 0 holddown, 0 hidden)
iso.0: 4 destinations, 4 routes (4 active, 0 holddown, 0 hidden)
inet6.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
bgp.evpn.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
5:10.255.1.2:100::0::10.1.0.0::16/248 (1 entry, 1 announced)
TSI:
Page 0 idx 0, (group overlay type External) Type 1 val 0x102b6500 (adv_entry)
Advertised metrics:
Flags: Nexthop Change
Nexthop: Self
AS path: [65200] 65100 I
Communities: target:100:100 bandwidth:65100:62500000 encapsulation:vxlan(0x8) evpn-bandwidth:0x00:600
Advertise: 00000003
Path 5:10.255.1.2:100::0::10.1.0.0::16
Vector len 4. Val: 0
*EVPN Preference: 170/-101
Next hop type: Fictitious, Next hop index: 0
Address: 0x825a814
Next-hop reference count: 4
Kernel Table Id: 0
Next hop:
State: <Secondary Active Int Ext>
Age: 6
Validation State: unverified
Localpref: 100
Task: vrf1-EVPN-L3-context
Announcement bits (1): 0-BGP_RT_Background
AS path: 65100 I
Communities: target:100:100 bandwidth:65100:62500000 encapsulation:vxlan(0x8) evpn-bandwidth:0x00:600
Route Label: 9100
Overlay gateway address: 0.0.0.0
ESI 00:00:00:00:00:00:00:00:00:00
Primary Routing Table: vrf1.evpn.0
Thread: junos-main
vrf1.evpn.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
5:10.255.1.2:100::0::10.1.0.0::16/248 (1 entry, 1 announced)
*EVPN Preference: 170/-101
Next hop type: Fictitious, Next hop index: 0
Address: 0x825a814
Next-hop reference count: 4
Kernel Table Id: 0
Next hop:
State: <Active Int Ext>
Age: 6
Validation State: unverified
Localpref: 100
Task: vrf1-EVPN-L3-context
Announcement bits (1): 1-rt-export
AS path: 65100 I
Communities: bandwidth:65100:62500000 encapsulation:vxlan(0x8) evpn-bandwidth:0x00:600
Route Label: 9100
Overlay gateway address: 0.0.0.0
ESI 00:00:00:00:00:00:00:00:00:00
Thread: junos-main
If you don't want both extended communities advertised with the Type 5 route, you can use a policy to remove the BGP Link Bandwidth extended community from the advertisement.
For example, the following policy removes the BGP Link Bandwidth extended community.
set policy-options policy-statement evpn_type5 term 1 from protocol bgp set policy-options policy-statement evpn_type5 term 1 from protocol ospf set policy-options policy-statement evpn_type5 term 1 from route-filter 10.1.0.0/16 orlonger set policy-options policy-statement evpn_type5 term 1 then community delete bgp-bw set policy-options policy-statement evpn_type5 term 1 then aggregate-bandwidth set policy-options policy-statement evpn_type5 term 1 then accept set policy-options policy-statement evpn_type5 term 2 from protocol bgp set policy-options policy-statement evpn_type5 term 2 from protocol ospf set policy-options policy-statement evpn_type5 term 2 from route-filter 10.2.0.0/16 orlonger set policy-options policy-statement evpn_type5 term 2 then accept set policy-options policy-statement evpn_type5 term 3 then reject set policy-options community bgp-bw members bandwidth:*:*
When you run the show route extensive command with the configuration
that removes the BGP Link Bandwidth extended community, you see only the EVPN
Link-Bandwidth Extended Community values in the route in the EVPN Type 5 route:
user@device> show route match-prefix 5:*10.1.0.0* extensive
inet.0: 20 destinations, 20 routes (20 active, 0 holddown, 0 hidden)
vrf1.inet.0: 6 destinations, 8 routes (6 active, 0 holddown, 0 hidden)
iso.0: 4 destinations, 4 routes (4 active, 0 holddown, 0 hidden)
inet6.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
bgp.evpn.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
5:10.255.1.2:100::0::10.1.0.0::16/248 (1 entry, 1 announced)
TSI:
Page 0 idx 0, (group overlay type External) Type 1 val 0x102b6500 (adv_entry)
Advertised metrics:
Flags: Nexthop Change
Nexthop: Self
AS path: [65200] 65100 I
Communities: target:100:100 encapsulation:vxlan(0x8) evpn-bandwidth:0x00:600
Advertise: 00000003
Path 5:10.255.1.2:100::0::10.1.0.0::16
Vector len 4. Val: 0
*EVPN Preference: 170/-101
Next hop type: Fictitious, Next hop index: 0
Address: 0x825a814
Next-hop reference count: 4
Kernel Table Id: 0
Next hop:
State: <Secondary Active Int Ext>
Age: 1:20
Validation State: unverified
Localpref: 100
Task: vrf1-EVPN-L3-context
Announcement bits (1): 0-BGP_RT_Background
AS path: 65100 I
Communities: target:100:100 encapsulation:vxlan(0x8) evpn-bandwidth:0x00:600
Route Label: 9100
Overlay gateway address: 0.0.0.0
ESI 00:00:00:00:00:00:00:00:00:00
Primary Routing Table: vrf1.evpn.0
Thread: junos-main
vrf1.evpn.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
5:10.255.1.2:100::0::10.1.0.0::16/248 (1 entry, 1 announced)
*EVPN Preference: 170/-101
Next hop type: Fictitious, Next hop index: 0
Address: 0x825a814
Next-hop reference count: 4
Kernel Table Id: 0
Next hop:
State: <Active Int Ext>
Age: 1:20
Validation State: unverified
Localpref: 100
Task: vrf1-EVPN-L3-context
Announcement bits (1): 1-rt-export
AS path: 65100 I
Communities: encapsulation:vxlan(0x8) evpn-bandwidth:0x00:600
Route Label: 9100
Overlay gateway address: 0.0.0.0
ESI 00:00:00:00:00:00:00:00:00:00
Thread: junos-main
Link Bandwidth Aggregation Across Data Centers
When a data center spine device receives a Type 5 route with the EVPN Link Bandwidth
Extended Community from a multihomed device, it installs the route into its
VRF.inet.x table. The spine device aggregates the
bandwidth received from each multihomed device and adds that to the EVPN Link Bandwidth
Extended Community when it advertises that route to another data center.
Verify and Troubleshoot Unequal Load Balancing
You can verify the bandwidth distribution with the show evpn ip-prefix-database
extensive command. This command provides detailed insights into bandwidth
allocation and balance factors. It will also indicate the Unequal ECMP status. For example,
in the first output below, the Unequal ecmp state: is
Enabled. However, in the second output, gateway 10.255.1.2 is not
advertising EVPN link bandwidth information, so Unequal ecmp state: is
Disabled. In this case the system defaults to ECMP routing to maintain
network stability.
The show evpn ip-prefix-database extensive outputs only display the
bandwidth information when the feature is enabled.
In the event of an error, such as a PE router not advertising bandwidth information, or
an inconsistent Value-Units in the EVPN Link Bandwidth Extended
Community, the output displays Unequal ecmp state: Disabled.
The outputs below illustrate what might be seen on the Remote PE device in Figure 1 when looking at the information received from Leaf 1 and Leaf 2.
This output shows the Remote PE receiving EVPN link bandwidth data from both Leaf 1 and Leaf 2.
user@device> show evpn ip-prefix-database direction imported prefix 100.1.0.0/16 extensive
L3 context: vrf1
EVPN->IPv4 Imported Prefixes
Prefix: 100.1.0.0/16, Ethernet tag: 0
Change flags: 0x0
State flags: 0x4
In Unknown Kernel ID queue: NO, In Underlay Change queue: NO, Unequal ecmp state: Enabled
Remote advertisements:
Route Distinguisher: 10.255.1.1:100
VNI: 9100
Router MAC: 2c:6b:f5:6a:e6:f0
BGP nexthop address: 10.255.1.1
IP route status: Created
Change flags: 0x0
In GW ADDR queue: NO
EVPN link bandwidth: Gateway-10.255.1.1 bandwidth=0x00:600, balance_factor=32768(50%) <<< Bandwidth information received from Leaf 1
Route Distinguisher: 10.255.1.2:100
VNI: 9100
Router MAC: 2c:6b:f5:6d:36:f0
BGP nexthop address: 10.255.1.2
IP route status: Created
Change flags: 0x0
In GW ADDR queue: NO
EVPN link bandwidth: Gateway-10.255.1.2 bandwidth=0x00:600, balance_factor=65535(50%) <<< Bandwidth information received from Leaf 2
This output shows the Remote PE receiving EVPN link bandwidth data from Leaf 1, but not receiving it from Leaf 2.
user@device> show evpn ip-prefix-database direction imported prefix 100.1.0.0/16 extensive
L3 context: vrf1
EVPN->IPv4 Imported Prefixes
Prefix: 100.1.0.0/16, Ethernet tag: 0
Change flags: 0x0
State flags: 0x4
In Unknown Kernel ID queue: NO, In Underlay Change queue: NO, Unequal ecmp state: Disabled
Remote advertisements:
Route Distinguisher: 10.255.1.1:100
VNI: 9100
Router MAC: 2c:6b:f5:6a:e6:f0
BGP nexthop address: 10.255.1.1
IP route status: Created
Change flags: 0x0
In GW ADDR queue: NO
EVPN link bandwidth: Gateway-10.255.1.1 bandwidth=0x00:600, balance_factor=32768(50%) <<< Bandwidth information received from leaf-1
Route Distinguisher: 10.255.1.2:100
VNI: 9100
Router MAC: 2c:6b:f5:6d:36:f0
BGP nexthop address: 10.255.1.2
IP route status: Created
Change flags: 0x0
In GW ADDR queue: NO
<<< Gateway 10.255.1.2 (leaf-2) is not advertising EVPN link bandwidth information. So Unequal ECMP is Disabled.
To help troubleshoot this feature, check the syslog for EVPN_WECMP
messages. These messages alert you to missing or inconsistent bandwidth information that
will trigger an ECMP fallback. For example, the following message alerts you to an ECMP
fallback due to a PE router not advertising the EVPN Link Bandwidth Extended Community. In
this case you also run the show evpn ip-prefix-database extensive command
(see the command output above) to find more information about unequal load balancing
issues.
user@device> show log messages | grep "EVPN_WECMP" Mar 1 15:39:59 device rpd[18506]: EVPN_WECMP: Ignoring the weighted load balance because one of the PEs is not advertised with the EVPN-bandwidth community.