Recommendations

Bridged Overlay Fabric and centralized Layer3 routing using Offramp Gateway

This validated design recommends deploying an IPv6-based EVPN-VXLAN Bridged Overlay (BO) fabric with centralized Layer 3 routing on the Off-Ramp Gateway. The Border Leaf switches are configured with Anycast IRB gateways, while the server-facing leaf switches operate as Layer 2 bridges within the EVPN-VXLAN fabric.

For JCNR services, the Border Leaf switches terminate the VXLAN overlay and forward MPLS-over-UDP traffic towards the Off-Ramp Gateway for inter-subnet, inter-VRF, and inter-tenant routing. For SR-IOV services, traffic is transported across the fabric using Q-in-Q encapsulation. Since Q-in-Q services cannot be terminated directly on the Border Leaf IRB interfaces, the service termination is performed on the Off-Ramp Gateway. The Offramp gateway is configured with “physical loopback” construct where two interfaces are connected to each other.

The approach minimizes changes to the standard Apstra-managed data center fabric, simplifies operations, and provides a consistent framework for supporting JCNR MPLS-over-UDP services and SR-IOV Q-in-Q services within a multi-tenant environment.

For this solution, the validated design recommends HPE Juniper Networking switches QFX5130-48C, QFX5130-32CD as leaf and border leaf switches respectively, and QFX5230-64CD as lean spine. PTX10002-36QDD is recommended as an offramp-gateway router.

The Junos OS release validated and recommended for this release is 25.2X100-D30. For JCNR and PTX10002-36QDD, the release recommended is 25.4.

New features Spported in Apstra 6.2 for Service Provider Fabric provisioning

With the release of Apstra Data Center Director 6.2, an intent-based Service Provider Data Center Fabric solution can be deployed and managed easily. The solution supports critical new features such as Q-in-Q packet encapsulation and overlapping VLAN with multiple EVPN instance. The intent-based feature of Apstra enables monitoring of the fabric. However, these features are not limited to Service Provider Telco Cloud solution but can be extended to any data center deployment that requires these capabilities, for instance enterprise and large-scale data center deployments that require multi-tenancy, workload isolation, flexible Layer 2 and Layer 3 services, high availability.

The solution recommends MPLS-over-UDP as the JCNR overlay transport to avoid VNI collisions with the EVPN-VXLAN fabric and to maintain a clear separation between fabric and workload overlays. This approach eliminates the need for Kubernetes hosts to operate as VTEPs, allowing Apstra to retain full management of the fabric overlay.

Virtual Route Reflector for Minimizing Mesh of BGP Connections from JCNR

JCNR should be deployed as the cloud-native networking platform for Kubernetes workloads. A Virtual Route Reflector (vRR) is recommended for scalable BGP route distribution and tenant connectivity. Without a vRR, every JCNR instance would need to establish direct BGP peering sessions with other JCNRs or routing devices, resulting in a complex and difficult-to-manage mesh of BGP connections. As the number of JCNR nodes grow, the number of BGP sessions increases significantly.

Q-in-Q Encapsulation for SR-IOV and Apstra Support

For SR-IOV workloads, the Customer VLAN (C-VLAN) should be configured on the BareMetal servers, virtual machines, or Kubernetes pods hosting the workload. A Service VLAN (S-VLAN) is then used to provide Q-in-Q encapsulation, allowing multiple customer VLANs to be transported as a tenant-specific service across the fabric. In Apstra 6.2, the S-VLAN should be mapped to a corresponding VXLAN Network Identifier (VNI), enabling the EVPN-VXLAN fabric to transport SR-IOV traffic while preserving tenant isolation.

Kubernetes Deployment with Multiple Control Plane

The solution recommends deploying Kubernetes in a highly available configuration with multiple control-plane nodes to eliminate single points of failure and improve cluster resiliency. Additional control-plane nodes can be added by updating the Kubespray inventory and redeploying the cluster configuration. Where high availability is required, each control-plane node should also participate in the etcd cluster to ensure redundancy of the Kubernetes control plane and cluster state database. An odd number of etcd members (for example, three or five) is recommended to maintain quorum and avoid split-brain conditions. Its strongly recommended that the Kubernetes documentation is followed for all best practices for production environments.