Frontend Fabric Components
The frontend fabric in this solution includes both hardware and software components for switching, automation, compute, and application load balancing, as well as benchmarking components required to validate AI inference traffic.
The following table describes the components included in the validated solution.
Table 6: Frontend Fabric Components
| Validated Components | Role in Solution | Description |
|---|---|---|
|
HPE Juniper Networks QFX5130-32CD HPE Juniper Networks QFX5140-24CD8O HPE Juniper Networks QFX5240-64OD |
Frontend leaf node | Provides frontend connectivity for compute devices and client applications. |
|
HPE Juniper Networks QFX5220-32CD HPE Juniper Networks QFX5230-64CD HPE Juniper Networks QFX5240-64OD |
Frontend spine node | Provides the spine layer for the three-stage Clos frontend fabric, enabling redundant high-speed connectivity, between the leaf and spine nodes. |
| HPE Juniper Apstra Data Center Director | Intent-based automation | Used to simplify fabric deployment and provide operational consistency. |
| AMD Instinct MI300X GPU servers | Inference compute nodes running model-serving frameworks. | Two AMD MI300X systems are used, each with eight AMD Instinct MI300X GPUs. The systems run SGLang and host the GPU-backed model-serving endpoints. |
| ConnectX-7 NICs | Frontend NICs on the AMD MI300X systems. | Provide 400G frontend connectivity from the MI300X inference servers to the QFX-based frontend fabric. |
| Lambda scalers | Client, benchmark, and load balancing hosts. | Two scaler systems are used. These systems include dual RTX 5000 Ada GPUs and ConnectX-6 frontend NICs. |
| Lambda-Scaler-01 | Envoy load balancer host. | Runs Envoy inside a container and distribute query traffic across the MI300X systems running SGLang. |
| Lambda-Scaler-02 | GenAI-Perf benchmark host. | Runs NVIDIA GenAI-Perf inside a container to generate high-volume inference traffic and collect inference performance metrics. |
| Envoy Proxy | Optional load balancer in front of SGLang inference endpoints. | Used for scale-out request distribution across multiple AMD MI300X inference servers. |
| SGLang | Inference serving framework running on the AMD Instinct MI300X GPU servers. |
Loads and runs the validated LLMs on the MI300X systems. It processes inference requests through GPU-backed worker processes to generate responses. In the JVD test environment, each GPU-backed worker runs a local model instance on one AMD Instinct MI300X GPU. |
| SGLang Router | Request routing component running on the AMD Instinct MI300X GPU servers. | Receives inference requests from GenAI-Perf directly or through Envoy and distributes them to local SGLang worker processes on the MI300X server. The router listens on the service port used by the benchmark tests and keeps worker distribution local to the inference server. |
| NVIDIA GenAI-Perf | Inference load generation and benchmark collection tool. | NVIDIA GenAI-Perf is used as the benchmark load generator in the validated lab environment. |
The configuration files used for these components are available in the Juniper JVD GitHub repository: https://github.com/Juniper/jvd/tree/main/data_center/aidc/aiml-inference-frontend