Validated Models

To characterize performance across multiple inference scenarios, the solution validates a range of commonly deployed Large Language Models (LLMs) with different model sizes and inference characteristics. This allows the benchmark methodology to evaluate how model scale, request concurrency, GPU utilization, and frontend fabric behavior interact under different workload profiles.

Table 4: Validated Models

Model	Role in Validation	Expected Inference Characteristic
Llama 3.1 8B	Represents a smaller LLM profile.	Lower latency, higher request concurrency, and smaller memory footprint.
Llama 3.3 70B	Represents a larger-scale LLM profile.	Higher compute and memory requirements; useful for validating larger model-serving behavior.
Qwen 2.5 72B	Represents an alternative large model architecture.	Useful for validating model diversity and advanced conversational or reasoning workload behavior.

The solution validates inference performance using AMD Instinct™ MI300X GPU systems and NVIDIA GenAI-Perf as the benchmark load generation tool.

Although NVIDIA GenAI-Perf originates from the NVIDIA ecosystem, it is used in this solution as an inference benchmarking and workload generation tool for services running on AMD Instinct MI300X GPU systems. It was selected due to its maturity, feature completeness, and successful operational integration within the validated Juniper lab environment.