Single-node Inference

GenAI-Perf sends inference requests directly to an SGLang endpoint running on a specific MI300X inference server. This mode is useful for validating single-server inference behavior, establishing baseline, and confirming the performance characteristics of an individual inference endpoint before introducing external load balancing.

In the example shown in Figure 4, GenAI-Perf runs on Lambda Scaler 2 using IP address 10.10.1.34. For direct testing, GenAI-Perf sends inference requests directly to the SGLang Router running on MI300-01 using destination port 30000. The destination address shown for this direct path is 10.10.5.25.

This traffic path bypasses Envoy and targets the MI300X inference server directly. The result is a simpler traffic pattern that is useful for isolating the performance of a single server, a single model-serving instance, and the frontend fabric path between the benchmark host and the inference endpoint.

Table 7: Single-node Inference Summary

Field	Example / Purpose
Source	GenAI-Perf client host
Example source address	10.10.1.34
Destination	MI300X inference server
Example destination address	10.10.5.25
Destination port	30000 for SGLang Router service port
Traffic behavior	GenAI-Perf sends requests directly to one inference server.
Purpose	Single-server baseline and isolated inference endpoint validation.