Data Centers for AI Cloud Service Providers

Service providers (SPs) and neocloud providers are deploying purpose-built AI data centers to offer custom, affordable, and quick-to-market AI services for enterprises, governments, and educational institutions. Cloud-hosted AI services offer virtualized and secure compute, storage, and networking to end users while enabling new revenue streams with increased efficiency and lower total cost of ownership.  

A neocloud is a new breed of AI cloud compute provider focused on offering virtualized GPU compute with supporting storage and secure networking. These pure play GPU clouds offer cutting-edge performance and flexibility to their customers with the ability to amortize the cost of their AI cloud infrastructure across a large customer base. Using cloud tools and automation, neoclouds gain efficiency in their underlying AI infrastructure with cloud agility to scale up and scale out to meet customer demand.

AI models are built using carefully crafted data sets during the training stage. Training happens across multiple GPUs spanning tens, hundreds, and even thousands of GPUs in a cluster—all connected across a network and constantly exchanging data with each other. After this training stage, the model is essentially complete. During the inference stage, users interact with the model, which can recognize images or generate pictures and text to provide answers to user questions. Training is typically an offline operation, whereas inference is generally online.

Massive AI data sets are creating the need for greater compute power, faster storage, and high-capacity, low-latency networking. Juniper helps meet these requirements in the following ways:

• Compute: AI/ML compute clusters place heavy requirements on the internode network. Lowering job completion time (JCT) is essential, and the network plays a key part in the efficient operation of the cluster. Juniper offers a range of high-performance, non-blocking switches with deep buffer capability and congestion management that, when architected optimally, eliminate any network bottleneck.
• Storage: In AI/ML clusters and high-performance computing, rarely can an entire data set or model be stored on the compute nodes, so a high-performance storage network is required. Juniper QFX Series switches can be used for IP storage connectivity. They offer full support for Remote Direct Memory Access (RDMA) networking, including Non-Volatile Memory Express/RDMA over Converged Ethernet (NVMe/RoCE) and Network File System (NFS)/RDMA.
• Network: AI training models involve large, intense computations distributed over hundreds or thousands of CPU, GPU, and TPU processors. These computations demand high-capacity, horizontally scalable, and error-free networks. Juniper QFX switches and PTX Series Routers support these large computations within and across data centers with industry-leading switching and routing throughput and data center interconnect (DCI) capabilities.

Juniper Apstra is Juniper’s leading platform for data center automation and assurance. It automates the entire network life cycle, from design through everyday operations, across multivendor data centers with continuous validation, powerful analytics, and root cause identification to assure reliability. With Marvis VNA for Data Center, this information is brought from Apstra into the Juniper Mist Cloud and presented in a common VNA dashboard for end-to-end insight. Marvis VNA for Data Center also provides a robust conversation interface (using GenAI) to dramatically simplify knowledgebase queries.  

Juniper high-performance, non-blocking data center switches provide deep buffering and congestion management to eliminate network bottlenecks. To balance traffic loads, we support dynamic load balancing and adaptive routing. For congestion management, Juniper fully supports Data Center Quantized Congestion Notification (DCQCN), Priority Flow Control (PFC), and Explicit Congestion Notification (ECN). Finally, to reduce latency, Juniper uses best-of-breed merchant silicon and custom ASIC architectures that maximize buffers where needed, virtual output queuing (VOQ), and cell-based fabrics within our spine architectures.

Our portfolio includes open, standards-based switches that provide IP-based storage connectivity using NVMe/RoCE or NFS/RDMA (see earlier FAQ). Our IP Storage Networking solution designs can scale from a small four-node configuration to hundreds or thousands of storage nodes.