New NVIDIA H100, H200 GPU Situations Coming to Microsoft Azure

As NVIDIA continues to collaborate with Microsoft to construct state-of-the-art AI infrastructure, Microsoft is introducing further H100-based digital machines to Microsoft Azure to speed up demanding AI workloads.

At its Ignite convention in Seattle in the present day, Microsoft introduced its new NC H100 v5 VM collection for Azure, the trade’s first cloud situations that includes NVIDIA H100 NVL GPUs.

This providing brings collectively a pair of PCIe-based H100 GPUs linked through NVIDIA NVLink, with practically 4 petaflops of AI compute and 188GB of sooner HBM3 reminiscence. The NVIDIA H100 NVL GPU can ship as much as 12x larger efficiency on GPT-3 175B over the earlier technology and is good for inference and mainstream coaching workloads.

Moreover, Microsoft introduced plans so as to add the NVIDIA H200 Tensor Core GPU to its Azure fleet subsequent 12 months to assist bigger mannequin inferencing with no improve in latency. This new providing is purpose-built to speed up the biggest AI workloads, together with LLMs and generative AI fashions.

The H200 GPU brings dramatic will increase each in reminiscence capability and bandwidth utilizing the latest-generation HBM3e reminiscence. In comparison with the H100, this new GPU will supply 141GB of HBM3e reminiscence (1.8x extra) and 4.8 TB/s of peak reminiscence bandwidth (a 1.4x improve).

Cloud Computing Will get Confidential

Additional increasing availability of NVIDIA-accelerated generative AI computing for Azure clients, Microsoft introduced one other NVIDIA-powered occasion: the NCC H100 v5.

These Azure confidential VMs with NVIDIA H100 Tensor Core GPUs enable clients to guard the confidentiality and integrity of their information and purposes in use, in reminiscence, whereas accessing the unsurpassed acceleration of H100 GPUs. These GPU-enhanced confidential VMs will probably be coming quickly to personal preview.

To study extra concerning the new confidential VMs with NVIDIA H100 Tensor Core GPUs, and join the preview, learn the weblog.

Be taught extra about NVIDIA-powered Azure situations on the GPU VM data web page.