Now in General Availability: NVIDIA H100 GPUs in Azure Confidential VMs

Enterprises know protecting their sensitive data is a priority while running AI.

With Microsoft Azure announcing today the general availability of confidential virtual machines equipped with NVIDIA H100 NVL Tensor Core GPUs, it has created a secure, scalable way to protect GPU-enabled workloads.

Azure is the first cloud provider to offer confidential computing with NVIDIA H100 GPUs, uniquely combining the power of accelerated computing with the security of confidential computing. Azure NCC H100 v5 VMs are purpose-built to deliver optimal performance while maintaining high levels of data protection throughout the entire lifecycle of code and data. They’re designed to cater to the needs of those working on the inference, fine-tuning and training of small- to medium-sized models like Whisper and Stable Diffusion (and its variants like SDXL and SSD), and language models like Falcon, GPT2, Llama 2, MP5, Wizard, Wxin and Zephyr.

H100 GPUs were the first to introduce support for confidential computing on GPUs, which keeps data encrypted while it’s being processed, ensuring that sensitive information stays private and protected.

“Our collaboration with NVIDIA has been pivotal in launching Azure confidential VMs with NVIDIA H100 Tensor Core GPUs,” said Vikas Bhatia, head of product for Azure confidential computing at Microsoft. “This work combines Microsoft’s expertise in building out the cloud infrastructure necessary to support confidential VMs with NVIDIA’s cutting-edge GPU technology, helping ensure our customers benefit from unparalleled security and performance for their most sensitive AI workloads.”

Confidential Computing in Action

In preview, Azure NCC H100 v5 VMs demonstrated immense potential for a variety of AI and data-intensive workloads. Customer use cases included:

Edgeless Systems extends access to secure large language models: Azure’s confidential computing environment enables broad, safe access to LLMs, protecting everything from the prompts and weights used in model training to the models themselves. This helps ensure that organizations can innovate with AI while safeguarding critical intellectual property.
Confidential inference on audio-to-text models: Customers can transform spoken content into text while safeguarding sensitive information by using the secure environment to run inference tasks on Whisper with audio data.
Detection anomalies in video feeds: Organizations can employ video input models to detect abnormal behavior for incident prevention, using confidential computing to help ensure data privacy during inference tasks.
Securing automotive design: Confidential VMs can be used with Stable Diffusion for inference and training on privacy-sensitive automotive design data, enabling secure, compliant innovation.
Delivering financial multiparty clean rooms: Financial sector customers can create multi-party clean rooms to run analytical tasks on massive datasets — including billions of transactions and terabytes of data across multiple subsidiaries — while adhering to data privacy regulations.