The latest release of NVIDIA Maxine brings new and updated features that enhance real-time communication and elevate high-impact video editing with AI.
The Maxine developer platform redefines video conferencing and editing by providing developers and businesses with GPU-accelerated AI services in the cloud, so they can enhance video and audio streams in real time. With the Maxine production release, now exclusively available on NVIDIA AI Enterprise, users can access advanced features across augmented reality, audio effects and video effects.
The New Face of Avatars
Digital avatars have been used for decades, often seen as stylized animated representations of a person or character.
Now, with NVIDIA Maxine’s Live Portrait feature, users can choose the perfect photo of themselves and animate it with a standard webcam. Live Portrait syncs a person’s head movement and facial expressions to the user’s chosen photo. Users can also choose 2D stylized character representations of themselves.
Live Portrait is now available as a production feature in Maxine and can use high-resolution model outputs of up to 1024×1024 resolution.
Finding Your Voice
A person’s voice can convey emotion and communicative nuances. NVIDIA Maxine’s new Voice Font feature, available in early access, enables users to generate a unique voice for themselves — almost like a digital avatar for voice.
The feature can convert audio samples into a digital voice with just 30 seconds of reference audio.
Voice Font can be helpful for people who have speech impediments, or who want to fine-tune the sound of their own voice. Voice Font is available for evaluation and testing in the early access release of Maxine.
Seeing Eye-to-Eye
NVIDIA Maxine’s Eye Contact feature uses AI and a webcam feed to direct the user’s gaze toward the camera in real time. Similarly, it can repose eyes in offline video to create more engaging and impactful videos.
Studies have shown that maintaining eye contact during conversations encourages personal connection, understanding and engagement. Maxine Eye Contact enhances communication by ensuring that the user is always looking at their audience, whether in a video conference or via prerecorded video.
The new version of Maxine Eye Contact preserves naturally occurring micro-eye movement and has the added ability to “look away” periodically, creating a more realistic experience on video conferences. The frequency and duration of the “look away” function is also adjustable.
The latest version of Eye Contact also brings quality improvements, including increased gaze stability, more robust occlusion handling and much lower latency when used with NVIDIA Ada Generation GPUs.
The Best of the Rest
The newest Maxine release also delivers new and updated real-time features across augmented reality, audio effects and video effects to advance video conferencing and editing capabilities. These include:
- 3D Body Pose, for pose estimation when only the upper body is in camera view, introducing a new dimension to virtual interactions
- Support for NVIDIA L4 Tensor Core and L40 GPUs, powered by the NVIDIA Ada Lovelace architecture, bringing up to 1.75x performance increases compared to GPUs from previous families
- NVIDIA Triton Inference Server support for AI Green Screen, Eye Contact, Landmark Detection and Face Detection, enabling higher throughput for both single- and multiple-GPU workflows
- Audio super resolution performance improvements
Partnering for Success
Many NVIDIA partners and customers, such as Quicklink and CoPilot AI, are already experiencing high-quality video conferencing and editing with Maxine.
Quicklink is a leading global provider of remote production solutions to the media, broadcast, production and sport industries. The company’s Cre8 video production tool allows users to deliver professional virtual, in-person and hybrid events.
“Our partnership with NVIDIA has been driven by broadcast industry challenges, beginning with remote guest contribution,” said Richard Rees, CEO of Quicklink. “Our integration of NVIDIA Maxine into Quicklink Cre8 resolves these challenges with the addition of Maxine’s Auto Framing, Video Noise Removal, Noise and Echo Cancellation, and Eye Contact features. These features have had an incredible reception across the industry.”
CoPilot AI, a Vancouver-based software-as-a-service startup operating in the cross-section between AI and sales enablement, incorporates Maxine to provide dependable solutions for content creators.
“Using NVIDIA Maxine microservices, we empower users to record on the whim and stand out in the crowd,” said Jackson Chao, cofounder of CoPilot AI Video. “Maxine’s Eye Contact feature allows users to record a script without compromising the connection with viewers. Even novice content creators are able to adopt video outreach with confidence and humanize the way they engage with their contacts.”
Maxine offers a collection of AI effects that enhance real-time audio and video and can be incorporated into existing customer infrastructures. And the solution can be deployed in the cloud, on premises or at the edge, enabling quality communication from nearly anywhere.
Availability
From enhancing day-to-day video conferencing needs to integrating AI technology, NVIDIA Maxine offers high-quality video communications for all professionals.
The latest Maxine production release is included exclusively with NVIDIA AI Enterprise 4.1, allowing users to tap into production-ready features such as Triton Inference Server, enterprise support and more.
For customers requiring access to NVIDIA Maxine’s limited early access program, fill in the relevant online application on the Maxine Microservices Early Access Program or Maxine SDK Early Access Program pages.
To help improve features in upcoming releases, participants can provide feedback by contributing to the NVIDIA Maxine and NVIDIA Broadcast App survey.