Editor’s note: This post is a part of our Meet the Omnivore series, which features individual creators and developers who use NVIDIA Omniverse to accelerate their 3D workflows and create virtual worlds.
Within the Mogao Caves, a cultural crossroads along what was the Silk Road in northwestern China, lies a natural reserve of tens of thousands of historical documents, paintings and statues of the Buddha.
And nearly 2,000 miles away, in eastern China, 3D artist Ting Song has brought one of these statues to life — with the help of NVIDIA Omniverse, a physically accurate 3D design collaboration platform available with RTX-powered GPUs and part of the NVIDIA Studio suite for creators.
The Forbes 30 under 30 artist explores the concept of fine art in the digital era, blending AI with traditional art, poetry and drama.
Song, who divides her time between Beijing and Shanghai, created the first digital art piece that was auctioned by traditional art houses across China — a work called “Peony Dream,” inspired by the classic Chinese play The Peony Pavilion.
She uses Adobe After Effects and Photoshop, Blender, and Unity software with Omniverse to vivify her work.
Accelerating Art-ificial Intelligence
An avid hackathon-goer growing up, Song has shared her love of cutting-edge, open-source technology by hosting hackathons in more than a dozen countries.
She saw a multitude of groundbreaking uses for technology at these events — and was particularly spurred to use AI as a tool to foster art and creativity.
Her recent works of AI-based, immersive, multidimensional art focus on portraying philosophical and aesthetic themes from traditional Chinese culture.
For her piece that reimagines the Buddha statue, Song used Adobe software to create its layers and NVIDIA StyleGAN2 to synthesize the colors of the murals in the Mogao Caves — before bringing it into Omniverse to “let it dance,” she said.
“My work aims to give traditional art forms new life, as many existing cultural creations don’t yet exist in a 3D world, only 2D,” Song said. “NVIDIA Omniverse apps like Kaolin and Audio2Face, and NVIDIA DIB-R models support artists who are switching from traditional creations to owning new experiences in virtual worlds.”
Song uses Kaolin — her favorite Omniverse app — to inspect 3D datasets, visualize 3D outputs of a model and render synthetic datasets. Song imported models and animations from Blender and Unity into Omniverse.
And with Omniverse Audio2Face, an app that quickly generates expressive facial animation from just an audio source, Song animated a virtual poet character that she plans to integrate with her “Peony Dream” piece.
In Song’s following demo, a digital human recites a Chinese poem written by AI: “Spring is still lingering when swallows come / Strings of rain and slanting wind / Which trees are kissed upon / Stringed instruments flourish in the bloom of youth / The sun shines, and the lyric flows.”
“Digging into our true humanistic power by designing an artistic concept based on a play or poem — and then productizing it using the proper technological tools — is all enabled by Omniverse,” Song said.
In addition to revitalizing traditional works, Song often writes her own poems or scripts off of which she bases stunning visual representations made in Omniverse.
The rapid iteration and collaboration capabilities of the open-source Omniverse ecosystem and the power of NVIDIA RTX technology — which save her months’ worth of model training time — provide Song with “inspiration and technical confidence” for her artistic endeavors, she said.
“I hope my work inspires people to dive deeper into their traditional cultural heritage — and encourages them to use AI as a tool to help reveal the unique creative talents they have as human beings,” Song said.
Learn More at GTC
Song’s work will go on display in the AI Art Gallery and AI Playground at GTC, which runs March 21-24. The virtual conference is free to attend and will have dozens of sessions and special events featuring visionaries from the Omniverse team, Adobe, Autodesk, Epic Games, Pixar, Unity, Walt Disney Studios and more.
Creatives will also have the opportunity to connect with one another and get a behind-the-scenes look at the Omniverse roadmap in the NVIDIA Omniverse User Group and Developer Days.
Creators and developers can download NVIDIA Omniverse for free and get started with step-by-step tutorials on the Omniverse YouTube channel. Follow Omniverse on Instagram, Twitter and Medium for additional resources and inspiration. Check out the Omniverse forums and join our Discord Server to chat with the community.