3D Reconstruction of Endangered Species with Sifei Liu

Endangered species can be difficult to study — they’re elusive, and the very act of observing them can disrupt their lives. Now, scientists can take a closer look at endangered species by studying AI-generated 3D representations of them.

Sifei Liu, a senior research scientist at NVIDIA, has worked with her team to create an algorithm that can reconstruct 3D meshes — graphics models used to display the edges, vertices and overall shape of an object — from 2D inputs like images and videos.

Liu spoke with NVIDIA AI Podcast host Noah Kravitz about her team’s project, called Online Adaptation for Consistent Mesh Reconstruction in the Wild. Liu and her team have presented the project at various prominent conferences, including NeurIPS 2020.

The AI Podcast · Wild Things: NVIDIA’s Sifei Liu Talks 3D Reconstructions of Endangered Species – Ep. 152

Key Points From This Episode:

The deep learning algorithm made by Liu and her team creates 3D meshes with just 2D annotations. This means the AI mimics natural human perception, which can predict and visualize the 3D shape of an object based on a 2D image of it.
Initially focused on birds and zebras, Liu hopes to expand the project for use by scientists who study any endangered species, since it can be difficult to make 3D models of animals that are hard to come across. The meshes could one day be 3D-printable.

Tweetables:

“There are a lot of opportunities to leverage deep learning and graphics, and 3D reconstruction is one of them.” — Sifei Liu [19:29]

With deep learning, “you can try to make our world more beautiful.” — Sifei Liu [20:56]

You Might Also Like:

Getting Clever With Kaolin: Researchers Accelerate 3D Deep Learning With New Tools

NVIDIA researchers and engineers discuss NVIDIA Kaolin, a collection of tools within the NVIDIA Omniverse simulation and collaboration platform that allows researchers to visualize and generate datasets, move between 3D tools and retain basic functions for other users.

Perfect Pairing: NVIDIA’s David Luebke on the Intersection of AI and Graphics

The NVIDIA Research team employs more than 200 scientists around the world who are driving innovation across a range of industries. David Luebke, who founded the team in 2006 and is now the company’s vice president of graphics research, describes how AI and graphics go together “like peanut butter and jelly.”

Picture Perfection: Topaz Labs CTO Albert Yang Will Take Your Snapshots to the Next Level

Topaz Labs pioneered the intersection of deep learning and photo noise reduction. The company’s sprawling suite of image editing plugins are used by pro and amateur photographers alike. Topaz founder and CTO Feng “Albert” Yang talks about where AI and photography are going next.