Using NVIDIA digital twin technologies, Amazon Devices & Services is powering big leaps in manufacturing with a new physical AI software solution.
Deployed this month at an Amazon Devices facility, the company’s innovative, simulation-first approach for zero-touch manufacturing trains robotic arms to inspect diverse devices for product-quality auditing and integrate new goods into the production line — all based on synthetic data, without requiring hardware changes.
This new technology brings together Amazon Devices-created software that simulates processes on the assembly line with products in NVIDIA-powered digital twins. Using a modular, AI-powered workflow, the technology offers faster, more efficient inspections compared with the previously used audit machinery.
Simulating processes and products in digital twins eliminates the need for expensive, time-consuming physical prototyping. This eases manufacturer workflows and reduces the time it takes to get new products into consumers’ hands.
To enable zero-shot manufacturing for the robotic operations, the solution uses photorealistic, physics-enabled representations of Amazon devices and factory work stations to generate synthetic data. This factory-specific data is then used to enhance AI model performance in both simulation and at the real work station, minimizing the simulation-to-real gap before deployment.
It’s a huge step toward generalized manufacturing: the use of automated systems and technologies to flexibly handle a wide variety of products and production processes — even without physical prototypes.
AI, Digital Twins for Robot Understanding
By training robots in digital twins to recognize and handle new devices, Amazon Devices & Services is equipped to build faster, more modular and easily controllable manufacturing pipelines, allowing lines to change from auditing one product to another simply via software.
Robotic actions can be configured to manufacture products purely based on training performed in simulation — including for steps involved in assembly, testing, packaging and auditing.
A suite of NVIDIA Isaac technologies enables Amazon Devices & Services physically accurate, simulation-first approach.
When a new device is introduced, Amazon Devices & Services puts its computer-aided design (CAD) model into NVIDIA Isaac Sim, an open-source, robotics simulation reference application built on the NVIDIA Omniverse platform.
NVIDIA Isaac is used to generate over 50,000 diverse, synthetic images from the CAD models for each device, crucial for training object- and defect-detection models.
Then, Isaac Sim processes the data and taps into NVIDIA Isaac ROS to generate robotic arm trajectories for handling the product.
The development of this technology was significantly accelerated by AWS through distributed AI model training on Amazon devices’ product specifications using Amazon EC2 G6 instances via AWS Batch, as well as NVIDIA Isaac Sim physics-based simulation and synthetic data generation on Amazon EC2 G6 family instances.
The solution uses Amazon Bedrock — a service for building generative AI applications and agents — to plan high-level tasks and specific audit test cases at the factory based on analyses of product-specification documents. Amazon Bedrock AgentCore will be used for autonomous-workflow planning for multiple factory stations on the production line, with the ability to ingest multimodal product-specification inputs such as 3D designs and surface properties.
To help robots understand their environment, the solution uses NVIDIA cuMotion, a CUDA-accelerated motion-planning library that can generate collision-free trajectories in a fraction of a second on the NVIDIA Jetson AGX Orin module. The nvblox library, part of Isaac ROS, generates distance fields that cuMotion uses for collision-free trajectory planning.
FoundationPose, an NVIDIA foundation model trained on 5 million synthetic images for pose estimation and object tracking, helps ensure the Amazon Devices & Services robots know the accurate position and orientation of the devices.
Crucial for the new manufacturing solution, FoundationPose can generalize to entirely new objects without prior exposure, allowing seamless transitions between different products and eliminating the need to collect new data to retrain models for each change.
As part of product auditing, the new solution’s approach is used for defect detection on the manufacturing line. Its modular design allows for future integration of advanced reasoning models like NVIDIA Cosmos Reason.
Watch the NVIDIA Research special address at SIGGRAPH and learn more about how graphics and simulation innovations come together to drive industrial digitalization by joining NVIDIA at the conference, running through Thursday, Aug. 14.