Steel, Sensors and Silicon: How Caterpillar Is Bringing Edge AI to the Jobsite

At CES this week, the future of technology was on display — and it wasn’t small. It was yellow, steel and six tons of working muscle. And it was too big to fit on stage.

That’s how Deepu Talla, vice president for robotics and edge AI at NVIDIA, wound up sharing the stage with Caterpillar for what is — when measured by sheer tonnage — the biggest demo at CES this year.

During Caterpillar’s keynote at the show, the camera cut to the construction equipment manufacturer’s booth where a Cat 306 CR Mini Excavator stood ready for a live demo.

A real‑time video feed from inside the cab appeared on the keynote screens, giving the audience a close-up look at something new in heavy equipment: natural language interaction.

“Hey Cat, how do I get started?”

A voice answered, generated by an AI system running directly on the machine. It interpreted the request, accessed information and responded in a natural voice. On screen, the arm lifted. The crowd leaned forward. For a moment, the future wasn’t a slide or a spec sheet. It was right there, in steel and silicon.

Caterpillar, as CEO Joe Creed put it, “builds and powers the invisible layer of the world’s modern tech stack.” Every device in the room, and every data center behind today’s AI boom, depends on minerals extracted from the earth and infrastructure that never sleeps.

“That’s the work Caterpillar does, at scale, all around the world,” Creed said.

At CES, that invisible layer streamed onto stage, paired with AI designed to help operators work more safely, efficiently and intuitively.

Caterpillar’s machines are built for versatility across climates, terrains and job demands. And the Cat 306 CR Mini Excavator, already recognized for precision and operator‑assist features, can be found at jobsites large and small across the world.

In the demo, the Cat AI Assistant ran on NVIDIA Jetson Thor, an edge AI platform built for real‑time inference in industrial and robotic systems.

NVIDIA Riva handles speech, using NVIDIA Nemotron speech models for fast and accurate natural voice interactions.

Qwen3 4B, served locally via vLLM, interprets requests and generates responses with low latency, no cloud link required.

Caterpillar’s Helios data platform supplies trusted machine context.

Here’s how the technology came together at CES:

Safety and Boundaries (E‑Ceiling): Operators demonstrated using assistance features to support machine positioning, including limiting range of motion to avoid hazards overhead or underground. Voice interaction accelerates this setup — for example, adjusting floor boundaries above known utilities.

Intuitive Control: In tight spaces, the Cat 306’s control options matter. AI acts as an assistant, helping operators locate features, understand machine behavior and troubleshoot through natural dialog.

In‑Cab AI Assistance: Caterpillar previewed the Cat AI Assistant’s potential to offer personalized tips, safety‑oriented alerts, and access to documentation and resources.

Together, Caterpillar and NVIDIA are exploring how these building blocks could extend across fleets and jobsites, using edge AI to turn growing volumes of machine data into timely, actionable insight.

The story doesn’t end in the field.

Caterpillar is piloting factory digital twins, built on NVIDIA Omniverse libraries and OpenUSD, of multiple U.S. sites, manufacturing building and construction products as well as large mining machines.

Teams are using these digital twins to simulate line changes, test scheduling scenarios and optimize material flow before any physical retooling.
AI’s rise is also driving new demand for physical infrastructure: roads, ports, power systems and the equipment required to build and sustain them.

That’s why Caterpillar is pledging $100 million over five years for workforce training and education, including a $25 million Global Workforce Innovation challenge to identify and scale solutions that prepare workers for the next generation of AI-enabled industrial systems.

As the Cat 306 mini excavator’s yellow paint gleamed on screen, Creed left the audience with a reminder:

“If you remember one thing today, I hope it’s this … Caterpillar is still the company that builds and powers the physical world you rely on every day — and now we’re making the invisible layer of the modern tech stack more intelligent.”

That world is getting smarter, with edge AI designed to help operators, technicians and fleet managers work with machines that can listen, explain and assist. And the rumble from these machines is the sound of the future beginning to break ground.

NVIDIA Jetson Thor: Edge AI platform for real-time inference on industrial and robotics systems

NVIDIA Riva: Speech AI framework using Parakeet ASR and Magpie TTS

Qwen3 4B LLM: Compact large language model for intent parsing and response generation

vLLM: Runtime for serving LLM inference efficiently at the edge

Caterpillar Helios: Unified data platform providing trusted machine context

NVIDIA Omniverse: Digital twin and simulation libraries and frameworks for industrial workflows