Plowing AI, Startup Retrofits Tractors with Autonomy

Smart Ag is developing driverless tractors to help with a farm labor shortage.
by Scott Martin

Colin Hurd, Mark Barglof and Quincy Milloy aren’t your conventional tech entrepreneurs. And that’s not just because their agriculture technology startup is based in Ames, Iowa.

Smart Ag is developing autonomy and robotics for tractors in a region more notable for its corn and soybeans than software or silicon.

Hurd, Barglof and Milloy, all Iowa natives, founded Smart Ag in 2015 and landed a total of $6 million in seed funding, $5 million of which came from Stine Seed Farm, an affiliate of Stine Seed Co. Other key investors included Ag Startup Engine, which backs Iowa State University startups.

The company is in widespread pilot tests with its autonomy for tractors and plans to commercialize its technology for row crops by 2020.

Smart Ag is a member of the NVIDIA Inception virtual accelerator, which provides marketing and technology support to AI startups.

A team of two dozen employees has been busy on its GPU-enabled autonomous software and robotic hardware system that operates tractors that pull grain carts during harvest.

“We aspire to being a software company first, but we’ve had to make a lot of hardware in order to make a vehicle autonomous,” said Milloy.

Wheat from Chaff

Smart Ag primarily works today with traditional row crop (corn and soybean) producers and cereal grain (wheat) producers. During harvest, these farmers use a tractor to pull a grain cart in conjunction with the harvesting machine, or combine, which separates the wheat from the chaff or corn from the stalk. Once the combine’s storage bin is full, the tractor with the grain cart pulls alongside for the combine to unload into the cart.

That’s where autonomous tractors come in.

Farm labor is scarce. In California, 55 percent of farms surveyed said they had experienced labor shortages in 2017, according to a report from the California Farm Bureau Federation.

Smart Ag is developing its autonomous tractor to pull a grain cart, addressing the lack of drivers available for this job.

Harvest Tractor Autonomy

Farmers can retrofit a John Deere 8R Series tractor using the company’s AutoCart system. It provides controllers for steering, acceleration and braking, as well as cameras, radar and wireless connectivity. An NVIDIA Jetson Xavier powers its perception system, fusing Smart Ag’s custom agricultural object detection model with other sensor data to give the tractor awareness of its surroundings.

“The NVIDIA Jetson AGX Xavier has greatly increased our perception capabilities — from the ability to process more camera feeds to the fusion of additional sensors —  it has enabled the path to develop and rapidly deploy a robust safety system into the field,” Milloy said.

Customers can use mobile devices and a web browser to access the system to control tractors.

Smart Ag’s team gathered more than 1 million images to train the image recognition system on AWS, tapping into NVIDIA GPUs. The startup’s custom image recognition algorithms allow its autonomous tractor to avoid people and other objects in the field, find the combine for unloading and return to a semi truck for the driverless grain cart vehicle to unload the grain for final transport to a grain storage facility.

Smart Ag has more than 12 pilot tests under its belt and uses those to gather more data to refine its algorithms. The company plans to expand its test base to roughly 20 systems operating during harvest in 2019 in preparation for its commercial launch in 2020.

“We’ve been training for the past year and a half. The system can get put out today in deployment, but we can always get higher accuracy,” Milloy said.