How AI Helped Me (Almost) Give up Brownies

by Jamie Beckett

It was my wobbly willpower vs. a double-chocolate brownie, and the brownie was winning.

My newest defense: the Lose It diet app’s deep learning calorie counter. Using a photo of the brownie, it warned me of the harrowing diet damage the brownie would inflict. I ate half.

I am not a dieter. The idea of keeping track of everything I eat – a winning strategy, the experts say – makes me lose my appetite. Which, I suppose, is another diet strategy.

But last week, a beta version of Lose It’s automatic calorie-counter lured me into the ranks of the estimated 45 million American dieters.

Deep Learning Calorie Counter

screenshot from Lose It's deep learning calorie counter
The Lose It diet app has a new automatic calorie counter that uses deep learning to tally calories from a photo.

Called Snap It, the app’s GPU-accelerated deep learning keeps tabs on what you eat based largely on photos you take of what’s on your plate. It returns a list of foods it thinks are in the photo. You choose one and select a portion size, and it tallies the calorie toll.

Like any beta, it has some glitches. The app easily identifies photos of some foods like salad, pasta or a banana. But the list it generated from a picture of a glass of white wine was off the mark, and increasingly desperate sounding: water, cake, milkshake, smoothie, applesauce, fried rice, cheesecake, edamame, sushi, dumpling. When it saw a picture of cereal in a bowl, it offered up pasta, granola, almonds, fried chicken, cake, risotto, pretzels, steak, sauce or an egg.

But one of the beauties of deep learning is that the AI gets better with more data and more training feedback. And millions of Lose It customers – the app currently averages 2 million users a month – gained access to the Snap It deep learning feature this week.

“The more people use this, the more it improves,” said Edward W. Lowe, data scientist at Lose It. “The goal is to get the accuracy high enough in six months so it won’t even need to ask you for validation.”

Tough Training for Neural Network

Although Google and others have created automated calorie-counters, Lowe said Lose It’s accuracy rate is about 87 percent for foods commonly entered by its users. That surpassed others tested using the standard measure in the Food-101 dataset.

He credits that to the rigorous neural network training – he trained the network 10 times – using a vast database of 230,000 food images and more than 4 billion foods logged by Lose It users since 2008.

Lowe trained the network using the NVIDIA DIGITS deep learning training system on four NVIDIA TITAN X GPUs. DIGITS uses the latest cuDNN 5.1 deep learning library for accelerated training on NVIDIA GPUs.

“Without the GPUs, we never would have initiated this project,” Lowe said.

A Little Help Losing Weight

Even before the automatic calorie counting, Lose It has helped lots of people lose weight. Since the company launched in 2008, its members have reported losing a total of more than 50 million pounds.

It’s a good thing something works. More than two-thirds of American adults are considered to be overweight or obese, according to the National Institutes of Health. Globally, 39 percent of adults are overweight or obese, the World Health Organization said.

As for my fight with brownies, let’s just say the Snap It feature continues to get regular feedback on what a small portion of chocolatey deliciousness looks like.