Keeping It Real: How an Online Shopping Site Uses Computer Vision AI to Weed Out Counterfeits

by Gary Rainville

Image-recognition engines powered by AI are typically built to identify categories of images for inclusion — say, creating a database of images of photos with dogs or spotting friends in group shots.

Sometimes, however, the goal is to identify images that aren’t wanted, such as counterfeit products on a shopping site.

That’s precisely what Barcelona-based AI startup is doing for Wallapop, a popular Spanish mobile marketplace for selling secondhand goods.

An AI-Powered Blacklist

It’s not the work Restb thought it would be doing when it first engaged with Wallapop. Founder and CEO Angel Esteban pitched Wallapop on his firm’s ability to categorize the items users were uploading to improve the browsing experience.

But he and the Restb team discovered a much bigger pain point: Wallapop couldn’t keep up with users trying to sell items such as counterfeit drugs, vitamins or food products that can get the company in big trouble with law enforcement authorities.

So, Restb set out to develop a computer vision algorithm to automate Wallapop’s process of filtering out content that violates its policies. In developing a “blacklist” classifier, Restb had to overcome three major challenges:

  • It had to enable a level of specificity that would allow the classifier to distinguish between very similar looking objects, such as a white button and a pill
  • It had to achieve at least 99 percent accuracy to minimize Wallapop’s potential exposure
  • It had to be able to analyze user-generated images captured from a variety of angles and lighting conditions

“Such a challenge cannot be tackled with ‘basic’ image recognition methods that are based on databases,” Esteban said. “This requires machine learning computer vision.”

To develop the underlying deep learning models, Restb pushed its neural networks to achieve maximum accuracy. It started by building an exhaustive dataset of possible products, as well as the variety of forms, backgrounds and environments in which they might be presented.

GPUs Play Critical Role

Restb is a preferred partner in NVIDIA’s Inception AI startup program, and an assortment of Tesla, TITAN X and GeForce GPUs figured prominently in its work. The GPUs enabled Restb to quickly iterate over its neural networks, with training experiments unfolding in days instead of months, or hours instead of days. It also used CUDA and cuDNN to assist in developing the deep learning models.

Additionally, Restb’s solutions are cloud-based, so achieving the desired accuracy and speed requires heavy computational power and parallel computing capabilities that only GPUs can deliver. The proof: Estaban reports that when Wallapop users upload images, the API Restb developed verifies images in under half a second, at which point they’re automatically rejected or allowed by Wallapop.

Stretching Computer Vision’s Limits

Restb’s work for Wallapop also has propelled its efforts to solve an even tougher image recognition challenge: getting computers to understand what they’re seeing. For example, helping the real estate industry automatically identify rooms with high ceilings or abundant natural light.

“One of the main frontiers that our work is laying the foundation for is the idea of computer vision going beyond just object recognition and moving into concept understanding,” said Esteban. “For us humans, such a classification is easy to do, but if you think about it, it’s not trivial to explain to a machine the non-tangible concept of natural light.”

Improving visual interpretation could be a game changer in healthcare, Esteban explains, noting that some countries short on medical staff have tens of thousands of exam results piled up, awaiting review. While most will turn out to be negative, the backlog can lead to patients finding out positive diagnoses too late.

AI-powered computer vision, he said, could make it possible to filter out all the negative cases, leaving only ambiguous or positive cases for review by medical professionals.

Esteban says there are multiple industries, such as real estate, sitting on millions of images with no understanding of the contents. Restb’s technology could help them boost search engine optimization results, filter out unwanted images and greatly improve the user experience.

Said Esteban: “The business applications of computer vision are broad and far reaching.”