Dan Rubenstein has spent a career dedicated to saving the Grevy’s zebra, among the world’s most endangered species.
A combination of deep learning-powered image recognition software and crowdsourcing are making his data gathering faster and more accurate than ever — and could lead to real strides in conservation efforts.
When Rubenstein began studying the Grevy’s zebra in 1980, the species’ population was in a precipitous decline. Just six years later, it would be declared an endangered species.
To prevent their demise, Rubenstein, Class of 1877 Professor of Zoology at Princeton University, started documenting them on film, going so far as trying to develop the pictures in the field.
That proved onerous. So he started drawing the animals’ unique stripe patterns by hand. That was more effective, but extremely time consuming.
The arrival of digital photography, and software that allowed him to code each animal’s features for future identification, provided a “magical” boost to his work, he said.
“Individual recognition is critical to understand individual decision-making to determine how environmental features shape animal behavior,” Rubenstein said.
The Great Grevy’s Rally
The latest game-changer for Rubenstein’s work: deep learning-powered image recognition software, called HotSpotter, and a little help from 500 citizen scientists, conservationists, and other participants in the “Great Grevy’s Rally,” an innovative census effort.
HotSpotter is a set of deep convolutional neural network algorithms developed and trained by Chuck Stewart, a computer science professor at Rensselaer Polytechnic Institute, in Troy, New York, on a system running NVIDIA GPUs.
Reading Barcode-Like Stripes
It can comb through images and identify a zebra by its barcode-like stripe pattern and body shape, as well as determine its pose and analyze the quality of the image. And it can do it in about a second per image, far faster than previous methods.
HotSpotter has been the technological engine behind the Great Grevy’s Rally, which brought together hundreds of volunteer scientists, rangers and members of the public in Kenya earlier this year. In small teams armed with GPS-enabled cameras, the volunteers fanned out over some staggering 25,000-square kilometers of rangeland to take as many photos of Grevy’s zebras as they could over two days, capturing 40,000 images in the process.
This is where HotSpotter showed its mettle. Of the 40,000 images collected, 15,000 were clear and captured Grevy’s zebras facing the desired direction (researchers wanted shots of the animals’ right sides to make identification easier). After analyzing those 15,000 images, HotSpotter revealed the degree to which the Great Grevy’s Rally had succeeded.
The software determined that volunteers had photographed nearly 2,000 uniquely identified and named individuals, or more than 80 percent of the entire Kenyan population of Grevy’s zebras. (Several hundred more live in Ethiopia.)
Literal Snapshot of the Species
Nearly 1,400 unique animals were photographed on the first day of the rally. More than 500 additional previously unseen animals were captured the second day, along with nearly 900 repeat sightings from the day before.
As a result of this census effort, Rubenstein said the population of Grevy’s zebras can now be estimated with more confidence than ever at 2,350.
“This makes the estimates believable, which has not been the case previously,” said Rubenstein. “Consequently, people are taking note.”
Equally important, HotSpotter algorithms helped researchers unearth key insights. Primary among them is that 30 percent of the Grevy’s zebra population is infant or juvenile, which researchers called a critical threshold at which populations tend to stabilize and sustain themselves.
The patterns of the animals’ locations when they were photographed (and, in many cases, re-sighted) reinforce the need to ensure there are sources of grazing and water for the Grevy’s zebra in critical locations.
The team of researchers and scientists plan to use the findings to push for grassland restoration, improved water access, development of a wildlife-friendly infrastructure and a concerted effort to address high lion predation rates in one of the areas where the population is sparsest.
GPU Role Large, and Growing
HotSpotter was developed on GPUs, and it runs on them during the detection stage of the photo-analysis process — when images are searched for the presence of the target animal and aspects of the image are categorized and labeled. In fact, Stewart said that GPUs enable the detection process to run in less than a second for each image, 20 to 30 times faster than on the CPU.
“This means we can use more sophisticated algorithms and train them on a much larger corpus of image training sets,” he said.
Stewart and his team at Rensselaer are now working on a more complete set of algorithms to help with detecting and identifying marine animals, such as whale sharks, rays and dolphins, work that would also rely on GPUs.
As long as Stewart’s work on the technical side keeps automating previously manual tasks and allowing researchers to focus instead on analyzing insights extracted from image data, ecologists like Rubenstein hope to be able to make more headway than ever on the animal conservation front.
Said Rubenstein: “Getting the slowness of people out of the process is key.”