After centuries of steadily decimating the world’s whale populations, humans have taken significant steps to reverse their catastrophic impact.
One of the latest: an AI-powered project spearheaded by the Canadian government that aims to minimize collisions between ships and North Atlantic right whales, 50-foot-long creatures that got their name from early whalers for being the “right” ones to kill.
The problem is relatively new. North Atlantic right whales are moving into new waters, possibly following plankton that are being driven north by climate change.
Forced to follow their prey, the whales have adopted a new migration path through an established shipping route. With the global population of right whales now estimated to be just 500, protecting these endangered creatures from ship strikes has quickly become an urgent issue.
Over the past few years, Transport Canada (the national transportation department) has been working to prevent commercial shipping vessels from striking the whales. The program flies aircraft manned with marine biologists over the shipping lanes during the migration season.
When the biologists spot whales in a shipping lane, vessels are alerted to slow down. The resulting disruption to shipping operations is significant – resulting in higher costs, longer shipping times and lower revenue.
In a proof of concept, Transport Canada ran a new variation of the program in August, using an unmanned aerial system (UAS) and AI software to detect whales in shipping lanes.
Planck Aerosystems, which makes autonomous drones for monitoring ships, ports and borders, was brought in to develop AI software that could help identify North Atlantic right whales within the data (still images and video) being collected by the UAS. The new solution has the potential of simultaneously helping to protect the whales and Canada’s shipping economy.
“The kind of decisions that the government needs to make can have huge financial impact,” said Gaemus Collins, chief technology officer at Planck. “Our software was made to assist biologists in searching through thousands of images to find the few that may contain whales. Ultimately the biologists make the final determination on whether ‘yes, this is a whale, and yes, it’s a right whale, and to provide feedback to Transport Canada regarding their presence.”
Read about other deep learning work to identify North Atlantic right whales from the air.
Mission: Dataset
The company has discussed with the U.S. National Oceanic and Atmospheric Administration a similar project off the coast of Southern California. In that effort, computer vision tools would automatically detect whales from aerial drone imagery.
NOAA officials knew of Transport Canada’s efforts, suspected that Planck’s technology would support that mission, and made the introductions.
Planck’s first task was to create a dataset to use in training a deep learning model.
“They had the mission pretty well scoped out, but they didn’t have data from previous missions to train the algorithm; this was a big challenge” said Collins. “We had to create a system to re-train the detection algorithm in the field, every day, after the UAS flights were already underway.”
Initially, Transport Canada officials provided Planck with 1080p imagery, which Planck was able to process on NVIDIA’s compact Jetson TX2 embedded AI computing device. But it turned out that 1080p images didn’t provide enough resolution to identify the whale species, or any additional markings on whales. (One of the project objectives was to collect sufficient image data to track the whales individually in the future.)
Transport Canada ended up switching to a 24-megapixel imager. Planck paired this with an NVIDIA Quadro P5000 GPU, which provides sufficient RAM to do inference on such large image files. CUDA 9 and cuDNN 7.1 were featured in all systems to allow for GPU-accelerated training and inference.
Training occurred in the cloud-based Google Compute Engine, where Planck could allocate as many NVIDIA Tesla V100 GPUs as it needed.
Planck leveraged a deep learning library called darknet. This was used for training and deploying Planck’s object detection algorithm. TensorRT was brought into the fold later to speed up the inference time of the algorithm.
Takeaways and Next Steps
Collins said one of the primary lessons Planck learned during the project, which has been handed off to Transport Canada, was just how computationally expensive such ventures are. Simply put, it’s just not possible to have too much speed, and eventually computing needs will stress the hardware.
“Even when you throw a very high-power GPU at this problem, it’s still worth it to optimize the software as well,” said Collins.
Collins said he hopes there will be further opportunities for Planck to work with other government agencies on similar environmental programs. The company already has had discussions with Fisheries and Oceans Canada about using the technology to detect and track seals and sea lions.
For now, however, Collins expressed pride at what Planck has accomplished and looks forward to using it to protect whales around the globe.
“We’ve proven with this project that deep learning tools can be used to automate the detection and classification of marine mammal species,” said Collins. “We’d like to see this system deployed on a larger scale in more locations where whale strikes are a regular occurrence.”