Bram Masselink doesn’t mind rocking the boat.
While in college, the Dutch entrepreneur founded a company that provided training courses that helped 15,000 students a year with their final exams — in a country that frowns upon privatized education.
Later, he decided to look for a less controversial, but equally challenging opportunity, preferably in a highly specialized market. He found it in bringing AI technology to enhance the safety systems on oil rigs, one of the most dangerous job sites in the world.
“We wanted to find a capital-intensive international industry in a complex environment,” said Masselink. “Our assumption was that the more complex the work is, the fewer people are working on it.”
And that’s what led him to Arie Rolloos, grandson of the man who founded Rolloos Oil & Gas BV 50 years prior to Masselink’s first venture. Rolloos got its start selling and maintaining forklifts and military equipment and ultimately evolved into a provider of crane safety and camera systems on oil drilling rigs.
Masselink decided that bringing some AI in the forms of smart cameras and sensors to Rolloos’ systems was an ideal project.
“If you design or develop something really smart in which clients can be even one percent more effective, you can send a pretty decent invoice,” he said.
Rolloos used NVIDIA’s technology stack — from edge GPUs to optimized deep learning containers — to build the system and then deploy it on offshore rigs.
Birth of a Life Saver
When Masselink, now managing director of Rolloos, and his team began developing algorithms and the tools they would inform, they found out just how important this work was.
A client had an accident on a rig in which a worker was killed, due to a combination of mechanical failure and human errors. The worker was standing in a dangerous place when a piece of equipment failed, and a heavy pipe that came loose hit the worker in the head.
“It was a shock to me that people still died in the work environment,” Masselink recalled of the incident, which occurred more than four years ago.
The Rolloos team worked with the client to develop what they call the “Red Zone System,” which uses AI-powered computer vision algorithms powered by NVIDIA GPUs and a network of cameras and sensors to locate rig workers in unsafe spots, and alert them before an accident occurs.
Rolloos and the client map procedures as they’re supposed to unfold, and then compare real-time situations to that blueprint. This allows them to determine if it’s the equipment or the people that are causing inefficiencies. What they’ve found is that the Red Zone System does more than save lives.
“It’s not just a safety system, it’s a tool for finding out how to work more effectively,” said Masselink. “Maybe with other tools or machinery, they can do a better job. They just don’t know. We supply them with the tools to learn this.”
This is a critical consideration: If a job can be done with three people instead of four, that’s one person who can’t be standing in an unsafe position.
Painstaking Development Process
In developing the Red Zone System, Rolloos started by mounting cameras — lots of them — on the client’s rig and capturing 60 days worth of various types of operations performed in all types of weather and conditions.
The company then employed human labelers to mark every person and object in the video data and used that information to build an algorithm that recognizes people in a complex rig environment.
As the labeled data was run through Rolloos’ model, the focus was on improving accuracy and limiting false alarms. Today, the model pinpoints locations within 18 inches and can run a complete sequence in 250 milliseconds. Every false alarm serves as an opportunity to refine the model further.
And because there’s no internet connectivity on rigs, all of this is accomplished on hardware running onsite.
Rolloos, an NVIDIA Inception partner, started out using an NVIDIA Jetson TX2 GPU for video stream compression. They recently upgraded to the Jetson AGX Xavier and are seeing even better performance. By using NVIDIA’s acceleration toolkits like DeepStream and TensorRT from NGC, the company can actively monitor red zones in real time.
Rolloos trained its deep learning models on NVIDIA data center GPUs and deployed them for inference on an NVIDIA T4 Tensor Core GPU. Masselink said they’re also evaluating the Jetson for inference.
Rolloos is currently collecting metrics to illustrate the Red Zone System’s impact, but perhaps the most important metric has become very clear.
“People have changed their behavior,” said Masselink.
In Search of Red Zones
Looking forward, Masselink sees a lot of potential to expand on Rolloos’ Red Zone System in other settings and industries. First, he plans to expand to other parts of drilling rigs, such as spots where equipment is loaded and offloaded. He’s also working with clients to develop more automated reporting capabilities.
The mining industry and wherever large containers or industrial tools are in use are other potential markets. The key, Masselink said, is to refine the technology to the point where it’s easily transferable.
“We want to make it very standardized and more plug-and-play,” he said. “The architecture should be as modular as possible so you can implement other use cases.”