With persistence and the right tools, Deborah Tylor was able to do the impossible.
A data scientist, she was tasked to comb a 3+ terabyte dataset at the Internal Revenue Service for patterns that might help uncover fraud. But even when she let the job run all night on a large bank of CPU servers the data refused to line up.
She returned in the morning to find the job had failed, so she tried again. It failed again.
About that time, Nasheb Ismaily of Cloudera knocked on the door of Rahul Tikekar, manager of a technical team that supports data analysts at the IRS. The Cloudera solutions engineer asked if Tikekar’s team had any uses for Cloudera Data Platform (CDP), implementing Apache Spark 3.0 software accelerated by GPUs.
“I jumped at the opportunity,” said Tikekar. “We have NVIDIA graphics cards on standalone servers, but using Spark to run them on a distributed cluster had eluded us for a while, so this was perfect timing for us and Deb had the perfect use case,” he said.
A Nerdy Knot Untied
A quick test of the software immediately speeded up many parts of Tylor’s work up to 5x with no code changes, but a few pieces still lagged.
Ismaily called in a team of data scientists at NVIDIA to examine the guts of the code. They quickly determined a few tasks with particularly gnarly data structures were still running on CPUs. They wrote code to handle those jobs and inserted it into Spark’s software interface for RAPIDS, the open library for running data analytics on GPUs.
Tylor ran another test, “and boom, it all went on the GPUs in a distributed Spark cluster and the speedup was remarkable — Deb’s running the whole program on a four-node cluster right now,” said Tikekar.
“The Cloudera and NVIDIA integration will empower us to use data-driven insights to power mission-critical use cases,” said Joe Ansaldi, technical branch chief of the research and applied analytics and statistics division at the IRS and Tikekar’s boss.
“We’re currently implementing this integration, and already seeing over 20x speed improvements at half the cost for our data engineering and data science workflows,” he added.
Spark 3.0 + GPUs = New Horizons
The work promises several payoffs the IRS team is already exploring.
With a Spark cluster of GPU-powered servers, the group can accelerate all its current jobs and run others previously thought impractical. And those jobs can tackle big datasets the team has at its disposal.
“Before Spark 3.0, this was not possible, but now we’re upping the ante with GPUs and we can dream of solving problems that were once impossible,” said Tikekar.
Charting a Course to AI
The team plans to apply what it learned with its success in data preparation, the so-called extract/transform/load (ETL) work of data analytics. Its next big step is accelerating full-blown AI inference jobs.
“The partnership with Cloudera and NVIDIA helped us harness GPUs in clusters. When such advances come along, it takes a while to realize their power and develop apps that can use them, so Deb is really charting a new course for us — she’s definitely the hero of the story,” Tikekar said.
Specifically, the team aims to provide this distributed Spark-GPU infrastructure to analysts. Together, they will build large deep learning neural networks to tackle natural language processing and other analytics jobs currently impossible on a single server.
Many Apps for Machine Learning
It’s the kind of transformation many enterprises are seeking today with machine learning.
“My personal feeling is that machine learning brings an incredible potential to make things that were difficult to achieve possible,” said Tikekar, a Ph.D. in computer science who spent a decade teaching at Southern Oregon University before joining the IRS more than 13 years ago.
“For example, today we scan in forms and then apply optical character recognition to read pieces of them, but with AI we can do a much better job of reading forms and finding patterns that can help find ID theft or reduce waste — a lot of applications can benefit from AI in numerous ways,” he added.
To learn more about accelerating Cloudera’s CDP 7.1.6 with NVIDIA GPUs, watch a GTC talk (free to view with registration) from October 2020, when the two companies announced their partnership.
And view Cloudera’s demo below of a 44x speed increase on a data science workload using NVIDIA GPUs and RAPIDS compared to CPUs.