NVIDIA is teaming up with the world’s largest tech companies and the U.S.’s top supercomputing labs to accelerate data analytics and machine learning, one of the fastest growing areas of high performance computing.
The new initiative marks a key moment in our work accelerating HPC, a market expected to grow considerably over the next few years. While the world’s data doubles each year, CPU computing has hit a brick wall with the end of Moore’s law.
Together with partners such as Microsoft, Cisco, Dell EMC, Hewlett Packard Enterprise, IBM, Oracle and others, we’ve already sped up data tasks for our customers by as much as 50x. And initial testing by the U.S. Department of Energy’s Oak Ridge National Laboratory is showing a remarkable 215x speedup related to climate prediction research.
A Rapid Evolution
Starting a decade ago, we brought acceleration to scientific computing. Since then, we’ve helped researchers — including multiple Nobel Prize winners — dramatically speed up their compute-intensive simulations, tackling some of the world’s greatest problems.
Then, just over five years ago, we enabled our GPU platform to accelerate deep learning through optimized software, setting in motion the AI revolution.
Now, through new open-source data science acceleration software released last month, a third wave is upon us.
At the center of this new movement is RAPIDS, an open-source data analytics and machine learning acceleration platform for executing end-to-end data science training pipelines completely on GPUs.
RAPIDS relies on NVIDIA CUDA primitives for low-level compute optimization, but exposes that GPU parallelism and high memory bandwidth through user-friendly Python interfaces. The RAPIDS dataframe library mimics the pandas API and is built on Apache Arrow to maximize interoperability and performance.
More Accelerated Machine Learning in the Cloud
Now we’re partnering with the world’s leading technology companies to bring accelerated machine learning to more users in more places.
Working closely with NVIDIA, Microsoft is introducing accelerated machine learning to its Azure Machine Learning customers.
“Azure Machine Learning is the leading platform for data scientists to build, train, manage and deploy machine learning models from the cloud to the edge,” said Eric Boyd, corporate vice president for Azure AI at Microsoft. “We’ve been partnering with NVIDIA to offer GPU-powered compute for data scientists and are excited to introduce software from the RAPIDS open source project to Azure users. I’m looking forward to seeing what the data science community can do with RAPIDS and Azure Machine Learning.”
More Systems for Accelerated Machine Learning
We’re also collaborating on a range of new products from leading computer makers based on the NVIDIA HGX-2 cloud-server platform for all AI and HPC workloads.
Delivering two petaflops of compute performance in a single node, NVIDIA HGX-2 can run machine learning workloads nearly 550x faster than a CPU-only server.
The first HGX-2 based servers are from Inspur Electronic Information, QCT and Supermicro. All three companies are featuring their new HGX-2 servers on the exhibit hall of the annual high performance computing show, SC18, in Dallas this week.
More Scientific Breakthroughs Using Accelerated Machine Learning
Our nation’s most important labs are engaged in work ranging from fusion research and human genomics to climate prediction — work that relies on scientific computing, deep learning and data science.
NVIDIA DGX-2, designed to handle the most compute-intensive applications, offers them performance breakthroughs in the most demanding areas. Now, paired with RAPIDS open-source machine learning software, DGX-2 is helping scientists at several U.S. Department of Energy laboratories accelerate their research.
Among those witnessing early success with DGX-2 and RAPIDS are researchers at Oak Ridge National Lab.
Currently, there are massive amounts of observational data available to create models to enhance energy security applications involving climate simulations. However, historically, machine learning training on climate datasets has been compute limited and slow. Until now.
Using DGX-2 and RAPIDS, researchers at ORNL are already seeing massive improvements in the speed of applying machine learning to massive datasets. Running XGBoost on their DGX-2, ORNL reduced the time to train a 224GB model from 21 hours on a CPU node down to just six minutes — a 215x speed-up.
All of the RAPIDS open-source libraries for accelerating machine learning and data analytics are available at no charge. To get started, visit the NGC container registry.