Days before a national lockdown in the U.S., Daniel McDonald realized his life’s work had put a unique tool in his hands to fight COVID-19.
The assay kits his team was about to have made by the tens of thousands could be repurposed to help understand the novel coronavirus that causes the disease.
McDonald is scientific director of the American Gut Project and the Microsetta Initiative, part of an emerging field that studies microbiomes, the collections of single-cell creatures that make up much if not most of life in and around us. The assay kits were the first to be able to safely take and ship samples from human feces preserved at room temperature.
The kits originally targeted broad research in microbiology. But McDonald and his colleagues knew they needed to pivot into the pandemic.
With careful screening, samples might reveal patterns of how the mutating coronavirus was spreading. That information would be gold for public health experts trying to slow the growth of new infections.
The team also hopes to gather just enough data from participants to let researchers explore another mystery: Why does the virus make some people very sick while others show no symptoms at all?
“Everybody here is absolutely excited about doing something that could help save lives,” said McDonald, part of the 50-person team in Rob Knight’s lab at the University of California, San Diego.
“We are lucky to work close and collaborate with experts in RNA and other areas applicable for studying this virus,” he added.
Hitting the Accelerator at the Right Time
As the kits were taking shape, the group had another stroke of good fortune.
Igor Sfiligoi, lead scientific software developer at the San Diego Supercomputer Center, ported to NVIDIA GPUs the latest version of the team’s performance-hungry UniFrac software, which is used to analyze microbiomes. The results were stunning.
A genetic analysis of 113,000 samples that would have required 1,300 CPU core-hours on a cluster of servers (or about 900 hours on a single CPU) finished in less than two hours on a single NVIDIA V100 Tensor Core GPU – a 500x speedup. A cluster of eight V100 GPUs would cut that to less than 15 minutes.
The port also enabled individual researchers to run the analysis in nine hours on a workstation equipped with an NVIDIA GeForce RTX 2080 Ti. And a smaller dataset that takes 13 hours on a server CPU now runs in just over one hour on a laptop with an NVIDIA GTX 1050 GPU.
“That’s game changing for people who don’t have access to high-performance computers,” said McDonald. For example, individual researchers may be able to use UniFrac as a kind of search tool for ad hoc queries, he said.
With the lab’s cluster of six V100 GPUs, it also can begin to tackle analysis of its expanding datasets.
Sfiligoi’s work on 113,000 samples “arguably represents the largest assessment of microbial life to date,” McDonald said. But the lab already has a repository of about 300,000 public samples and “it won’t be much longer before we’re well in excess of a million samples,” he added.
GPUs Speed Up UniFrac Three Ways
Three techniques were key to the speedups. OpenACC accelerated the many tight loops in the Striped UniFrac code, then Sfiligoi applied memory optimizations. Downshifting from 64-bit to 32-bit floating-point math delivered additional speedups without affecting the accuracy the experiments needed.
Sfiligoi completed the initial OpenACC port in a matter of days. Other optimizations came in incremental steps over a few weeks as the team got a better understanding of UniFrac’s compute and memory-access needs.
The work came on the heels of landmark effort Sfiligoi described in a session at GTC Digital. He was part of a team that harnessed exascale performance from GPUs on public cloud services for research in astronomy.
NVIDIA is collaborating with Sfiligoi on his next task. He aims to integrate his GPU optimizations on UniFrac into the software microbiologists use daily.
Data Flood Would Swamp CPU-only Systems
Meanwhile, McDonald and his team need to adapt UniFrac to work with viral data. They also face heady challenges turning vast amounts of data they will generate into well organized and error-free datasets they can process.
On the tech front, the group needs lots of storage and compute performance. To analyze what could one day amount a million microbiomes could require 20 petabytes of storage and more than 100 million CPU cycles/year.
“I would love to see a lot of that pushed onto GPUs,” McDonald said.
The work has broad potential given how long the extended family of coronaviruses has been affecting both humans and farm animals.
“Everybody on the planet has felt these impacts on productivity in some way. Now we can start to understand how to better manage this family of viruses that have been with us a long time,” he added.
The efforts in San Diego are part of a broad network of research projects leveraging NVIDIA GPUs and high performance computing to fight COVID-19.
More than 30 supercomputing centers worldwide spanning centers in Asia, Australia, Europe and the United States are engaged in the effort. The COVID-19 High Performance Computing Consortium alone has more than 30 active projects with access to 420 petaflops of power that includes 41,000 GPUs.
Image at top: Rob Knight (left) and Daniel McDonald in the UCSD Knight Lab. Photo courtesy Erik Jepsen/UC San Diego Publications