The fight against antibiotic-resistant superbugs has been taken up by researchers wielding MASSIVE computing power. Specifically, the Multi-modal Australian ScienceS Imaging and Visualisation Environment — Australia’s specialized high performance computing facility for data processing.
The system is fueling cryo-electron microscopy (CryoEM), which allows researchers to analyze the forms and shapes of molecules at extremely high resolution. CryoEM provides such a level of detail that it’s led to the coining of the term “resolution revolution,” and its inventors were awarded the Nobel Prize in chemistry in 2017.
Dr. Matt Belousoff and his colleagues at Monash University, in Australia, are using the technology in an effort to create superdrugs that combat superbugs. They’ve set their sights on the design of new drugs that are lethal to bacteria despite structural changes to their ribosomes, the place where biological protein synthesis occurs inside all living cells.
The CryoEM process involves taking high-resolution images — about 500,000 individual ones — of pure preparations of ribosomes isolated from hospital-acquired, antibiotic-resistant Staphylococcus aureus. These noisy images are then transformed into a 3D electron density map to provide a molecular model for the research team to work with.
Processing this volume of low-resolution 2D images into a high-resolution 3D model is incredibly computationally intensive and requires months of processing time on a desktop computer. MASSIVE provides the data processing engine to resolve the models in days.
“The knowledge gained about the effects of mutations that lead to drug resistance on the ribosome binding site will help us to re-engineer the binding site in the future,” said Belousoff.
MASSIVE HPC Power
MASSIVE is a collaboration among Monash University, the Commonwealth Scientific and Industrial Research Organisation, the Australian Nuclear Science and Technology Organisation, the University of Wollongong and two affiliates — the ARC Centre of Excellence in Integrative Brain Function and the ARC Centre of Excellence in Advanced Molecular Imaging.
In addition to CryoEM, it’s used for advanced research across a range of imaging modalities, including synchrotron X-ray and infrared imaging, functional and structural magnetic resonance imaging, X-ray computer tomography, electron microscopy, and optical microscopy.
MASSIVE connects all four new-generation CryoEM instruments in Australia at Monash University and the University of Wollongong. The microscopes produce high-resolution images — as well as incredible data quantity and analysis challenges.
To address these issues, the facility provides researchers with high-performance data processing on its M3 computer, which is designed to meet core data processing and visualization requirements, including fast reading and writing, as well as storage of the enormous amounts of data.
Powered by 162 NVIDIA Tesla V100, P100 and K80 GPU accelerators and two NVIDIA DGX-1 AI supercomputers, M3 is used for fast rendering and remote visualization applied across a wide range of data processing and simulation problems.
While many of MASSIVE’s users are new to HPC, the system provides easy data processing using a powerful, application-rich remote desktop environment powered by NVIDIA GPUs.
“To provide biologists with the computing power available on MASSIVE, we’ve integrated our M3 computer directly with CryoEM instruments across Australia,” said Dr. Wojtek James Goscinski, associate director of the eResearch Centre at Monash University. “This allows researchers new to HPC access our capabilities as easily as possible.”
Getting More, Doing More
MASSIVE has one of the largest CryoEM HPC user communities in the world with more than 25 research groups and over 100 users. The M3 supports around 1,000 researchers, with about 10 groups using the DGX-1.
The system uses the DGX-1 to get more performance for CryoEM than traditional approaches. DGX-1 is powered by eight NVIDIA Tesla V100 Tensor Core GPUs configured in a hybrid cube-mesh topology. It uses NVIDIA NVLink for ultra-fast, low-latency inter-GPU communications that are essential to delivering the performance MASSIVE requires.
A scheduler sits on top of a file system and is integrated with the HPC environment as well as the research network at the university. This means those producing data on the CryoEM instruments are only one hop away from the DGX-1.
The results have been impressive, with some tasks running up to 20x faster on DGX-1 than with earlier hardware. “We’ve had very strong results from the DGX-1, which can scale further than anything else,” said Goscinski.
In addition to CryoEM, the DGX-1 supercomputers are used for machine learning and deep learning applications. These techniques have grown from under 10 percent to about 30 percent of the overall workload over the last three years, says Goscinski.
Image credit: Monash University CryoEM (ThermoFischer Titan Krios).