High-performance computing and neuroimaging

Seven cognitive control resting-state connectivity images. From left to right: anterior default mode network (DMN), (B) the posterior DMN, (C) the precuneus DMN, (D) the left frontoparietal network, (E) the right frontoparietal network, (F) the dorsal attention  network, and (G), the salience network.Image: Jonathan Ipser

Brain imaging methods, known as neuroimaging, allow researchers to study the structure and function of our brains in a way that is non-invasive and insightful. Dr Jonathan Ipser, senior research officer in neuroimaging and co-chair of the Psychiatry Neuroimaging Group (PNG), has been working with UCT eResearch to accelerate his own research on the impact of substance abuse and HIV on the brain.

Our brains are not immune from the impact of disease and substance abuse, but the effects of these on the brain are perhaps not as well understood as the effects they have on our bodies. Ipser has been using neuroimaging techniques, specifically structural magnetic resonance imaging (sMRI) and functional magnetic resonance imaging (fMRI), to see how our brains are affected by, among other things, HIV and substance abuse, particularly the abuse of methamphetamine.

“My particular field of expertise is known as resting-state fMRI, where we look at activity in the brain when a person is not engaged in any particular task,” says Ipser. “So I will capture brain activity for anything from five to 15 minutes, and in that time we ask the participant to just relax while we look at connectivity within the brain,” he explains. “By identifying areas that activate at the same time, we can see which brain regions are functionally connected.”

 

“We have a number of challenges around data, including storage, processing and archiving. For years now I have also been interested in issues of research reproducibility and making data more reusable for researchers down the road.”

His research has been focused on whether particular factors, such as substance abuse or HIV, are associated with differences in this connectivity, and if a normalisation of these abnormal patterns can be observed with treatment.

eResearch Seminars

“Neuroimaging is a very data-intensive field,” says Ipser. “Every brain scan creates at least a few gigs of data per participant. If you have dozens of participants scanned regularly over a period of several years, it quickly turns into a big data set.”

This is why, when Ipser saw that UCT eResearch was offering seminars for departments across campus, he was quick to sign up the PNG.

“We have a number of challenges around data, including storage, processing and archiving,” he says. “For years now I have also been interested in issues of research reproducibility and making data more reusable for researchers down the road.”

In response to Ipser’s request UCT eResearch tailored a six-session eResearch series over three months to explore the range of solutions to these neuroimaging-specific data challenges.

Data processing with high-performance computing

Immediately after one of the seminars, Ipser reached out to the high-performance computing (HPC) team at Information and Communication Technology Services (ICTS) and asked them to install specific software for processing fMRI data on the HPC cluster. He has since been using the HPC facilities to process hundreds of data sets at a much faster rate than before.

“On my own workstation it would take about 10 hours to process one patient scan, whereas on the HPC facilities I can process at least 14 scans in a 24-hour period. This has been particularly useful in processing large datasets for global collaborations with which we actively participate, such as the Enhancing Neuro Imaging Genetics through Meta-Analysis (ENIGMA) consortium.”

What Ipser also appreciated was the support with using new tools such as the Singularity and Docker software containers.

“These allow you to install the particular software package you are interested in, as well as the entire software environment, in one go,” he says. “This means all the packages that your programme is dependent on are installed at once, and you are always using the most up-to-date software.”

He adds that the HPC team are very responsive, and that working with them has afforded opportunities to experiment with different software options in a supported environment.

Working with HPC is just the beginning of the relationship between UCT eResearch and PNG. Ipser says his group is likely to request further support throughout the research project lifecycle. From data management plans to storage and archiving, the group appreciates the value of eResearch’s services to their work.