National Radio Astronomy Observatory forges agreement with new SA astronomy institute
The National Radio Astronomy Observatory (NRAO) and Inter-university Institute for Data Intensive Astronomy (IDIA) have signed a Memorandum of Understanding (MoU) to collaborate on algorithm and software development in support of forthcoming surveys on the Jansky Very Large Array (VLA) and MeerKAT.
Several next generation all-sky surveys in the 1-5 GHz range are planned in the next few years with wide-band telescopes, including the VLA in the U.S. and MeerKAT in South Africa. Pilot observations with the VLA are underway and Early Science with MeerKAT is scheduled to begin later this year, with full operations in late 2017.
These surveys will be pathfinders for the scientific direction and technical challenges of even larger surveys with the Square Kilometre Array (SKA) in the future. Due to high sensitivity, wide field-of-view, and relatively low frequencies, these surveys require high fidelity and in some cases, high dynamic range imaging. This requires use of compute-intensive imaging algorithms to correct for multiple direction-dependent effects including corrections for non-coplanar baselines and antenna primary beams as a function of time, frequency and polarization. These surveys will also generate PetaByte-scale data volumes.
Advanced imaging algorithms required
Processing data from such surveys therefore poses the combined problem of Big Data and High Performance Computing. Advanced imaging algorithms required for these surveys are currently being tested for release in the Common Astronomy Software Applications (CASA) software package. Software for processing large data volumes using compute-intensive imaging algorithms on high performance computing platforms and pipeline processing, where possible, are also under development.
Making these solutions robust and reliable for automatic processing of these data-intensive surveys requires more research and development for scientific computing and related computer science problems, and use of large massively parallel hardware computing platforms to test and deploy the software solutions. Students, postdoctoral fellows and scientists with multidisciplinary skills thus become crucial components in realizing the full scientific potential of such surveys and the involved telescopes.
The Jansky VLA and MeerKAT have a large overlap in the parameters of the telescope and the planned surveys. Both telescopes therefore also have high synergy in the required post-processing technical solutions. In January 2016, to pool the available expertise and resources, NRAO and the inter-university Institute for Data Intensive Astronomy (IDIA) completed an agreement to work together to explore the research and development challenges, and to build the data processing solutions within NRAO’s CASA software environment.
Collaboration of three research-intensive universities
IDIA is a new South African institute established as a collaboration of three research-intensive universities engaged in MeerKAT large survey programs and SKA pathfinder projects: University of Cape Town, University of the Western Cape, and North-West University. A primary goal of the IDIA is to develop capacity for data processing and build data science solutions for processing and analysis of the large data sets from MeerKAT. Scientists, developers, postdocs and students at NRAO and IDIA will collaborate on algorithm development, and deploy and test the solutions on data-centric high performance computing platforms with the intent to provide the project teams for next generation surveys with the tools to process and analyze the large data sets in time for the data onslaught.
IDIA has begun acquiring hardware for large compute clusters located at the University of Cape Town that will be available for development and testing of parallel processing solutions in CASA. Developers and post-docs will be hired at IDIA later this year to begin collaborative software development of mutual interest in the CASA software system. Exchange visits from personnel at both ends are expected to also begin later this year and continue in the coming years. A work plan and processes to track the work is also being developed.
Written by S. Bhatnagar (NRAO), R. Taylor (IDIA, South Africa). Image courtesy of SKA South Africaâ