Data Challenges for Next-generation Radio Telescopes

Abstract: Radio-astronomy is about to embark on a new way of doing science. The revolution that is about to take place is not due to the enormous sensitivity of the Square Kilometre Array, which is still a decade away, but due to its pathfinders, which are pioneering new ways of doing radio-astronomy. These new ways include multi-pixel phased-array feeds, the goal of producing science-ready images from a real-time pipeline processor, and from the vast amounts of survey data that will be available in the public domain soon after observing. Here I review the data challenges that need to be addressed if we are to reap all the science that potentially resides in SKA Pathfinder data. Some challenges are obvious, such as petabytes of data storage, and some are less obvious, such as the techniques we have yet to develop to perform cross- identifications on millions of galaxies.