Large radio surveys such as those proposed for several SKA pathfinder and precursor telescopes require (automated) source-finding algorithms that are highly reliable, complete, and fast. Similarly, sophisticated visualisation tools are needed to explore the resulting survey data together with their multi-wavelength counterparts.

In this PASA Special Issue on Source Finding and Visualisation, several advanced source-finding algorithms, including novel methods for radio continuum, polarisation and spectral line surveys are described, tested and compared. The process of finding sources can be considered one of many important steps (e.g., pre-processing, source finding and characterisation, cataloguing/post-processing) in the production of astronomical source catalogues, on which much of the survey science is based.

The Australian SKA Pathfinder (ASKAP), equipped with novel Chequerboard phased array feeds, will be a powerful 21-cm survey machine. For the large volumes of ASKAP data, a special version of Duchamp, called Selavy, is being developed in consultation with the ASKAP survey science teams (Whiting & Humphreys 2012). Extensive testing and comparisons of existing and new source-finding algorithms were carried out for this PASA Special Issue and are presented by Westmeier, Popping & Serra (2012), Westerlund, Harris & Westmeier (2012), Popping et al. (2012), Huynh et al. (2012), Hollitt & Johnston-Hollitt (2012), Walsh et al. (2012), George, Stil & Keller (2012), Marsh & Jarrett (2012), Allison, Sadler & Whiting (2012) and Jurek & Brown (2012). An overview on spectral line source finding and visualisation is given by Koribalski (2012), while Hassan, Fluke & Barnes (2012) look into real-time 3D volume rendering of large (Tbytes) astronomical data cubes.

Detection thresholds in polarized intensity and polarization bias correction are investigated for surveys where the polarization information is obtained from rotation measure (RM) synthesis. Considering unresolved sources with a single RM, a detection threshold of 8 σQU applied to the Faraday spectrum will retrieve the RM with a false detection rate less than 10−4, but polarized intensity is more strongly biased than Ricean statistics suggest. For a detection threshold of 5 σQU, the false detection rate increases to ∼4%, depending also on λ2 coverage and the extent of the Faraday spectrum. Non-Gaussian noise in Stokes Q and U due to imperfect imaging and calibration can be represented by a distribution that is the sum of a Gaussian and an exponential. The non-Gaussian wings of the noise distribution increase the false detection rate in polarized intensity by orders of magnitude. Monte Carlo simulations assuming non-Gaussian noise in Q and U give false detection rates at 8 σQU similar to Ricean false detection rates at 4.9 σQU.

The large spectral bandwidth and wide field of view of the Australian SKA Pathfinder radio telescope will open up a completely new parameter space for large extragalactic HI surveys. Here we focus on identifying and parametrising HI absorption lines which occur in the line of sight towards strong radio continuum sources. We have developed a method for simultaneously finding and fitting HI absorption lines in radio data by using multi-nested sampling, a Bayesian Monte Carlo algorithm. The method is tested on a simulated ASKAP data cube, and is shown to be reliable at detecting absorption lines in low signal-to-noise data without the need to smooth or alter the data. Estimation of the local Bayesian evidence statistic provides a quantitative criterion for assigning significance to a detection and selecting between competing analytical line-profile models.

The process of determining the number and characteristics of sources in astronomical images is so fundamental to a large range of astronomical problems that it is perhaps surprising that no standard procedure has ever been defined that has well-understood properties with a high degree of statistical rigour on completeness and reliability. The Evolutionary Map of the Universe (EMU) survey with the Australian Square Kilometre Array Pathfinder (ASKAP), a continuum survey of the Southern Hemisphere up to declination +30°, aims to utilise an automated source identification and measurement approach that is demonstrably optimal, to maximise the reliability, utility and robustness of the resulting radio source catalogues. A key stage in source extraction methods is the background estimation (background level and noise level) and the choice of a threshold high enough to reject false sources, yet not so high that the catalogues are significantly incomplete. In this analysis, we present results from testing the SExtractor, Selavy (Duchamp), and SFIND source extraction tools on simulated data. In particular, the effects of background estimation, threshold and false-discovery rate settings are explored. For parameters that give similar completeness, we find the false-discovery rate method employed by SFIND results in a more reliable catalogue compared to the peak threshold methods of SExtractor and Selavy.

Today, image denoising by thresholding of wavelet coefficients is a commonly used tool for 2D image enhancement. Since the data product of spectroscopic imaging surveys has two spatial dimensions and one spectral dimension, the techniques for denoising have to be adapted to this change in dimensionality. In this paper we will review the basic method of denoising data by thresholding wavelet coefficients and implement a 2D–1D wavelet decomposition to obtain an efficient way of denoising spectroscopic data cubes. We conduct different simulations to evaluate the usefulness of the algorithm as part of a source finding pipeline.

The spectral line datacubes obtained from the Square Kilometre Array (SKA) and its precursors, such as the Australian SKA Pathfinder (ASKAP), will be sufficiently large to necessitate automated detection and parametrisation of sources. Matched filtering is widely acknowledged as the best possible method for the automated detection of sources. This paper presents the Characterised Noise Hi (cnhi) source finder, which employs a novel implementation of matched filtering. This implementation is optimised for the 3-D nature of the Hi spectral line observations of the planned Wide-field ASKAP Legacy L-band All-sky Blind surveY (WALLABY). The CNHI source finder also employs a novel sparse representation of 3-D objects, with a high compression rate, to implement the Lutz one-pass algorithm on datacubes that are too large to process in a single pass. WALLABY will use ASKAP's phenomenal 30 square degree field of view to image ∼70% of the sky. It is expected that WALLABY will find 500 000 Hi galaxies out to z ∼0.2.

The H2O Southern Galactic Plane Survey (HOPS) has observed 100 deg2 of the Galactic plane, using the Mopra radio telescope to search for emission from multiple spectral lines in the 12-mm band (19.5–27.5 GHz). Perhaps the most important of these spectral lines is the 22.2-GHz water-maser transition. We describe the methods used to identify water-maser candidates and subsequent confirmation of the sources. Our methods involve a simple determination of likely candidates by searching peak emission maps, utilising the intrinsic nature of water-maser emission, spatially unresolved and spectrally narrow-lined. We estimate completeness limits and compare our method with results from the duchamp source finder. We find that the two methods perform similarly. We conclude that the similarity in performance is due to the intrinsic limitation of the noise characteristics of the data. The advantages of our method are that it is slightly more efficient in eliminating spurious detections and is simple to implement. The disadvantage is that it is a manual method of finding sources and so is not practical on datasets much larger than HOPS, or for datasets with extended emission that needs to be characterised. We outline a two-stage method for the most efficient means of finding masers, using duchamp.

We discuss the optimal detection of point sources from multiwavelength imaging data using an approach, referred to as MDET, which requires no prior knowledge of the source spectrum. MDET may be regarded as a somewhat more general version of the so-called ‘chi-squared’ technique. We describe the theoretical basis of the technique, and show examples of its performance with four-channel infrared broadband imaging data from the WISE mission. We also discuss the potential benefits of applying it to the multifrequency data cubes of the ASKAP surveys, and suggest that it could increase the detection sensitivity of searches for neutral hydrogen emission at moderately high redshifts.

This paper presents and discusses the results of basic source finding tests in three dimensions (using spectroscopic data cubes) with duchamp, the standard source finder for the Australian Square Kilometre Array Pathfinder. For this purpose, we generated different sets of unresolved and extended Hi model sources. These models were then fed into duchamp, using a range of different parameters and methods provided by the software. The main aim of the tests was to study the performance of duchamp on sources with different parameters and morphologies and assess the accuracy of duchamp's source parametrisation. Overall, we find duchamp to be a powerful source finder capable of reliably detecting sources down to low signal-to-noise ratios and accurately measuring their position and velocity. In the presence of noise in the data, duchamp's measurements of basic source parameters, such as spectral line width and integrated flux, are affected by systematic errors. These errors are a consequence of the effect of noise on the specific algorithms used by duchamp for measuring source parameters in combination with the fact that the software only takes into account pixels above a given flux threshold and hence misses part of the flux. In scientific applications of duchamp these systematic errors would have to be corrected for. Alternatively, duchamp could be used as a source finder only, and source parametrisation could be done in a second step using more sophisticated parametrisation algorithms.

We describe a simple method to determine the reliability of source finders based on the detection of sources with both positive and negative total flux. Under the assumption that the noise is symmetric and that real sources have positive total flux, negative detections can be used to assign to each positive detection a probability of being real. We discuss this method in the context of upcoming, interferometric Hi surveys.

This work presents a method for determining the accuracy of a source finder algorithm for spectral line radio astronomy data and the Source Finder Accuracy Evaluator (SFAE), a program that implements this method. The accuracy of a source finder is defined in terms of its completeness, reliability, and accuracy of the parameterisation of the sources that were found. These values are calculated by executing the source finder on an image with a known source catalogue, then comparing the output of the source finder to the known catalogue. The intended uses of SFAE include determining the most accurate source finders for use in a survey, determining the types of radio sources a particular source finder is capable of accurately locating, and identifying optimum parameters and areas of improvement for these algorithms. This paper demonstrates a sample of accuracy information that can be obtained through this method, using a simulated ASKAP data cube and the duchamp source finder.

While automatic detection of point sources in astronomical images has experienced a great degree of success, less effort has been directed towards the detection of extended and low-surface-brightness features. At present, existing telescopes still rely on human expertise to reduce the raw data to usable images and then to analyse the images for non-pointlike objects. However, the next generation of radio telescopes will generate unprecedented volumes of data making manual data reduction and object extraction infeasible. Without developing new methods of automatic detection for extended and diffuse objects such as supernova remnants, bent-tailed galaxies, radio relics and halos, a wealth of scientifically important results will not be uncovered. In this paper we explore the response of the Circle Hough Transform to a representative sample of different extended circular or arc-like astronomical objects. We also examine the response of the Circle Hough Transform to input images containing noise alone and inputs including point sources.

The large size of the ASKAP Hi surveys DINGO and WALLABY necessitates automated 3D source finding. A performance difference of a few percent corresponds to a significant number of galaxies being detected or undetected. As such, the performance of the automated source finding is of paramount importance to both of these surveys. We have analysed the performance of various source finders to determine which will allow us to meet our survey goals during the DINGO and WALLABY design studies. Here we present a comparison of the performance of five different methods of automated source finding. These source finders are duchamp, gamma-finder, a cnhi finder, a 2d–1d wavelet reconstruction finder and a sigma clipping method (s + c finder). Each source finder was applied to the same three-dimensional data cubes containing (a) point sources with a Gaussian velocity profile and (b) spatially extended model-galaxies with inclinations and rotation profiles. We focus on the completeness and reliability of each algorithm when comparing the performance of the different source finders.

We present a framework to volume-render three-dimensional data cubes interactively using distributed ray-casting and volume-bricking over a cluster of workstations powered by one or more graphics processing units (GPUs) and a multi-core central processing unit (CPU). The main design target for this framework is to provide an in-core visualization solution able to provide three-dimensional interactive views of terabyte-sized data cubes. We tested the presented framework using a computing cluster comprising 64 nodes with a total of 128 GPUs. The framework proved to be scalable to render a 204 GB data cube with an average of 30 frames per second. Our performance analyses also compare the use of NVIDIA Tesla 1060 and 2050 GPU architectures and the effect of increasing the visualization output resolution on the rendering performance. Although our initial focus, as shown in the examples presented in this work, is volume rendering of spectral data cubes from radio astronomy, we contend that our approach has applicability to other disciplines where close to real-time volume rendering of terabyte-order three-dimensional data sets is a requirement.

The Square Kilometre Array and its pathfinders ASKAP and MeerKAT will produce prodigious amounts of data that necessitate automated source finding. The performance of automated source finders can be improved by pre-processing a dataset. In preparation for the WALLABY and DINGO surveys, we have used a test Hi datacube constructed from actual Westerbork Telescope noise and WHISP Hi galaxies to test the real world improvement of linear smoothing, the Duchamp source finder's wavelet denoising, iterative median smoothing and mathematical morphology subtraction, on intensity threshold source finding of spectral line datasets. To compare these pre-processing methods we have generated completeness-reliability performance curves for each method and a range of input parameters. We find that iterative median smoothing produces the best source finding results for ASKAP Hi spectral line observations, but wavelet denoising is a safer pre-processing technique. In this paper we also present our implementations of iterative median smoothing and mathematical morphology subtraction.

Here I will outline successes and challenges for finding spectral line sources in large data cubes that are dominated by noise. This is a 3D challenge as the sources we wish to catalog are spread over several spatial pixels and spectral channels. While 2D searches can be applied, e.g. channel by channel, optimal searches take into account the 3-dimensional nature of the sources. In this overview I will focus on Hi 21-cm spectral line source detection in extragalactic surveys, in particular HIPASS, the Hi Parkes All-Sky Survey and WALLABY, the ASKAP Hi All-Sky Survey. I use the original HIPASS data to highlight the diversity of spectral signatures of galaxies and gaseous clouds, both in emission and absorption. Among others, I report the discovery of a 680 km s−1 wide Hi absorption trough in the megamaser galaxy NGC 5793. Issues such as source confusion and baseline ripples, typically encountered in single-dish Hi surveys, are much reduced in interferometric Hi surveys. Several large Hi emission and absorption surveys are planned for the Australian Square Kilometre Array Pathfinder (ASKAP): here we focus on WALLABY, the 21-cm survey of the sky (δ < + 30°; z < 0.26) which will take about one year of observing time with ASKAP. Novel phased array feeds (‘radio cameras’) will provide 30 deg2 instantaneous field-of-view. WALLABY is expected to detect more than 500 000 galaxies, unveil their large-scale structures and cosmological parameters, detect their extended, low-surface-brightness disks as well as gas streams and filaments between galaxies. It is a precursor for future Hi surveys with SKA Phase I and II, exploring galaxy formation and evolution. The compilation of highly reliable and complete source catalogs will require sophisticated source-finding algorithms as well as accurate source parametrisation.

The Australian Square Kilometre Array Pathfinder (ASKAP) presents a number of challenges in the area of source finding and cataloguing. The data rates and image sizes are very large, and require automated processing in a high-performance computing environment. This requires development of new tools, that are able to operate in such an environment and can reliably handle large datasets. These tools must also be able to accommodate the different types of observations ASKAP will make: continuum imaging, spectral-line imaging, transient imaging. The ASKAP project has developed a source-finder known as selavy, built upon the duchamp source-finder. selavy incorporates a number of new features, which we describe here.

Since distributed processing of large images and cubes will be essential, we describe the algorithms used to distribute the data, find an appropriate threshold and search to that threshold and form the final source catalogue. We describe the algorithm used to define a varying threshold that responds to the local, rather than global, noise conditions, and provide examples of its use. And we discuss the approach used to apply two-dimensional fits to detected sources, enabling more accurate parameterisation. These new features are compared for timing performance, where we show that their impact on the pipeline processing will be small, providing room for enhanced algorithms.

We also discuss the development process for ASKAP source finding software. By the time of ASKAP operations, the ASKAP science community, through the Survey Science Projects, will have contributed important elements of the source finding pipeline, and the mechanisms in which this will be done are presented.