1. Introduction
Neutral atomic hydrogen (H i) is a fundamental ingredient in cosmic star formation and galaxy evolution and is crucial to understanding the cosmic baryon cycle (Péroux & Howk, Reference Péroux and Howk2020).
For nearby galaxies, large-area surveys for 21 cm H i emission like HIPASS (Koribalski et al., Reference Koribalski, Staveley-Smith and Kilborn2004; Meyer et al., Reference Meyer, Zwaan and Webster2004), ALFALFA (Haynes et al., Reference Haynes, Giovanelli and Martin2011), and WALLABY (Koribalski, Reference Koribalski2012; Koribalski et al., Reference Koribalski, Staveley-Smith and Westmeier2020; Westmeier et al., Reference Westmeier, Deg and Spekkens2022), combined with targeted surveys (e.g. Walter et al., Reference Walter, Brinks and de Blok2008; Chung et al., Reference Chung, van Gorkom, Kenney, Crowl and Vollmer2009; de Blok et al., Reference de Blok, Healy and Maccagni2024) provide a comprehensive picture of the typical H i content of galaxies, the physical state of the neutral gas, and its relationship to the galaxy environment and star formation rate.
For galaxies beyond the local (
$z \lt 0.1$
) Universe, the faintness of the 21 cm line means that very long integration times are needed to detect H i emission from even the most gas-rich galaxies. A further limitation for H i studies at
$0.09 \lt z \lt 0.2$
is the presence of strong radio frequency interference (RFI) from the Global Navigation Satellite Systems (GNSS). In spite of these challenges, direct detections of H i emission at
$z\gt0.2$
have been made in several surveys, including BUDHIES (Jaffé et al., Reference Jaffé, Poggianti, Verheijen, Deshev and van Gorkom2013), HIGHz (Catinella & Cortese, Reference Catinella and Cortese2015), CHILES (Fernández et al., Reference Fernández, Gim and van Gorkom2016; Hess et al., Reference Hess, Luber and Fernández2019) and FUDS (Xi et al., Reference Xi, Peng, Staveley-Smith, For and Liu2022, Reference Xi, Peng and Staveley-Smith2024). In addition, Chakraborty & Roy (Reference Chakraborty and Roy2023) have detected H i emission from a star-forming galaxy at
$z\sim1.3$
that was lensed by a foreground galaxy along the line of sight.
1.1 Neutral hydrogen in the distant Universe
Most of our current knowledge about H i in the distant Universe comes from studies of the redshifted 1215.7 Å Lyman-alpha (Ly
$\alpha$
) line detected in absorption against background quasars (Wolfe et al., Reference Wolfe, Turnshek, Smith and Cohen1986; Wolfe et al., Reference Wolfe, Gawiser and Prochaska2005). Large ground-based quasar absorption-line surveys (e.g. Noterdaeme et al., Reference Noterdaeme, Petitjean and Carithers2012; Zafar et al., Reference Zafar, Péroux and Popping2013; Sánchez-Ram rez et al., 2016) have enabled measurements of the cosmic H i mass density (
$\Omega_\textrm{HI}$
) over a wide redshift range at
$z\geq1.7$
, as shown in Figure 1.
At redshift
$z\lt1.7$
however, the Ly
$\alpha$
line lies in the ultraviolet part of the spectrum where it is only observable with space-based telescopes. As a result, samples of Ly
$\alpha$
absorbers at
$z\lt1.7$
remain small (Neeleman et al., Reference Neeleman, Prochaska and Ribaudo2016; Rao et al., Reference Rao, Turnshek, Sardane and Monier2017) and (as can be seen in Figure 1), our knowledge of the amount and distribution of H i in the redshift range
$0.3\lt z\lt1.7$
, a timespan of 6.5 Gyr, or almost half the age of the Universe, is patchy and highly incomplete.
21 cm emission-line stacking techniques can also provide estimates of
$\Omega_\textrm{HI}$
beyond the local Universe (Lah et al., Reference Lah, Chengalur and Briggs2007; Delhaize et al., Reference Delhaize, Meyer, Staveley-Smith and Boyle2013; Rhee et al., Reference Rhee, Lah and Briggs2018; Chowdhury et al., Reference Chowdhury, Kanekar, Chengalur, Sethi and Dwarakanath2020; Chowdhury et al., Reference Chowdhury, Kanekar and Chengalur2022; Sinigaglia et al., Reference Sinigaglia, Elson, Rodighiero and Vaccari2022; Bera et al., Reference Bera, Kanekar, Chengalur and Bagla2023). Such measurements can give valuable insights into the H i properties of galaxies at different cosmic epochs but they provide statistical measures rather than information about the properties of individual galaxies.
As discussed by recent studies (e.g. Sadler et al., Reference Sadler, Moss and Allison2020; Allison et al., Reference Allison, Sadler and Amaral2022; Deka et al., 2024), the 21 cm H i absorption line provides an alternative tool for probing the cold gas content of galaxies beyond the local Universe, since the strength of the absorption line is independent of the distance to the absorber. The 21 cm line is also unaffected by dust extinction, and surveys for 21 cm H i absorption can be carried out over a wide redshift range if a sufficiently large sample of bright background radio sources is used. The optical depth of the 21 cm H i absorption line is inversely related to the gas spin temperature T
$_\textrm{S}$
, so radio H i absorption surveys are most sensitive to gas within the cold neutral medium of galaxies with T
$_\textrm{S} \sim100$
K (Braun, Reference Braun2012). This cold component is likely to be an effective tracer of star formation in galaxies across cosmic time (e.g. Kanekar et al., Reference Kanekar, Prochaska and Smette2014).
1.2 Surveys for intervening 21 cm H i absorption
Until recently, almost all 21 cm H i absorption searches were targeted in redshift, using some form of optical pre-selection such as the detection of Mg ii absorption lines (Briggs & Wolfe, Reference Briggs and Wolfe1983) or the presence of a known damped Ly
$\alpha$
(DLA) system (York et al., Reference York, Kanekar, Ellison and Pettini2007; Kanekar et al., Reference Kanekar, Prochaska and Smette2014).
As a result, existing samples of intervening H i absorption systems (which trace cold gas in ‘normal’ galaxies along the line of sight to the background source) are relatively small and may suffer from selection effects that are difficult to quantify. In particular, the equivalent width of the MgII absorption line mainly measures kinematic spread rather than gas density (Rao et al., Reference Rao, Turnshek, Sardane and Monier2017) so there is no simple relationship between H i column density and MgII absorption-line strength in DLA systems (Ellison, Reference Ellison2006; Ellison et al., Reference Ellison, Murphy and Dessauges-Zavadsky2009; Dessauges-Zavadsky et al., Reference Dessauges-Zavadsky, Ellison and Murphy2009). The use of optical pre-selection may also introduce a bias against sightlines through dusty, gas-rich galaxies (e.g. Fall & Pei, Reference Fall and Pei1993; Carilli et al., Reference Carilli, Menten, Reid, Rupen and Yun1998).
In a targeted search with the Arecibo telescope using pre-selection based on Mg ii absorption, Briggs & Wolfe (Reference Briggs and Wolfe1983) detected two H i lines from their sample of 18 Mg ii systems at
$0.36\lt z\lt1.94$
. They found no correlation between Mg ii equivalent width and H i optical depth, and suggested that the lack of an observed correlation between the optical and radio absorption properties was related to the multi-component nature of the absorbing gas.
Kanekar et al. (Reference Kanekar, Prochaska and Smette2014) used the Green Bank Telescope (GBT) to search for 21 cm H i absorption at the redshift of 22 known quasar DLA systems with
$0.8\lt z\lt3.5$
and made three detections. By combining the 21 cm and optical DLA data, these authors were able to derive spin temperatures (or lower limits) for each system. They found statistically significant evidence for an increase in the spin temperature of DLA systems at higher redshift. Other targeted searches, mainly using Mg ii pre-selection (Lane et al., Reference Lane, Smette and Briggs1998; Lane & Briggs, Reference Lane and Briggs2001; Gupta et al., Reference Gupta, Srianand and Petitjean2012; Kanekar et al., Reference Kanekar, Prochaska and Smette2014; Dutta et al., Reference Dutta, Raghunathan, Gupta and Joshi2020) have also produced a small number of intervening 21 cm detections.
Cosmic H i gas density
$\Omega_\textrm{HI}$
as a function of redshift (bottom axis) and lookback time (top axis), adapted from Rhee et al. (Reference Rhee, Lah and Briggs2018); Rhee et al. (Reference Rhee, Meyer and Popping2023). This plot uses damped Ly
$\alpha$
absorption (
$0.1 \lt z \lt 5$
) and H i stacking (up to
$z \sim 1$
) and direct detection of H i emission at lower-z. The redshift range observed in the FLASH survey is shown by light red shading. The black solid line and background grey shade show a weighted linear fit of all
$\Omega_\textrm{HI}$
measurements and its 95 per cent confidence interval. The orange dashed line is a power-law fit. The predictions from semi-analytic models (Kim et al., Reference Kim, Wyithe and Power2015) and cosmological hydrodynamical simulations (Davé et al., Reference Davé, Rafieferantsoa, Thompson and Hopkins2017) are denoted with blue solid line and red dashed line, respectively. Further details on the observational data points used for this plot, as well as relevant references, are provided in Figure 14 of Rhee et al. (Reference Rhee, Lah and Briggs2018) and Figure 25 of Rhee et al. (Reference Rhee, Meyer and Popping2023).
A few intervening H i absorption detections have been made in untargeted surveys with the GBT in the redshift range
$0.3\lt z\lt0.8$
(Brown & Roberts, Reference Brown and Roberts1973; Brown & Spencer, Reference Brown and Spencer1979; Brown & Mitchell, Reference Brown and Mitchell1983; Darling et al., Reference Darling, Giovanelli, Haynes, Bolatto and Bower2004). More recently, Grasha et al. (Reference Grasha, Darling, Leroy and Bolatto2020) used the GBT to search for H i absorption against 260 radio sources in the redshift range
$0\lt z\lt2.74$
. They re-detected ten known absorption systems, but made no new detections. From their results, Grasha et al. (Reference Grasha, Darling, Leroy and Bolatto2020) derived measurements of
$\Omega_\textrm{HI}$
consistent with other methods, inferring a relatively mild evolution in H i mass density over the redshift range
$0\lt z\lt2.74$
.
1.3 Surveys for associated 21 cm H i absorption
The detection of cold gas within the host galaxies of radio-loud AGN (“associated H i absorption”) can provide unique insights into the distribution and kinematics of gas in the nuclear regions of active galaxies and the role gas may play in the evolution of AGN (Morganti & Oosterloo, Reference Morganti and Oosterloo2018). In particular, the radio jets in these systems may drive rapid outflows of neutral gas (e.g. Morganti et al., Reference Morganti, Tadhunter and Oosterloo2005; Morganti et al., Reference Morganti, Veilleux, Oosterloo, Teng and Rupke2016; Schulz et al., Reference Schulz, Morganti and Nyland2021). Previous searches for associated 21 cm absorption lines have generally used a known optical redshift to target a specific radio frequency. Most of these searches have been carried out at low redshift, though some higher-redshift lines have also been detected.
Large and sensitive targeted searches for associated H i absorption in nearby radio galaxies have been carried out by Geréb et al. (Reference Geréb, Morganti and Oosterloo2014) and Maccagni et al. (Reference Maccagni, Morganti, Oosterloo, Geréb and Maddox2017), with detection rates of up to 30%. Recent studies have expand these searches to investigate gas in the environments of radio-loud AGN at higher redshifts, reaching beyond
$z \sim 3$
(e.g. Kanekar et al., Reference Kanekar, Chengalur and Lane2007; Srianand et al., Reference Srianand, Gupta and Petitjean2012; Aditya & Kanekar, Reference Aditya and Kanekar2018a; Aditya & Kanekar, Reference Aditya and Kanekar2018b; Aditya, Reference Aditya2019; Murthy et al., Reference Murthy, Morganti, Kanekar and Oosterloo2022, and references therein).
The results so far suggest that the detection rate of associated H i absorption lines is higher in compact radio sources than in extended sources (Vermeulen et al., Reference Vermeulen, Pihlström and Tschager2003; Gupta et al., Reference Gupta, Srianand and Baan2016), and appears to be lower at redshift
$z\gt0.5$
than in nearby galaxies (Aditya & Kanekar, Reference Aditya and Kanekar2018b; Su et al., Reference Su, Sadler and Allison2022; Aditya et al., Reference Aditya, Yoon and Allison2024). However, the detection rate at high redshift appears to be influenced by the morphology of the sources. Most H i detections at high-z are associated with peaked-spectrum sources (Aditya & Kanekar, Reference Aditya and Kanekar2018a; Glowacki et al., Reference Glowacki, Allison and Moss2019; Aditya et al., Reference Aditya, Yoon and Allison2024), whereas compact, flat-spectrum quasars exhibit a lower detection rate (Aditya & Kanekar, Reference Aditya and Kanekar2018b).
Curran & Whiting (2012) postulate that there is a critical UV continuum luminosity of L
$_\textrm{1216} \sim 10^{23}$
W Hz
$^{-1}$
above which neutral hydrogen is completely ionised and 21 cm absorption is no longer detected. They suggest that this may partly account for the lower H i detection rate at higher redshifts where powerful radio-loud quasars are more common.
More recently, Murthy et al. (Reference Murthy, Morganti, Kanekar and Oosterloo2022) investigated selected targets where the UV luminosity fell below the threshold of 10
$^{23}$
W Hz
$^{-1}$
, finding that neither UV nor radio luminosity of the AGNs is likely to cause the lower detection rate. This work suggests that the lower detection rate of associated H i 21 cm absorption lines at high redshifts may be due to evolution in the physical conditions of H i, possibly either lower H i column densities or higher spin temperatures in high-z AGN environment. There have also been studies of associated absorption in Ly
$\alpha$
, i.e. the proximate DLAs (PDLAs), which do indeed seem to be more common towards radio selected QSOs (e.g. Ellison et al., Reference Ellison, Yan and Hook2002; Ellison et al., Reference Ellison, Prochaska and Hennawi2010; Russell et al., Reference Russell, Ellison and Benn2006).
1.4 H i absorption studies with SKA precursor telescopes
A new parameter space for H i absorption studies has recently been opened up by the development of wide-band correlators for radio interferometers that provide instantaneous redshift coverage approaching that of optical spectrographs, and by the construction of the SKA precursor telescopes ASKAP (Johnston et al., Reference Johnston, Taylor and Bailes2008; Hotan et al., Reference Hotan, Bunton and Chippendale2021) and MeerKAT (Jonas, Reference Jonas2009) on radio-quiet sites where RFI contamination is minimised. Importantly, these radio-quiet sites uniquely allow near-continuous coverage of the 21 cm line at frequencies between 0.5 and 1 GHz, enabling spectral -line studies of H i at
$0.4\lt z\lt1.8$
(e.g. Allison et al., Reference Allison, Moss and Macquart2017; Deka et al., Reference Deka, Gupta and Chen2023).
The MeerKAT Absorption-Line Survey (MALS; Gupta et al., Reference Gupta, Srianand and Baan2016) is currently conducting a search for H i absorption lines at redshift
$0\lt z\lt1.45$
in several hundred fields (each of area
$\sim1\,\textrm{deg}^2$
, the total sky coverage of
$\sim1000\,\textrm{deg}^2$
) centred on bright radio-loud quasars with a known optical redshift (Gupta et al., Reference Gupta, Shukla and Srianand2022). This survey is optimised for lines with low H i column densities, reaching limits as low as N
$_\textrm{HI} \sim 10^{19}$
cm
$^{-2}$
for the brightest background sources and N
$_\textrm{HI} \sim 10^{21}$
cm
$^{-2}$
for the other sources in general.
The wide (36 deg
$^2$
) field of view, broad spectral coverage and radio-quiet site of the Australian SKA Pathfinder (ASKAP) radio telescope (Hotan et al., Reference Hotan, Bunton and Chippendale2021) makes it possible to carry out the first ‘all-sky’ survey for 21 cm H i absorption across a wide redshift range (
$0\lt z\lt1.0$
) without any optical pre-selection of targets. The design and science goals for such a survey, the ASKAP First Large Absorption Survey in H i (FLASH) are described in detail in the survey design paper by Allison et al. (Reference Allison, Sadler and Amaral2022). In contrast to the Meerkat MALS survey, FLASH is optimised for the detection of high column-density H i absorption lines (typically with N
$_\textrm{HI} \gt 10^{21}$
cm
$^{-2}$
) against relatively faint radio sources.
The first H i absorption-line searches with ASKAP (Allison et al., Reference Allison, Sadler and Moss2015; Moss et al., Reference Moss, Allison and Sadler2017; Glowacki et al., Reference Glowacki, Allison and Moss2019; Sadler et al., Reference Sadler, Moss and Allison2020; Mahony et al., Reference Mahony, Allison and Sadler2022) used a smaller array of 6–12 ASKAP dishes with a single bright continuum source placed near the field centre. These observations detected a variety of associated and intervening H i absorption lines, and showed that ASKAP can produce high-quality radio spectra that are essentially free of terrestrial RFI in the 700–1000 MHz band.
Allison et al. (Reference Allison, Sadler and Bellstedt2020) presented the first ASKAP detection of H i absorption against a source away from the field centre in a wide-field observation with 12-14 dishes (the GAMA 23 Early Science field). Like the earlier ASKAP observations, the GAMA23 data were processed with a custom data pipeline based around the Miriad software package (Sault et al., Reference Sault, Teuben, Wright, Shaw, Payne and Hayes1995). The much larger datasets produced by the full 36-dish ASKAP array require the use of a dedicated observatory-built data pipeline, ASKAPsoft (Wieringa et al., Reference Wieringa, Raja, Ord, Pizzo, Deul, Mol, de Plaa and Verkouter2020), and so a key aim of the ASKAP Pilot Survey program was to test and verify this pipeline.
The huge size of the ASKAP spectral-line datasets means that spectral-line visibilities cannot be stored once the data have been processed through ASKAPsoft (Hotan et al., Reference Hotan, Bunton and Chippendale2021), so re-processing of the raw data is not possible. Testing and verification of the full ASKAPsoft data pipeline for FLASH data is therefore essential, and this was achieved through the ASKAP Pilot Survey program described in this paper.
1.5 Outline of this paper
Section 2 of this paper describes the two ASKAP FLASH pilot surveys carried out from 2019–22, which together covered around 3,000 deg
$^2$
of sky. Section 3 gives details of the fields observed, Section 4 describes the data processing and Section 5 discusses the data products released through the CSIRO ASKAP Science Data Archive (CASDA)Footnote
a
. Section 6 briefly describes the continuum images and catalogues released from the pilot surveys, while Section 7 discusses the overall quality of the processed spectral-line data released in CASDA.
Section 8 presents the results from the FLASHfinder, an automated tool to search for H i absorption and discusses machine learning classifications of the detected lines. A Discussion section, an outline of future work, and a brief summary can be found in Sections 9 and 10. Some further technical details are included in the Appendices, and plots of the detected H i lines are shown in 14.
2. The ASKAP-FLASH pilot surveys
A program of ASKAP pilot surveys was carried out before the start of the full set of five-year ASKAP surveys. Along with other ASKAP Survey Science teams, the FLASH team was allocated 100 h of observing time for a first round of pilot surveys in 2019–20 and a further 100 h for the second pilot survey round in 2021–22.
2.1 FLASH survey strategy and data pipeline
The FLASH pilot surveys had two main goals: to test and validate the observing and data processing strategies for the full all-sky FLASH survey, and to provide a first look at the population of H i absorption systems detected in a wide-field radio absorption-line survey conducted without any optical pre-selection.
In the pilot surveys, we carried out end-to-end tests and validation of the ASKAPsoft data pipeline to ensure that it was able to produce the required spectra, continuum images and catalogues to the quality specified in the original FLASH survey proposal as outlined by Allison et al. (Reference Allison, Sadler and Amaral2022). We also assessed the variation in sensitivity across the full ASKAP field of view at 711.5-999.5 MHz and measured the system performance and noise properties over a wide range in declination (from at least
$+20$
deg to
$-50$
deg declination) and for both daytime and nighttime observations. Figure 2 shows the sky coverage of the survey.
FLASH Pilot Survey Sky Coverage. Orange squares show the Pilot Survey 1 fields, and green squares the Pilot Survey 2 fields. Fields observed in both Pilot 1 and 2 are marked with star symbols. The coordinates are equatorial, with right ascension versus declination. The footprints of multi-wavelength surveys, such as BOSS, DES, MRC 1Jy, Stripe 82, WiggleZ, VIKING, and SPT-SZ, are included.
2.2 Choice of observing parameters
The choice of observing frequency for FLASH is a trade-off between the frequency-dependent sensitivity of the ASKAP telescope and our desire to optimise the redshift path-length for detecting H i absorption. This trade-off is discussed in detail by Allison et al. (Reference Allison, Sadler and Amaral2022), who show that (i) a frequency range of 711.5–999.5 MHz maximises the total absorption path length sampled, and (ii) an integration time of around 2 h per field maximises the sky area that can be covered to reasonable depth in a fixed time, which in turn maximises the number of absorption lines detected and the discovery potential of the survey.
Table 1 lists the observing parameters used for the FLASH pilot survey. Individual observations used the grid of fields and pointing centres listed in Table 2 of Allison et al. (Reference Allison, Sadler and Amaral2022), which is designed to provide uniform sensitivity across the survey area and is the same grid used by the first RACS low survey (The Rapid ASKAP Continuum Survey; McConnell et al., Reference McConnell, Hale and Lenc2020). We used the square_6
$\times$
6 ASKAP beam footprint (Hotan et al., Reference Hotan, Bunton and Chippendale2021) shown in Figure 3. In the frequency range used for FLASH, this provides an rms spectral-line noise that is uniform across the central 6.4 deg
$\times$
6.4 deg of the ASKAP field of view, as also shown in Figure 3.
2.3 Spectral-line beamforming intervals with ASKAP
ASKAP’s Phased Array Feeds allow us to digitally form up to 36 beams across the field of view. By default, the beam-forming weights are calculated in 1 MHz intervals across the full 288 MHz bandwidth (see Hotan et al. (Reference Hotan, Bunton and Chippendale2021) for further details), which is prone to produce bandpass imperfections at the same frequency interval. At the redshifts covered by the FLASH survey this corresponds to velocity widths of 235–420 km s
$^{-1}$
which would severely limit our ability to detect broader H i absorption lines.
To overcome this, fixing the beam-weights over larger bandwidths is required. FLASH pilot survey observations carried out observations using both 5 and 9 MHz beam-forming intervals to assess which mode was most suitable moving forward. The majority of Pilot 1 fields used 9 MHz beamforming intervals with the exception of SBIDs (Scheduling block IDs) 10849, 10850, 11051, 11052, 11053, 11068 which used 5 MHz intervals. Observations using 9 MHz intervals showed larger phase jumps between neighbouring intervals and required higher order polynomials for the bandpass smoothing so it was concluded that 5 MHz intervals provided the optimum observing set-up for all Pilot Survey 2 observations and future FLASH observations.
3. Observations
3.1 Pilot Survey 1: 2019–20
The first FLASH pilot survey, covering
$\sim1510$
deg
$^2$
of sky was carried out with the full 36-dish ASKAP telescope between December 2019 and September 2020. We observed 40 fields with an integration time of 2 h each (the 2-hr fields). For six of these fields, we also carried out a longer (6 hr) integration (the 6-hr fields). The observed fields were chosen to span a wide range in declination, but also to include some areas of the sky with good quality optical data.
In Pilot Survey 1, 37 of the 2-hr fields and four 6-hr fields were successfully processed through the ASKAPsoft data pipeline and released in CASDA. One of these fields (listed as J2022-2507P in Table A2) was observed at the wrong RA because of a scheduling errorFootnote b , but still contains useful data. The fields released are shown in Figure 2 and are listed in Appendix F.
3.2 Pilot Survey 2: 2021–22
After some further refinements to the data processing pipeline, a second pilot survey, was carried out from November 2021 to August 2022. The fields observed in this second pilot survey are also shown in Figure 2 and are listed in Appendix F.
In Pilot Survey 2, we observed 50 fields with an integration time of 2 h each. Twelve of these were repeats of fields observed in Pilot 1, and were used to test the reliability and reproducibility of the absorption lines detected by ASKAP as discussed in Section 8.7. The other 38 fields were observed for the first time and and cover
$\sim1550$
deg
$^2$
of sky.
Observing parameters for the FLASH Pilot Surveys. For a detailed description of the ASKAP array, see Hotan et al. (Reference Hotan, Bunton and Chippendale2021).
Comparison of continuum properties for images from three ASKAP surveys: FLASH, RACS-Low, and EMU.
3.3 Sky areas covered by the two Pilot Surveys
The data released in CASDA from the two FLASH pilot surveys include 75 unique ASKAP fields with a total sky coverage of 3000 deg
$^2$
(see Figure 2).
The Pilot Survey fields include several well-observed regions with multi-wavelength photometric and spectroscopic information. These include:
-
• The three equatorial fields (at 9h, 12h and 15h RA) from the GAMA galaxy survey (Galaxy And Mass Assembly; Driver et al., Reference Driver, Hill and Kelvin2011), each covering 48 deg
$^2$
of sky. The GAMA fields were observed on non-standard field centres in the first pilot survey, so that each field was completely covered by two ASKAP pointings (Su et al., Reference Su, Sadler and Allison2022). In the second pilot survey, the GAMA fields were re-observed on the standard FLASH pointing centres listed by Allison et al. (Reference Allison, Sadler and Amaral2022). -
• All of SDSS Stripe 82 (Annis et al., Reference Annis, Soares-Santos and Strauss2014), a well-studied 300 deg
$^2$
region of sky with RA between 20h and 04h (300 to 60 deg) and declination between -1.26 and +1.26 deg. The available multi-wavelength data products include deep optical (Jiang et al., Reference Jiang, Fan and Bian2014,
$r \lt 24.6$
mag) and 1.4 GHz radio imaging (Hodge et al., Reference Hodge, Becker, White, Richards and Zeimann2011) as well as extensive optical spectroscopy. -
• Part of the WiggleZ (Drinkwater et al., Reference Drinkwater, Byrne and Blake2018) redshift survey area. WiggleZ targeted a UV-selected galaxy sample of candidate star-forming galaxies at redshift
$0.2\lt z\lt1.0$
across several fields covering a total of
$\sim1000$
deg
$^2$
of sky, so is well-matched to the redshift range covered by FLASH. Further analysis of the FLASH WiggleZ data is provided by Eden et al. (Reference Eden, Sadler, Pimbblet, Mahony and Yoon2025). -
• Part of the MRC 1Jy (McCarthy et al., Reference McCarthy, Kapahi and van Breugel1996) survey area, which covers much of the southern extragalactic sky between declination
$-20$
and
$-30$
degrees. An early analysis of FLASH data from the MRC 1Jy region is given by Aditya et al. (Reference Aditya, Yoon and Allison2024).
4. Data processing
4.1 Pipeline processing
The observations were processed through the standard ASKAPsoft pipeline Whiting (Reference Whiting, Ballester, Ibsen, Solar and Shortridge2020). This is a scripted workflow that, for FLASH pilot processing, ran on the galaxy supercomputer at the Pawsey Supercomputing CentreFootnote c . It performs all the necessary calibration, imaging and source-extraction tasks required to produce science-ready data products. A detailed description of the pipeline can be found elsewhere (e.g. Hotan et al., Reference Hotan, Bunton and Chippendale2021), but we give specific details relevant for the FLASH processing here.
The bandpass and primary flux calibration were done using PKS B1934
$-$
638, which was observed at the centre of each beam for
$\sim200$
sec. The bandpass solutions were derived for each beam, antenna, polarisation combination and smoothed over intervals of 1 MHz to improve the signal-to-noise of the calibration solutions. Each beam of the science observation was calibrated with the solutions from the corresponding beam of the bandpass observation.
All imaging of the science observations was done independently for each beam. A continuum image (using visibilities averaged to 1 MHz resolution) was made first, through an iterative self-calibration approach. This involves imaging, then calibrating the time-dependent complex gains against a model derived from the image, then re-imaging (with more cleaning) after applying those gain solutions. The imaging used multi-scale (scales of 0, 6, 15, 30, 45, 60 pixels) multi-frequency (2 Taylor-terms) synthesis, with Wiener preconditioning applied using robustness = 0.0. This gives an average PSF size (averaged over all Pilot observations) of
$18\times13$
arcsec. A second image was also made with a robustness = 1.0 (giving an average resolution of
$28\times20$
arcsec) to more closely match the spatial resolution of the spectral line cube.
The full-spectral resolution data were then imaged over the full band. First, the continuum was subtracted from the visibilities by forming a model from the continuum image and transforming to the (u,v) plane. After this, the data were imaged in each channel to form a spectral cube. Two cuts on the (u,v) distance were applied, by removing baselines
$\lt70$
m to limit solar interference, and
$\gt4000$
m to limit the size of the full image. Multi-scale imaging (scales 0, 3, 10, 30 pixels) was done, using Wiener preconditioning with robustness = 1.0 – this, in addition to the (u,v) cuts, provides an average PSF resolution in the middle of the band of
$29\times22$
arcsec, ranging from
$34\times36$
arcsec to
$25\times19$
arcsec across the band. Following imaging, a further round of continuum-subtraction was then applied, to further remove any continuum residuals. Each spectrum in the cube (that is, each spatial pixel) had the continuum level fitted within each beam-forming interval (or, in the case of Pilot Phase 1, each 1 MHz interval) and subtracted. Breaking at the edge of each beamforming interval allows the removal of any residual discontinuities not completely removed by the bandpass calibration (this is particularly important for brighter sources that might have a higher signal-to-noise than we get on B1934
$-$
638 in 200 sec).
The continuum (1 MHz-resolution) data were also imaged in each channel to form continuum cubes. For Pilot 2 these were done with the same (u,v) cuts and preconditioning as the spectral cubes, to provide matching-resolution data to enable more accurate extraction of continuum spectra in post-processing.
Once imaging was complete, the individual beams were mosaicked together to form the full field images in each of continuum, continuum-cube and spectral-cube modes. Source-finding was run with Selavy (Whiting & Humphreys, Reference Whiting and Humphreys2012) to create catalogues of continuum components and islands, and the full-spectral-resolution spectra at the location of each component brighter than 45 mJy (20 mJy for the 6-hour observations) were extracted and stored as 1-dimensional spectra.
FLASH field of view and noise properties: (left) a FLASH continuum image showing the position of the 36 ASKAP beams in the square_6
$\times$
6 foootprint used to form the image, (right) measurements of spectral-line noise for extracted spectra in a 6-hour Pilot 1 observation (SBID 13293), showing that the noise is roughly uniform within the central red dashed square, covering 6.4 deg
$\times$
6.4 deg on the sky (total area 40.9 deg
$^2$
).
4.2 Pilot 1 data and post-processing
Observations for the first FLASH pilot survey used 5 or 9 MHz beam-forming intervals instead of the standard 1 MHz (see Sec 2.3). However, during data processing we found that these larger intervals were not functioning as anticipated, still causing amplitude and phase jumps at 1 MHz intervals. As these jumps occurred precisely at the 1 MHz boundaries, they could be corrected for in the pipeline processing by smoothing the bandpass solutions on 1 MHz intervals as well as removing any residual artefacts by using 1 MHz intervals for the image-based continuum subtraction. This procedure corrects for the large amplitude jumps and leads to a clean, smooth bandpass across the entire band. However, 1 MHz corresponds to a line width of approximately 300 km s
$^{-1}$
at these redshifts, meaning that processing the data using 1 MHz intervals risks removing any real H i absorption signatures of similar line widths. This could mean that some H i absorption lines are either completely subtracted out or (if detected) the shape of the line is significantly altered because of the limited line-free channels used for the continuum subtraction in each 1 MHz interval.
An example of a broad H i absorption line detected after post-processing of the FLASH pilot survey data. The spectrum was extracted toward PKS 2311-477 in FLASH 160P field (SB 15873 component 5a,
$S = 1.04$
Jy). The grey shaded region is set at five times the rms spectral-line noise. The absorption is found at
$z = 0.5811$
, and the FWHM linewidth from a single Gaussian fit is 112 km s
$^{-1}$
.
To address this issue, we carried out some further post-processing outside of the ASKAPsoft pipeline. This involved downloading the spectral cubes from CASDA that were made prior to the image-based continuum subtraction, and extracting the spectra towards all bright radio sources. For the majority of the fields observed the flux limit for spectrum extraction was set at 30 mJy, which is deeper than the pipeline limit of 45 mJy. For the longer 6 hr integrations, a flux density threshold of 10 mJy was used. A spectrum was extracted at the peak pixel of each catalogued continuum component.
After extracting the spectra from the spectral cube, some further continuum subtraction was carried out to remove any residual continuum not subtracted in the visibility domain. This involved fitting a second-order polynomial within each 5 or 9 MHz beam-forming interval, while masking any strong lines (either due to real absorption or RFI) that were more than 3-
$\sigma$
above the noise.
This post-processing procedure allowed us to recover some H i absorption lines that were inadvertently subtracted out by using the default pipeline processing parameters, but did lead to some broader spectral ripples in some frequency ranges, particularly towards the brighter continuum sources. Figure 4 shows an example of a broad H i absorption line that was subtracted out in the initial processing, but was recovered after re-doing the continuum subtraction over wider frequency intervals. The post-processed data will be publicly released in CASDA alongside this publication.
4.3 Pilot 2 data
The overall data processing method for Pilot Survey 2 was similar to that for Pilot 1. Improvements in correctly implementing larger beam-forming intervals were made prior to the Pilot 2 survey, meaning that 5-MHz intervals were used for all Pilot-2 survey observations and post-processing of the data was not required.
Unfortunately, over half of the Pilot 2 fields show significant ripples in their spectra, caused by a processing error in the bandpass calibration for a large batch of observations made between 17 December 2021 and 15 January 2022 (see Section 7.2 for more details). These ripples are particularly noticeable in the spectra of bright sources, and the features occur across the whole frequency band.
Although 24 of the SBIDs with spectral ripples were re-observed after February 2022, more than half of these repeated fields were affected by tropospheric ducting of RFI (see Section 7.2) across at least 20% of the frequency range. These fields will be repeated in the full FLASH survey.
We are confident that the problems seen in the Pilot 2 spectra are unlikely to recur for the full FLASH survey. In particular, we now monitor for the presence of tropospheric ducting of RFI (which mainly affects the 700-1000 MHz ASKAP band) and take steps to avoid observing FLASH fields when ducting is present.
5. Data products released in CASDA
5.1 Processed data products
Processed data products from the FLASH pilot surveys have been released through the CSIRO ASKAP Science Data Archive (CASDA) at https://research.csiro.au/casda/ under project code AS 109. To download data, users need to register and create a CASDA account.
The output data products loaded into CASDA for each observation are listed with the examples of the filenames as follows:
-
1. Continuum catalogues: A component catalogue and a separate island catalogue are produced by the Selavy source finder (Whiting & Humphreys, Reference Whiting and Humphreys2012). These catalogues are used for data validation, and the component catalogue positions are also used to extract spectra for the absorption-line search. The island catalogues provide a useful resource for optical cross-matching.
$-$
File examples: “selavy-image.i.SB15873.cont.taylor.0.restored.conv.components.xml”, “selavy-image.i.SB15873.
cont.taylor.0.restored.conv.islands.xml”
-
2. Continuum images and cubes: We produce both a Stokes I continuum image at
$\sim15$
arcsec resolution (rms noise
$\sim90\,\mu$
Jy/beam) and a continuum cube. The continum cubes are matched to the 30 arcsec resolution of the spectral-line cube for the later Pilot fields.
$-$
File examples: “image.i.SB15873.cont.taylor.0.restored.conv.fits”, “image.restored.i.SB15873.contcube.conv.fits”
-
3. Spectral-line cubes: Two full spectral-line cubes are produced for each SBID. An initial spectral cube is created after continuum subtraction in the visibility domain, and a final spectral cube after the image-based continuum subtraction. This second cube is used to extract the individual H i spectra. Both cubes cover the full 288 MHz bandwidth at 18.5 kHz spectral resolution with all 36 beams mosaiced together to produce a single cube of the full ASKAP field.
$-$
File examples: “image.restored.i.SB15873.cube.contsub. fits” -
4. Individual spectra at positions of continuum sources: The ASKAPsoft pipeline extracts individual source and noise spectra for all radio components with flux above 45 mJy (20 mJy for 6-hr fields), which is the predicted detection limit by Allison et al. (Reference Allison, Sadler and Amaral2022). The spectra can be downloaded individually, or as a bulk.tar file for each SBID.
$-$
File examples: “spec
$\_$
SB15873
$\_$
component_1a.fits” -
5. Validation reports: These reports include metrics and general notes on data quality.
5.2 Data validation and quality assessment
The processed data products from the ASKAPsoft pipeline are assessed and validated by the FLASH science team before their public release. This data validation uses a checklist that takes into account the metrics provided by the ASKAPsoft pipeline (such as measurements of the rms noise in line and continuum), as well as a visual inspection of the images and a representative sample of spectra by members of the science team.
The completed checklist provides a numerical score, which the team use to classify each SBID as ‘Good’, ‘Uncertain’ or ‘Bad’. SBIDs classified as Good or Uncertain are released, but those classified as Bad remain unreleased and are flagged for reobservation. Among the released Pilot Survey data, 17 and 99 SBIDs are classified as ‘Good’ and ‘Uncertain’, respectively. The high fraction of ‘Uncertain’ is from the early observations requiring post-processing (Pilot 1; see Section 4.2) and the presence of artefacts (Pilot 2; see Section 7.2). The overall quality of the data are good for Pilot 1, and the artefacts observed in Pilot 2 have been mostly resolved for the Full Survey. The team also provide some brief Release Notes to accompany the CASDA data.
5.2.1 Continuum data validation
The validation data provided in CASDA for the continuum images include beam diagnostics, the spatial distribution of identified sources, an measurement of the rms noise in the continuum, and a flux density comparison with other surveys (NVSS, SUMSS or RACS). Astrometric checks are also carried out by measuring the median position offset from published surveys.
As part of the data validation process, the FLASH team checked that the continuum catalogues were present and contained a reasonable number of sources, typically between 10,000 and 20,000. We also visually inspected each continuum image to ensure that there were no obvious problems such as large-scale patterns or artefacts around strong sources.
5.2.2 Spectral-line data validation
In addition to the diagnostics provided by ASKAPsoft, the FLASH team verifies the presence of the image cubes, and the presence of several hundred spectra that are expected from each of our observations. We also measured the rms noise in the spectral line data both for the whole frequency range and across three sub-bands as shown in Figure 5, and checked that this was in line with expectations. Further details will be described in Section 7.
Measurements of median spectral-line noise (in mJy beam
$^{-1}$
ch
$^{-1}$
; channel width: 18.5 kHz) in three sub-bands of FLASH field 307 (SBID 13268), showing the values in each sub-band within the 712-1000 MHz observing band. Grey lines show the values for individual sources within 3.2 degrees of the field centre, and the dashed blue line shows the median for the field as a whole. The dotted red lines indicate the range of predicted noise across the entire band (3.2–5.1 mJy beam
$^{-1}$
ch
$^{-1}$
), as estimated by Allison et al. (Reference Allison, Sadler and Amaral2022).
Finally, as part of the post-processing workflow, we produce spectral-line plots like that shown in Figure 6. The extracted spectra of the ten brightest sources in each field are then inspected to check for any artefacts or non-uniformity of the bandpass.
An example of a spectral-line plot produced by the FLASH team during post-processing and used in data validation. In this figure, the gray band is expanded to five times the RMS spectral-line noise to clearly visualise the noise level across the band. The source shown here (NVSS J015516-251423, SB 37475 component 95a) has an H i absorption line at
$z=0.7251$
(
$f=823.4$
MHz), which extends beyond the gray band, indicating a peak S/N of > 5. (see Table 3).
6. Continuum images and catalogues
6.1 Wide field continuum images
FLASH observations provide high-quality continuum products, including wide field images and source catalogues, in addition to the spectral-line data. As can be seen from Table 2, the FLASH continuum images are intermediate in sensitivity between the first epoch of RACS-Low (McConnell et al., Reference McConnell, Hale and Lenc2020) and the EMU Pilot Survey (Norris et al., Reference Norris, Marvil and Collier2021).
The restored total-intensity (Stokes I) continuum images are accumulated over the entire bandwidth being processed. Within 3.2 deg from the field centre, the rms noise in the continuum images is uniform. Most continuum images are of excellent quality and revealed several radio sources with complex asymmetric morphologies.
The rms noise level in the 2-hour continuum images is typically
$\sim90\,\mu$
Jy beam
$^{-1}$
, i.e. roughly five times deeper than NVSS (450
$\mu$
Jy beam
$^{-1}$
) and an order of magnitude deeper than SUMSS (
$\sim1$
mJy beam
$^{-1}$
). The initially expected median image noise, estimated from the results of RACS-Low (McConnell et al., Reference McConnell, Hale and Lenc2020), is
$\sim93\,\mu$
Jy beam
$^{-1}$
(Allison et al., Reference Allison, Sadler and Amaral2022), which is close to our observed values.
6.2 Selavy radio source catalogues
As described in Section 5.1, the ASKAPsoft pipeline generates two different continuum source catalogues via the ASKAP source-finder Selavy:
-
(i) An island catalogue is generated first - this is a catalogue of groups of contiguous pixels that are above some detection threshold.
-
(ii) A component catalogue is then generated, where each component is a two-dimensional Gaussian, parameterised by a location, flux, size and orientation. Each island has one or more components fitted to it, so there is a one-to-many relationship between the island and component catalogues.
Both catalogues are available for download through CASDA.
6.3 Astrometric accuracy
The astrometric accuracy of the listed ASKAP continuum catalogues is determined by the combination of a statistical component (set by the S/N of the source and the angular resolution of the image), and a systematic component (set by the accuracy to which the measured source positions can be aligned with a standard reference frame) (Heywood et al., Reference Heywood, Bannister and Marvil2016). For the FLASH continuum sources of interest for this project, the statistical component is small (
$\lt0.1$
arcsec for a 10 mJy source) and the position uncertainty is dominated by the systematic component. The size of this systematic component is estimated at the data validation stage through a cross-comparison with other large-area source catalogues. For the pilot surveys, these catalogues were NVSS (Condon et al., Reference Condon, Cotton and Greisen1998) and SUMSS (Mauch et al., Reference Mauch, Murphy and Buttery2003). The continuum data validation reports, available via CASDA, present astrometric offsets between the sky positions of FLASH and NVSS/SUMSS for individual SBIDs.
The validation reports show systematic offsets of up to 1 arcsec (and occasionally larger) in both RA and Dec for individual ASKAP fields. In this paper we adopt an indicative astrometric error of 1 arcsec for the FLASH positions, noting that Hale et al. (Reference Hale, McConnell and Thomson2021) found offsets of a similar size for the ASKAP RACS-Low fields. Although we could improve the astrometric accuracy of the pilot survey positions by correcting for the known RA and Dec offsets of each SBID, we chose not to do so in this paper because the pipeline positions are accurate enough for us to be able to match our bright (
$\gt30$
mJy) sources with published radio catalogues. We plan to revisit the ASKAP astrometry in future, in a paper that identifies and discusses the host galaxies of the H i absorption lines found in the FLASH pilot surveys.
6.4 Flux-density scale
To check the ASKAP flux density scale in the FLASH band (712–1000 MHz), we cross-matched sources in our southern (Dec
$\lt-30^\circ$
) pilot fields with the SUMSS catalogue (Mauch et al., Reference Mauch, Murphy and Buttery2003). SUMSS was chosen because its 843 MHz survey frequency is close enough to the FLASH band centre at 856 MHz that we expect the flux densities to be directly comparable. If the field is outside the SUMSS coverage, NVSS is used instead. The flux-density ratio as function of S/N and its variation by sky position for individual SBIDs can be found in the continuum data validation reports on CASDA.
From a cross-matched sample of over 5000 bright (
$\gt20$
mJy) and spatially unresolved FLASH continuum components, we find that the FLASH and SUMSS sources are on the same flux density scale to within 2–3%. Since SUMSS itself is consistent with the 1.4 GHz NVSS flux density scale (Mauch et al., Reference Mauch, Murphy and Buttery2003), this gives us confidence in the reliability of the ASKAP flux density measurements.
RMS spectral-line noise estimates measured from the noise spectra of all FLASH Pilot fields. The estimates account for all sources located at the edge of the field. Blue and orange symbols represent Pilot 1 and Pilot 2 fields, respectively. Individual dots on the dashed and solid lines indicate the mean and median noise from the mid-5184 channels of all noise spectra in each field. The 6-hr fields with lower rms levels are marked as stars. The x-axis is ordered by SBID, following the observation sequence except for the 6-hr fields.
RMS spectral noise comparison across different right ascension (left), declination (middle), and day/night observation (right) demonstrating uniform noise. Blue and orange crosses are from Pilot 1 and Pilot 2, respectively. 6-hr fields with lower rms levels are marked as stars. The median rms of each field is derived from the middle 5184 channels of the individual noise spectra for all sources in that field. Each mean noise level spans 3.16–3.39 mJy beam
$^{-1}$
ch
$^{-1}$
(Pilot 1, 6-hr fields), 5.10–6.13 mJy beam
$^{-1}$
ch
$^{-1}$
(Pilot 1), and 4.78 to 6.80 (Pilot 2) mJy beam
$^{-1}$
ch
$^{-1}$
.
7. Spectral-line data
7.1 Uniform sensitivity of the data
One aim of the Pilot Surveys was to check the uniformity and reproducibility of the ASKAP data. To do this, we tested for variations in sensitivity either with declination or between day-time and night-time observations.
7.1.1 Spectral-line sensitivity
Figure 7 shows the overall rms noise in the spectral-line, measured across all FLASH fields in date order. The rms noise in both continuum and spectral-line data remained roughly uniform over the period of almost three years during which the observations were carried out.
The rms noise in the extracted spectra from the 2-hour fields has a median value of 4.6 mJy per beam per channel in the central third of the frequency band (808–904 MHz) and ranges from 4.2 to 5.3 mJy beam
$^{-1}$
ch
$^{-1}$
in this sub-band (Figure 5). This is consistent with the range of 3.2–5.1 mJy beam
$^{-1}$
ch
$^{-1}$
quoted in Table 1 of Allison et al. (Reference Allison, Sadler and Amaral2022). The rms noise for the 6-hour fields was around 3.5 mJy per beam per channel, which is slightly higher but still close to the value expected for the longer observing time. The expected noise level for a 6-hour observation, reduced by a factor of
$\sqrt{3}$
compared to the 2-hour estimate from Allison et al. (Reference Allison, Sadler and Amaral2022), ranges from 1.8 to 2.9 mJy beam
$^{-1}$
ch
$^{-1}$
.
The spectral-line sensitivity varies by 25–30% across the observing band because of changes in the ASKAP effective system temperature T
$_\textrm{sys}$
with frequency (see Figure 22 of Hotan et al., Reference Hotan, Bunton and Chippendale2021, for details). Figure 5 shows this variation for a typical SBID.
7.1.2 Sensitivity versus sky position
FLASH fields are widely spread over the sky, so we were able to compare the noise across a wide range in R.A. and declination, and between daytime and nightime. Figure 8 shows that the noise properties are roughly uniform with respect to both the right ascension and declination.
7.1.3 Daytime versus night-time observing
The right-hand plot in Figure 8 shows the rms noise as a function of local observing time (AWST). We see no significant difference in sensitivity between daytime observations and observations made at night.
7.2 Spectral-line artefacts
We identified several artefacts affecting the data quality of the spectra as shown in Figure 9. The underlying cause of three of the artefacts seen in the pilot survey data has now been addressed, and these artefacts are not expected to occur in future FLASH observations. These artefacts were:
-
1. Glitches at the edges of beam-forming intervals. Data from the first 18 Pilot 1 fields randomly showed glitches on 1 MHz interval due to the issues with bandpass smoothing parameterisation. This problem has been fixed, and is not seen in data taken after April 2020.
-
2. Correlator dropouts. These are artefacts in specific frequency ranges, caused by data dropouts in an ASKAP correlator block that were not properly accounted for in processing. The main frequency ranges affected are 915–920 MHz (corresponding to H i redshifts in the range
$0.543\lt z\lt0.552$
) and 987–992 MHz (H i at
$0.431\lt z\lt0.439$
. This problem has largely been fixed, and mainly affected data taken before 2021. -
3. Ripples. Wobble-like features are seen in the processed spectra of bright sources in almost all Pilot 2 fields observed between 17 December 2021 and 15 January 2022. These ripples were produced during processing by the use of incorrect parameters in fitting the spectral bandpass, and affect data across the whole spectral band. Delays in uploading the processed data to the CASDA archive for data validation meant that the problem was not identified immediately, and because the ASKAP spectral-line visibilities are deleted immediately after processing it was not possible to reprocess the data with the correct parameters. The problem is not expected to recur.
There is also one ongoing effect that can produce significant spectral-line artefacts:
-
1. Tropospheric ducting of RFI. This occurs under particular atmospheric conditions, when radio signals from distant transmitters are refracted in the troposphere and can propagate over large distances. Ducting can occur in the ASKAP frequency range as well as at lower frequencies (Sokolowski et al., Reference Sokolowski, Wayth and Ellement2017). For FLASH observations, tropospheric ducting can result in the telescope seeing RFI from mobile phone bands used in Perth and other communities well beyond the radio-quiet zone in which ASKAP is located. When ducting is present, ducted RFI signals can occupy up to 20% of the FLASH frequency band. The frequencies most commonly affected are
$758-788$
MHz,
$870-890$
MHz, and
$943-960$
MHz.
Ducted RFI was rarely seen in FLASH pilot survey data taken before January 2022, but became much more common, after that time. The reason for this is not yet understood, but well over half the FLASH pilot survey observations between May and August 2022 (17/24 fields) showed ducted RFI in the spectra of bright sources (see Figure 9 for an example).
Examples of spectral-line artefacts seen from the Pilot data. From top to bottom, each panels presents glitches, correlator dropouts, ripples, and tropospheric ducting of RFI in spectra. The grey background shade represents five times the rms spectral-line noise.
If this high rate of ducting occurs again in future, we can minimise its effects in two ways. The weather conditions that give rise to tropospheric ducting can often be predicted, and we can avoid scheduling FLASH observations when these conditions are present. Alternatively, if ducting conditions are detected at the telescope while a FLASH observation is underway, the observation will be stopped, the data discarded, and the observation rescheduled for a later time. A more detailed discussion of ducting effects at the ASKAP site is given by Indermuehle et al. (Reference Indermuehle, Harvey-Smith, Marquarding and Reynolds2018) and Lourenço et al. (Reference Lourenço, Chippendale and Indermuehle2024).
8. FLASHfinder – An automated search for H i absorption lines
8.1 An automated absorption line-finder for ASKAP
The FLASHfinderFootnote d Bayesian line-finding tool (Allison et al., Reference Allison, Sadler and Whiting2012a) was designed to identify and parameterise H i absorption lines in large ASKAP spectral-line datasets in an automated and efficient way, and to assign a statistical significance to individual line detections. The line finder has been tested on simulated data by Allison et al. (Reference Allison, Sadler and Whiting2012a), and its application to real data is discussed in detail by various recent studies (e.g. Allison et al., Reference Allison, Curran and Emonts2012b; Glowacki et al., Reference Glowacki, Allison and Moss2019; Allison et al., Reference Allison, Sadler and Bellstedt2020; Allison, Reference Allison2021; Mahony et al., Reference Mahony, Allison and Sadler2022; Su et al., Reference Su, Sadler and Allison2022; Aditya et al., Reference Aditya, Yoon and Allison2024).
8.2 Line identification in the Pilot Survey spectra
The FLASHfinder was initially run across the full 711.5–999.5 MHz range for each of the (several hundred) extracted spectra from each SBID. We use a single Gaussian model with nlive = 1000, which is the number of live points used for sampling (Allison et al., Reference Allison, Sadler and Whiting2012a). The output data file produced by the line finder includes measurements of the following parameters and their 1
$\sigma$
uncertainties, based on fitting a single Gaussian component to each candidate line:
-
•
$z_\textrm{HI}$
, the redshift at the peak of the candidate line. -
•
$\tau_\textrm{pk}$
, the peak optical depth of the line. -
•
$\tau_\textrm{int}$
, the integrated optical depth of the line, in km s
$^{-1}$
. -
• The FWHM linewidth with a single Gaussian fit of the line, in km s
$^{-1}$
. -
• The Bayesian evidence value, ln (B), which reflects the extent to which the line detection is favoured over the null hypothesis in which no line is present (for a discussion of the relationship between ln (B) and the signal-to-noise ratio, Appendix A).
In this initial run the linefinder was set to search at all possible redshifts in each spectrum, with no prior constraints on where a line might be. The output files for each SBID contain a set of candidate detections that may be genuine astronomical signals, spectral-line artefacts, or noise peaks.
8.3 Identifying genuine absorption lines
To distinguish astronomical signals from noise peaks, we took a conservative approach by considering only lines for which ln (B)
$\geq30$
. We chose this value by inverting the original spectra and re-running the line finder on the inverted spectra (which we expect to contain only spectral artefacts and noise, since the H i emission line is too weak to be seen in the FLASH redshift range). As can be seen from Figure 10, a value of ln (B)
$\sim30$
represents an upper envelope to most of the linefinder points from the inverted spectrum, with the exception of several strong spectral artefacts seen as vertical lines in both plots.
Plot of Bayesian evidence with ln (B) versus redshift (left) and linewidth (right) for extracted spectra from FLASH field 719, with the Pilot 1 linefinder results (top) and the Pilot 2 results (bottom). In both plots the lines found from the original absorption spectra are plotted in blue and those from the inverted spectra in orange.
The false detection (artefact) count normalised by the number of fields in the absorption and inverted spectra as a function of redshift across the FLASH pilot surveys. The bin sizes correspond to the redshift range covered by 1 MHz interval. The highest peaks for the first pilot survey are mainly caused by correlator dropouts, while those from the second pilot survey are mainly due to ducted RFI.
We next aimed to distinguish astronomical signals from spectral artefacts like those shown in Figure 9. In general, we expect that an H i absorption line at any given redshift should occur in only one or two spectra in an SBID, while the spectral artefacts occur in the same position in many different spectra.
To identify repeated artefacts and spectral-line glitches at specific frequencies across many SBIDs, we aggregated the total number of detections returned by the FLASHfinder algorithm using a log-likelihood cut of ln (B)
$\gt0$
. From the histograms in Figure 11, the sharp spikes with counts above
$\sim$
40 correspond to regions with correlator dropouts or ducting. From these plots, we compiled a list of frequencies at which the most common spectral artefacts occurred. In general, the Pilot 2 data more often contains artefacts (see Section 7.2) than the Pilot 1 data after normalising by the number of fields observed.
8.4 Properties of the detected lines
In all, we detected 30 lines with ln (B)
$\gt 30$
that we are confident are robust detections of genuine H i absorption lines. Additionally, three lines from well-known sources have been identified but are not included in this paper as they have been previously studied (see Appendix B). As we will show in section 8.6, it is likely that other H i lines with smaller ln (B) values are present in the Pilot survey data, but we have not attempted to identify them at this stage and this will be the subject of a future paper.
Table A1 in Appendix E lists these lines, along with details of the continuum sources against which they were detected. We next extracted spectra in a smaller region around each line, and plotted the optical depth against velocity in the region around the line peak, as shown in Figure 12. We then re-ran the line finder on these extracted spectra, after first co-adding the spectra of sources with two or more observations in Table A1. The final results for these co-added spectra are listed in Table 3 and plots of the individual spectra are shown in Figure A4. Figure 13 shows the optical depth, linewidth, and logarithmic Bayes number as a function of redshift. The 33 detections, including the three known detections, were found across the entire redshift range covered by the survey with linewidths ranging from
$\sim$
20 to 218 km s
$^{-1}$
.
Two strong H i absorption lines identified in Pilot Survey Phase 1 and 2. The line profiles of all the detections are added in Appendix D. The grey shaded region in this figure and the figures in Appendix D represents 1
$\sigma$
level of the rms spectral-line noise.
FLASHfinder results for co-added spectra; (1) ID; (2) Source name; (3) Number of spectra combined; (4) Field name; (5) Continuum flux density; (6) H i redshift; (7) Peak optical depth and uncertainty; (8) Integrated optical depth and uncertainty; (9) Linewidth and uncertainty; (10) ln (B); (11) Machine Learning (ML) classification, using the methodology of Curran (Reference Curran2021), where ‘As’ indicates a probable associated line and ‘In’ a probable intervening line, see Section 8.5; (12) Notes (N = Individual Note in Appendix B). The uncertainties listed in columns (7) to (9) represent the 68% credible interval about the median. The individual spectra before combining are listed in Table A1 in Appendix E.
Currently, genuine absorption lines are differentiated from spectral artefacts through visual inspection and investigation by the team. This is highly inefficient and not sustainable for the entire FLASH survey. As such, we are undertaking a separate project to investigate the use of tree-based machine learning methods to improve source-identification efficiency (Liu et al. in prep.) for future FLASH observations.
8.5 Machine Learning classification of detected absorption lines
We expect the lines identified from the observations (listed in Table A1 in Appendix E) to be a mixture of intervening and associated H i absorption systems. As a preliminary way of distinguishing these in the absence of optical spectroscopy for many of our objects, we used an automated machine learning (ML) methodology (Curran et al., Reference Curran, Duchesne, Divoli and Allison2016a; Curran, Reference Curran2021) to classify each of the absorbers detected in this survey.
We use the logistic regression (ML) classifier of Curran (Reference Curran2021), which was the best performing of the other common classifiers and compile the results in the final column of Table 3. To classify the lines, we used a training set of 116 out of a sample of 138 known absorbers from the literature, selected to have equal numbers of associated and intervening systems. 16 of the lines in Table 3 were classified as intervening and 14 as associated, with the line width (FWHM) being the main driver of the classification (Figure 14).
The broader associated profiles could be due to the additional nuclear component, hypothesised by unified schemes of active galactic nuclei (AGN, e.g. Antonucci, Reference Antonucci1993; Urry & Padovani, Reference Urry and Padovani1995; Curran et al., Reference Curran, Reeves, Allison and Sadler2016b) being preferably detected through the disk of the host (cf. Curran & Whiting, Reference Curran and Whiting2010), whereas intervening absorption favours face-on systems, where the coverage of the background flux is maximised (Curran et al., Reference Curran, Duchesne, Divoli and Allison2016a).
In the Pilot Survey sample, we find an equal mix of associated and intervening systems. A potential limitation that we note here is that the training set used consists mainly of low-z systems and contain an optical pre-selection, which may not be fully representative of the sources that could be found from an untargeted survey such as FLASH. We will return to this question in a subsequent paper discussing the host galaxies of the Pilot Survey absorption systems.
We also caution that a match between the redshift of a detected H i line and the optical redshift of the galaxy in which the H i absorption occurs does not necessarily imply that the line can be classified as associated. Figure 9 of Allison et al. (Reference Allison, Sadler and Bellstedt2020) shows one example of an intervening H i detection in a galaxy (GAMA J224500.05–343031.7) where the radio and optical redshifts match. Here, the background radio source against which the line is detected is offset by 2.6 arcsec from the centre of the galaxy in which the detected H i is located. Although the background radio-source host is obscured by the foreground galaxy in this case, the astrometric offset shows that the line is intervening rather than associated. Section 4.1 of Allison et al. (Reference Allison, Sadler and Amaral2022) includes a more detailed discussion of the problem of distinguishing intervening and associated lines in an untargeted survey like FLASH.
8.6 Completeness tests
Next, we investigated the completeness of the FLASHfinder H i 21-cm detections using the techniques described by Allison et al. (Reference Allison, Sadler and Bellstedt2020) and Allison et al. (Reference Allison, Sadler and Amaral2022). The completeness is defined as the probability of recovering an absorption feature with a given peak signal-to-noise ratio (S/N; in an 18.5 kHz channel) and line width (in
$\textrm{km~s}^{-1}$
) from the spectrum, using the linefinder (Allison et al., Reference Allison, Sadler and Whiting2012a).
This is estimated by selecting 100 spectra randomly from a sample of ‘test’ spectra, and inducing a Gaussian absorption line with a specific peak signal-to-noise ratio (S/N) and line width in each spectrum at random locations. The linefinder is then run on these 100 spectra, after which we estimate the fraction of the recovered absorption lines. We made two separate runs. The first run used a detection limit of ln (B)
$\gt 20$
, which is slightly deeper than the threshold described in the earlier sections, to test the depth of our threshold. The second run used a detection limit of ln (B)
$\gt 30$
, consistent with our usual threshold.
We repeated this process over a wide range of combinations of S/N and line width. The completeness, i.e. the fraction of recovered lines, is plotted as a bivariate function for each peak S/N - line width combination (see e.g. Figure 15). The function is finally smoothed using a Savitzky-Golay filter (Savitzky & Golay, Reference Savitzky and Golay1964).
For these tests, we used spectra from two SBIDs, SB 34917 and SB 34581. The spectra of SB 34917 are known to be clean, without any significant RFI ducting features or ripples in the spectra. The completeness fractions with a detection limit of ln (B)
$ = 20$
show a relatively smaller scatter compared to those with ln (B)
$=30$
. Also, the curves are slightly shifted towards lower peak S/N values in the plot with ln (B)
$= 20$
, implying that a larger fraction of weaker lines are detected.
In Figure 16 we plot the completeness fractions for SB 34581, using a detection limit of ln (B)
$= 20$
. The spectra in these observations are corrupted by ripples and ducting. The effects of these can be seen in the completeness plots; the detection fractions are not optimum for various FWHM and peak S/N combinations.
We see from Figure 15 that the completeness functions of SB 34917 are smooth, and reach 100% detection fractions quickly for peak S/N
$\gt 10$
and line widths
$ \gt 10 \textrm{km s}^{-1}$
. However, for SB 34581 in Figure 16, the functions are not smooth and the fractions do not reach 100% for most combinations of the S/N peak and line width. This is because of the reduced efficacy of the automated line finder in identifying the induced Gaussian absorption feature among spectral ripples and ducting features.
In our tests using the clean spectra (Figure 15, the right panel), we achieve 80 to 100% completeness in recovering broader lines with widths above 80 km s
$^{-1}$
(red and orange lines), for a given peak S/N of 3. This corresponds to the significance of ln(B)
$= 30$
as detailed in 11. Meanwhile, in order to reach a certain completeness level, narrower lines require higher peak S/N values. For instance, to achieve 50% completeness overall, peak S/N values of approximately 4, 6, 8, and 12 are needed for line widths of 40, 20, 10, and 5 km s
$^{-1}$
, respectively (light green to purple lines). It is important to note that the channel resolution for each observation ranges from 8 to 19 km s
$^{-1}$
.
8.7 Comparison of repeat observations
To test the robustness of the linefinder results, we compared data from FLASH Pilot Survey fields that were observed more than once. From the data in Table A1 in Appendix E, we identified 14 pairs of detected lines where the linefinder measurements could be compared. Ten of these were Pilot 2 re-observations of lines detected in Pilot 1, and four were pairs of Pilot 2 observations.
Figure 17 shows the values of redshift z, peak optical depth
$\tau_\textrm{pk}$
, integrated optical depth
$\tau_\textrm{int}$
, and velocity width
$\Delta\textrm{V}$
measured for these pairs of lines. From this comparison, we find that redshift measurements are highly reproducible (differing by
$\lt\lt0.1$
%). The peak and integrated optical depth measurements have typical uncertainties of around 25% and the line width measurements (fitting a single component) have a typical uncertainty of around 20%, though with a small number of outliers in each case.
Peak optical depth (red) and linewidth (yellow) with 1
$\sigma$
error as a function of redshift. ln (B) (green) is presented in the bottom panel with the dotted line of ln (B) = 30, the cutoff used in the linefinder search.
8.8 Verification of FLASH data using MeerKAT
A subset of the H i absorption lines detected in the FLASH pilot surveys were observed with MeerKAT to better characterise the absorption lines with higher S/N. These observations were carried out under the program ID SCI-20210212-EM-01 and used the UHF band covering the frequency range of
$580 - 1015$
MHz with the 32K mode correlator giving a similar spectral resolution of 16.6 kHz compared to ASKAP’s 18.5 kHz. Each target was observed for a total integration time of 1 hr. A complete analysis of the MeerKAT observations will be the subject of future papers (Mahony et al. in prep., and Glowacki et al. in prep.), but we present initial results here for data verification purposes.
Figure 18 shows the ASKAP-FLASH and MeerKAT spectra of two Pilot Survey detections (J090425+012015 at
$z_\textrm{HI}=0.80$
and J092012+161238 at
$z_\textrm{HI}=0.44$
). With the exception of the higher S/N provided by the MeerKAT observations, the ASKAP and MeerKAT spectra match remarkably well and nicely demonstrate the spectral line capabilities and fidelity afforded by these two SKA precursor telescopes on radio-quiet sites.
8.9 General remarks on spectral-line data quality
The quality of the spectral-line data taken at the radio-quiet ASKAP site can be extremely good (see Figure 6), with the FLASH band from 712–1000 MHz generally free from terrestrial RFI.
Overall though, the quality of the pilot survey data was variable and most of the processed data released in CASDA are classified as being of ‘uncertain’ quality. This is perhaps to be expected from early observations with a new telescope coupled with a sophisticated data processing pipeline. As discussed earlier, data from the first pilot survey suffered from correlator dropouts and needed post-processing because of an incorrect implementation of the beam-forming intervals, while the data from the second pilot survey was affected by several spectral-line artefacts. These problems have now been overcome, and should not affect future FLASH observations.
The distributions of the dominant model features for the putative intervening (In) and associated (As) absorbers. For the redshift and optical depth, t-tests give probabilities of p(t) = 0.64 and 0.70 of the means being the same, respectively. For the FWHM this is
$p(t) = 1.68 \times 10^{-6}$
.
The completeness fractions plotted as a function of the peak signal-to-noise ratio for data from a good-quality observation (SB 34917). The colours represent a set of lines of different FWHMs, as indicated in the legend. The linefinder detection limit is set at
$\textrm{ln (B)}~\gt 20$
for the left-hand plot, and at
$\textrm{ ln (B)}~\gt 30$
for the right-hand plot. Further details are provided in Section 8.6.
9. Discussion
9.1 H i absorption-line systems at intermediate redshift
The 30 new H i lines listed in Table 3 have almost doubled the number of known H i absorption-line systems in the FLASH redshift range. Previously, Curran (Reference Curran2021) listed 44 known H i absorbers at redshift
$0.42\lt z\lt1$
across the whole sky, almost all detected in targeted observations of optically-selected quasars. Our untargeted absorption-line survey allows us to detect H i lines against the full range of radio-source populations, and in the next section we discuss the radio continuum properties of the sources listed in Table 3.
9.2 Radio continuum properties of H i detected sources
Spectral energy distribution (SED) modelling in the radio regime can provide insights into the nature of the AGN in a regime where its synchrotron emission typically dominates. All the continuum sources against which we detected H i absorption were unresolved in the highest-resolution (12-15 arcsec beam) FLASH continuum images, and we can learn more about these compact sources by fitting the broadband radio continuum using flux density measurements from the literature. The fitting process assumes minimal broadband variability between epochs, but this appears reasonable given the low variability of most radio AGN at frequencies below a few GHz (e.g. Bannister et al., Reference Bannister, Murphy, Gaensler, Hunstead and Chatterjee2011; Mooley et al., Reference Mooley, Hallinan and Bourke2016)
Our SED construction and modelling was performed using RadioSED, a Bayesian inference package designed for use with broadband radio continuum data, and which also automatically compiles published flux densities from large area surveys (Kerrison et al., Reference Kerrison, Sadler, Moss and Mahony2024)Footnote e . This inference procedure can capture various degrees of spectral curvature allowing for the identification of peaked spectrum sources, thought to be radio AGN within a few thousand years of triggering (O’Dea & Saikia, Reference O’Dea and Saikia2021).
Figure A4 shows the radio SEDs of the sources with H i detections plotted alongside the H i spectra. As discussed by Kerrison et al. (Reference Kerrison, Sadler, Moss and Mahony2024), there is a clear excess of peaked-spectrum (PS) SEDs among the Pilot Survey detections, with 20/30 (67%) showing a broadband peak. Previous studies of the radio population as a whole found a peaked fraction of 5-20% (O’Dea, Reference O’Dea1998; Callingham et al., Reference Callingham, Ekers and Gaensler2017), so it is remarkable to see so many PS sources in our untargeted search for H i absorption. To quantify this further, if we assume that 20% of all radio galaxies are PS sources (O’Dea, Reference O’Dea1998), then the binomial probability of finding 21 or more PS sources in our H i -absorbed sample is only
$\text{Pr}(\textit{N}_{\text{PS}} \ge 21) = 3 \times 10^{-8}$
. This number is even lower if we assume the 5% population fraction from Callingham et al. (Reference Callingham, Ekers and Gaensler2017). More recent targeted searches have reported numerous H i detections in PS sources at high redshift (e.g. Aditya & Kanekar, Reference Aditya and Kanekar2018a; Glowacki et al., Reference Glowacki, Allison and Moss2019; Aditya et al., Reference Aditya, Yoon and Allison2024, and references therein).
Similar to the left-hand plot of Figure 15, but for an observation (SB 34681) where the data suffered from the spectral ‘ripples’ discussed in Section 7.2.
In this untargeted survey, we find that the SED radio properties of the continuum sources against which we detect H i absorption are not typical of the overall radio AGN population. To say more about this emerging trend requires larger statistical samples of absorbers, which the full FLASH survey will provide.
9.3 Properties of the detected H i lines
The H i lines detected in the pilot survey span a wide redshift range from
$z=0.43$
to
$z=0.91$
, and have velocity widths between 19 and 199 km s
$^{-1}$
. Figure 19 shows the distribution of optical depth and line width for the detected lines.
Comparison of linefinder results for repeat observations. All values are for a single gaussian fit to the line profile, and the error bars show the uncertainties listed in the linefinder output. Blue points show a comparison between Pilot 1 and Pilot 2 data, and red points a comparison between two different Pilot 2 observations.
Spectra of FLASH detections observed with MeerKAT (black line) overlaid with the ASKAP spectrum (blue). The grey band denotes the 5-
$\sigma$
noise level calculated from the MeerKAT spectra.
Properties of the H i lines detected against continuum sources of different flux densities. (left) The peak optical depth
$\tau_\textrm{pk}$
, with a dashed line showing the expected sensitivity of the FLASH survey as calculated by Allison et al. (Reference Allison, Sadler and Whiting2012a) and a vertical dotted line showing the 45 mJy limit adopted for the H i line search. (right) The line width in km s
$^{-1}$
, as listed in Table 3. In both plots, horizontal dashed lines show the median values for the associated and intervening absorption lines in the literature compilation of Curran (Reference Curran2021).
The left-hand plot in Figure 19 shows that the limiting peak optical depth that can be detected increases at lower flux densities, as expected for an absorption-line search. The lower bound for the detected lines lies close to the diagonal line in this plot, which shows the expected FLASH sensitivity limit calculated by Allison et al. (Reference Allison, Sadler and Whiting2012a). In other words, we are reaching the optical-depth sensitivity expected when the FLASH survey was first proposed more than a decade ago.
A second point to note from this plot is the high peak optical depth of most of our detected lines. Our sample contains three lines with a peak optical depth
$\tau_\textrm{pk}\gt1.0$
, larger than any of the 136 lines in the Curran (Reference Curran2021) compilation of literature data. The median value of
$\tau_\textrm{pk}=0.148$
for our detected lines is significantly higher than the median
$\tau_\textrm{pk}=0.030$
for the literature sample compiled by Curran (Reference Curran2021). This is consistent with the prediction by Allison et al. (Reference Allison, Sadler and Amaral2022) that FLASH would detect a population of lines with very high H i optical depth.
From the right-hand plot, we can see that the detections span a wide range in line width but there are very few lines narrower than about 25 km s
$^{-1}$
. There is no significant difference in the median line width for sources across a wide range in continuum flux density.
9.4 The H i detection rate in the pilot surveys
In the first pilot survey we detected 17 new linesFootnote
f
from 37 fields, a rate of 0.46
$\pm0.11$
lines per field. In the second pilot survey the rate was lower, with 12 detections from the 38 fields not previously observed - a rate of 0.32
$\pm0.09$
lines per field. The lower detection rate for the second pilot survey may be due to the large number of spectra with spectral ripples that reduce the completeness of the linefinder search (see Figure 16). We therefore adopt the Pilot 1 value of 0.5 lines per field as an indicative H i detection rate for the pilot surveys.
In Table 3 of their paper, Allison et al. (Reference Allison, Sadler and Amaral2022) estimate the total number of H i detections expected from a FLASH survey covering 903 ASKAP fields. They also caution that their estimates are highly uncertain - particularly for the population of associated H i lines. For an assumed H i spin temperature T
$_\textrm{S}=300$
K, Allison (Reference Allison2021) estimated a detection rate of 3.1 lines per ASKAP field, with 70% of these being associated lines and 30% intervening lines.
Our observed detection rate of
$\sim0.5$
lines per ASKAP field is significantly lower than the Allison et al. (Reference Allison, Sadler and Amaral2022) estimate. There are at least two plausible reasons for this:
-
1. Associated H i absorption lines at redshift
$0.4\lt z\lt1$
are less common than expected. Recent work by Su et al. (Reference Su, Sadler and Allison2022) and Aditya et al. (Reference Aditya, Yoon and Allison2024) has shown that the detection rate of associated H i absorption lines in radio AGN within the FLASH redshift range is only 2-3%, significantly lower than the value of 10% assumed by Allison et al. (Reference Allison, Sadler and Amaral2022) but consistent with the low detection rate (0 lines from 29 sources) found by Murthy et al. (Reference Murthy, Morganti, Kanekar and Oosterloo2022) for radio galaxies at
$0.7\lt z\lt1$
. If we adjust the Allison et al. (Reference Allison, Sadler and Amaral2022) values to take into account the lower expected rate of associated lines, then the predicted H i detection rate drops to
$\sim1.5$
lines per FLASH field. -
2. Some narrow absorption lines may be missed in our survey. The results shown in section 8.6 imply that our sample will be less complete for narrow lines (FWHM
$\lt$
20 km s
$^{-1}$
) than for broad lines across a wide range in peak S/N. How this affects the data shown in the right-hand plot of Figure 15 is harder to assess, since there is no simple relationship between peak S/N and continuum flux density (a weak line seen against a bright source and a stronger line seen against a fainter source may have similar peak S/N values), but it is notable that we have so far detected vary few lines with velocity widths below 20-30 km s
$^{-1}$
. Such lines also lie close to the spectral resolution limit of the ASKAP data, and may be difficult to distinguish from the single-channel ‘glitches’ seen in some pilot survey spectra.
Figure 20 shows the range of peak SNR and line width observed in the pilot survey detected lines, along with the 80% completeness limit derived from the data shown in Figure 15. While we do detect some lines down to the expected completeness limit, the variable quality of the Pilot Survey observations means that that not all fields will reach this completeness limit. A more detailed analysis of the observed detection rate will be carried out in a future paper using a larger, higher-quality dataset from the full FLASH survey.
Plot of velocity width against peak SNR for the detected lines. The dashed line shows the level at which completeness tests imply that at least 80% of lines should be detected in a good-quality FLASH spectrum.
9.5 Future work
Following the two phases of Pilot observations, we began the full five-year FLASH survey in late 2022. The full FLASH survey will cover 600 fields ASKAP fields (a total of 24,000 deg
$^2$
) rather than the 903 fields originally planned, with the allocated 1 200 h of observing time. The coverage will span declinations from
$-90$
to
$+18$
degrees at Galactic latitudes
$|b|\geq8.5$
deg. Our pilot survey results suggest that the full FLASH survey will detect between 300 and 600 new absorption lines, with around 60% of these being associated lines.
The next paper in this series will identify the host galaxies of the H i lines discovered in the pilot surveys, and will also discuss how best to determine whether an individual line is associated or intervening.
10. Summary
The FLASH pilot surveys are pioneering studies that search for H i absorption lines in and around distant galaxies without any optical preselection of targets. We have identified a sample of 30 new robustly-detected absorption lines, several of which have higher optical depths than any previously-known line. This work has led us to make new predictions about the detection rate for the full FLASH survey, highlighting its potential impact on our understanding of H i absorption systems.
We successfully tested and demonstrated the automatic data pipeline for the survey and made public immediately after validation, with no proprietary period. We also used a powerful Bayesian line finder to identify and parameterise absorption lines in a consistent and uniform way. Observations of a subset of these lines with the MeerKAT telescope show a high degree of consistency, implying that both telescopes produce high-quality, reproducible spectral-line data.
The three main aims of the FLASH Pilot Surveys have all been achieved.
Firstly, we were able to test and demonstrate an end-to-end data pipeline for the FLASH survey and make line detections that are robust, reproducible, and have a statistical significance attached. We also identified several improvements that could be made in future, including masking out spectral-line artefacts before running the line finder, fitting multiple components to the absorption lines, and further investigating the completeness and detectability of faint and narrow spectral lines.
Our second goal was to determine the H i detection rate for associated absorption lines at
$0.4\lt z\lt1$
. This was done in two companion papers that were able to use optical redshifts as a prior for the line search (Su et al., Reference Su, Sadler and Allison2022; Aditya et al., Reference Aditya, Yoon and Allison2024). In the redshift range covered by FLASH, the detection rate for associated H i lines is 2–3% for the source population as a whole, rising to around 23% for compact radio sources less than about 1 arcsec in size.
Finally, we have made new predictions of the detection rate for the full FLASH survey, showing that FLASH is likely to detect 300–600 new H i absorption systems at intermediate redshifts (
$0.4 \lt z \lt 1.0$
). This represents an order of magnitude more detections than currently available in this redshift range, and should enable us to constrain the average spin temperature and/or H i mass density at intermediate redshifts.
Acknowledgement
This scientific work uses data obtained from Inyarrimanha Ilgari Bundara/the Murchison Radio-astronomy Observatory. We acknowledge the Wajarri Yamaji People as the Traditional Owners and native title holders of the Observatory site. CSIRO’s ASKAP radio telescope is part of the Australia Telescope National Facility (https://ror.org/05qajvd42). Operation of ASKAP is funded by the Australian Government with support from the National Collaborative Research Infrastructure Strategy. ASKAP uses the resources of the Pawsey Supercomputing Research Centre. Establishment of ASKAP, Inyarrimanha Ilgari Bundara, the CSIRO Murchison Radio-astronomy Observatory and the Pawsey Supercomputing Research Centre are initiatives of the Australian Government, with support from the Government of Western Australia and the Science and Industry Endowment Fund.
This research was supported by the Australian Research Council Centre of Excellence for All Sky Astrophysics in 3 Dimensions (ASTRO 3D) through project no. CE170100013.
This work was supported by the Global-LAMP Program of the National Research Foundation of Korea (NRF) grant funded by the Ministry of Education (No. RS-2023-00301976). This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (RS-2025-00516062).
RZ acknowledges support from the China Postdoctoral Science Foundation (Grant No. 2024M752979).
ACE acknowledges support from STFC grant ST/P00541/1. KG is supported by the Australian Research Council through the Discovery Early Career Researcher Award (DECRA) Fellowship (project number DE220100766) funded by the Australian Government. MG is supported by the Australian Government through the Australian Research Council’s Discovery Projects funding scheme (DP210102103), and through UK STFC Grant ST/Y001117/1. MG acknowledges support from the Inter-University Institute for Data Intensive Astronomy (IDIA). IDIA is a partnership of the University of Cape Town, the University of Pretoria and the University of the Western Cape. For the purpose of open access, the author has applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising from this submission.
The MeerKAT telescope is operated by the South African Radio Astronomy Observatory, which is a facility of the National Research Foundation, an agency of the Department of Science and Innovation.
We thank the anonymous referee for the valuable comments and suggestions that allowed us to improve our manuscript. We acknowledge technical support from the Australian SKA Regional Centre (AusSRC). We express our gratitude to Dr. Jonghwan Rhee and Prof. Nissim Kanekar for their insightful suggestions.
Data availability statement
The FLASH Pilot Survey data products have been released through the CSIRO ASKAP Science Data Archive (CASDA; https://research.csiro.au/casda) under project code AS109. The post-processed data are available upon request.
Appendix A. A note on Bayesian ln (B) values and statistical significance
In this paper we have used a Bayesian evidence parameter ln (B) to identify absorption lines in ASKAP spectra. Here, we briefly discuss the relationship between ln (B) and the statistical n
$\sigma$
uncertainty often used by the wider astronomy community to assess the significance of a result.
The calibration from frequentist p-values, or ‘sigma significance levels’ to the Bayes factor has been discussed in detail by Sellke et al. (Reference Sellke, Bayarri and Berger2001) and Trotta (Reference Trotta2008). Trotta (Reference Trotta2008) also provides an empirical scale for evaluating the strength of evidence when comparing two models, with ln (B) = 5 corresponding to ‘strong evidence’ (a 3.6
$\sigma$
result, with p
$=0.0003$
) and ln (B) = 11 to a 5
$\sigma$
value. Benneke & Seager (Reference Benneke and Seager2013) apply Bayesian methods to a specific problem in exoplanet atmospheres and provide an expanded version of the Trotta (Reference Trotta2008) Bayes to frequentist ‘translation table’.
In the context of our FLASH absorption-line finding problem, the Bayes value is the odds ratio of obtaining the observed data (assuming the absorption-line model is correct) versus just gaussian noise. The p-value is the probability of obtaining data features that are at least as extreme as those observed assuming just Gaussian noise. In both cases no consideration is made for non-Gaussian noise or artefacts - which we know are present in the ASKAP Pilot Survey data (see Figure 9).
We therefore stress that all statistics quoted in this paper (whether B, p-values and/or sigmas) are calculated with respect to the assumed Gaussian noise and not any other non-Gaussian noise features.
Relationship between the linefinder ln (B) value and estimated peak SNR for the H i absorption lines detected in the Pilot Survey. Blue points represent lines with velocity width
$\lt80$
km s
$^{-1}$
, and red points show broader lines with velocity width
$\geq80$
km s
$^{-1}$
. The horizontal line at ln (B) = 30 shows the cutoff value used in this paper to select line detections and the vertical dotted line corresponds to a 3
$\sigma$
detection of the line peak.
As in Figure A1, except that the SNR values plotted are now integrated over the whole line, assuming a gaussian profile. In this plot, black open circles at lower ln (B) show values tabulated by Benneke & Seager (Reference Benneke and Seager2013) and the vertical dotted line corresponds to the 3.6
$\sigma$
value considered by Trotta (Reference Trotta2008) to be ‘strong’ evidence.
To examine the empirical relationship between ln (B) and the signal-to-noise ratio (SNR) in our current set of line detections, we calculated an approximate peak SNR for each FLASH detection using the single spectrum in which the line was first detected. We then calculated the size of the ‘dip’ in the continuum at the line peak from the continuum flux density and the observed peak optical depth
$\tau_\textrm{pk}$
and divided this by an assumed noise of 5.5 mJy per beam in a single channel.
The results are shown in Figure A1, where dotted lines show a peak SNR of 3
$\sigma$
and our adopted cutoff for detections of ln (B) = 30. We need to keep in mind that the SNR in this plot is a peak value, while ln (B) is calculated for the whole line rather than a single channel. Nevertheless, there is a reasonable correlation that suggests we are detecting lines with peak SNR greater than about 2.5 to 3
$\sigma$
.
Figure A2 shows SNR values integrated over the whole line, which takes into account the range of line wdiths present in our data. Values from Table 2 of Benneke & Seager (Reference Benneke and Seager2013) are plotted as open circles, and our data extend the Benneke & Seager (Reference Benneke and Seager2013) values out to much higher values of ln (B). This plot implies that an absorption line with ln (B)
$\geq30$
has a line SNR
$\geq7-8$
and is therefore a highly significant detection. It also implies that (in the absence of artefacts) we should be able to detect lines down to values of ln (B)
$\sim10$
. Such detections may become possible in future if we can apply machine learning algorithms to distinguish artefacts from astronomical signals in a reliable way.
Appendix B. Notes on individual objects
PKS 0011-023 (J001425-020556): Dunlop et al. (Reference Dunlop, Peacock and Savage1989) list an optical redshift of
$z=2.08$
for the host galaxy, and give the reference for this as ‘Spinrad (private communication)’. Although no further redshift information is available in the literature, PKS 0011-023 is generally assumed to be a high-redshift radio galaxy (e.g. Pentericci et al. Reference Pentericci, Van Reeven, Carilli, Röttgering and Miley2000). If the redshift of 2.08 is correct, then our FLASH 21 cm H i detection at
$z=0.678$
arises in an intervening system.
NVSS J002331+010114 (J002331+010114): The unpublished LOFAR data point at 144 MHz in the SED plot in Figure A4 was supplied by J. Callingham (private communication) and implies that this is a peaked-spectrum source.
MRC 0023-482 (J002604-475618): This object is classified as a peaked-spectrum radio source by Callingham et al. (Reference Callingham, Ekers and Gaensler2017). The H i absorption line at
$z=0.6746$
shown in Figure A4 is clearly visible in both Pilot Survey spectra of this source, but was only identified by the linefinder in the second (SBID 37448) spectrum.
NVSS J015516-251423 (J015516-251423): The peak H i optical depth in this source (
$\tau=1.83$
) is the highest seen in our Pilot Survey sample.
PKS 0253-259 (J025544-254739): An absorption line was independently detected in two adjoining FLASH fields, 308 (SB 15212) and 309 (SB 13269). In both cases the source lies quite close to the edge of the ASKAP field, but the detection can be considered reliable. As can be seen from Figure A4, the H i line appears to have two narrow velocity components separated by about 80 km s
$^{-1}$
.
PKS 0405-280 (J040757-275705): This object was identified as a peaked-spectrum radio source by both Snellen et al. (Reference Snellen, Lehnert, Bremer and Schilizzi2002) and Callingham et al. (Reference Callingham, Ekers and Gaensler2017).
NVSS J051806-245502 (J051805-245502): This compact ICRF source was observed with the VLBA by Petrov et al. (Reference Petrov, Kovalev, Fomalont and Gordon2008) and shows resolved structure on scales of a few milliarcseconds.
MRC 0531-237 (J053354-234430): This source has a deep and broad H i absorption line at
$z=0.851$
, which is discussed in more detail by Aditya et al. (Reference Aditya, Yoon and Allison2024). McCarthy et al. (Reference McCarthy, Kapahi and van Breugel1996) list an optical redshift of
$0.851\pm0.050$
, from which Aditya et al. (Reference Aditya, Yoon and Allison2024) identify this as an associated absorption-line system. It was identified as a peaked-spectrum radio source by Callingham et al. (Reference Callingham, Ekers and Gaensler2017).
NVSS J090331+010846 (J090331+010847): This object lies in the GAMA G09 field (Driver et al. Reference Driver, Hill and Kelvin2011; Liske et al. Reference Liske, Baldry and Driver2015). The line detected in the pilot survey is discussed in detail by Su et al. (Reference Su, Sadler and Allison2022), who identified it as an associated H i absorption system at
$z=0.52$
.
NVSS J090425+012015 (J090425+012015): This absorber lies in the GAMA G09 field, with an H i redshift of
$z_\textrm{ HI}=0.8004$
. No optical redshift is available. The line is detected in two adjoining fields in the second Pilot Survey, 546 (SBID 34559) and 547 (SBID 34539), but the radio source is very close to the edge of field 547, and the line detection has lower significance in this field.
NVSS J091256+030021 (J091256+03002): This 33 mJy source lies below the 45 mJy flux density limit used to extract spectra for our linefinder search in the 2-hour Pilot Survey fields. A lower flux density limit of 30 mJy was used for the linefinder search in the 6-hour GAMA fields, and this line was identified at
$z_\textrm{HI}=0.859$
. The continuum source has been observed with the VLBA by Deller & Middelberg (Reference Deller and Middelberg2014), who measured a source size of
$6.4\times3.8$
mas at 1.4 GHz.
PKS 0917+18 (J092011+175324): This object was identified as a peaked-spectrum source by Callingham et al. (Reference Callingham, Ekers and Gaensler2017). Two photometric redshift estimates have been published,
$z\sim0.68$
(Yan et al. Reference Yan, Stocke, Darling and Hearty2012) and
$z\sim0.42$
(Ma et al. Reference Ma, Xu and Zhu2019). Our detection of 21 cm absorption at
$z_\textrm{HI}=0.904$
is at higher redshift than either of these values, suggesting that the photometric redshifts may be underestimated because of a significant non-thermal contribution in optical and infrared light.
NVSS J094650-202044 (J094650-202045): This source lies within the 6 deg
$^2$
region of sky covered by the CENSORS survey (Best et al. Reference Best, Arts and Röttgering2003), which has deep K-band imaging and optical spectroscopy available for many sources. Brookes et al. (Reference Brookes, Best, Peacock, Röttgering and Dunlop2008) derive a redshift of
$z=0.91$
for the optical counterpart of NVSS J094650-202044, based on the detection of a single emission line of [OII].
NVSS J113622+004850 (J113622+004850): This object lies in the GAMA G12 field (Driver et al. Reference Driver, Hill and Kelvin2011; Liske et al. Reference Liske, Baldry and Driver2015). It is discussed in detail in a recent paper by Su et al. (Reference Su, Sadler and Allison2022), who identified it as an associated H i absorption system at
$z=0.56$
.
NVSS J150506+022927 (J150506+022928): Callingham et al. (Reference Callingham, Ekers and Gaensler2017) identified this object as a peaked-spectrum radio source. Hewett & Wild (Reference Hewett and Wild2010) list an optical redshift of
$z=0.8055\pm0.0004$
based on an SDSS spectrum.
NVSS J170135-294918 (J170135-294917): This source is at low Galactic latitude (
$b=-7.0$
deg).
PKS 2007-245 (J201045-242545): This object was identified as a peaked-spectrum radio source by Callingham et al. (Reference Callingham, Ekers and Gaensler2017).
NVSS J215924-241752 (J215924-241752): This source is in the MRC-1Jy sample (McCarthy et al. Reference McCarthy, Kapahi and van Breugel1996), and was identified by Baker et al. (Reference Baker, Hunstead, Kapahi and Subrahmanya1999) as a quasar at
$z=0.862$
. The object is discussed in more detail by Aditya et al. (Reference Aditya, Yoon and Allison2024), who identified it as an associated H i absorption system and noted that the H i absorption line (at
$z=0.868$
) is redshifted relative to the optical lines.
PKS 2311
$-$
477 (J231351-472911): This object was identified as a peaked-spectrum radio source by Callingham et al. (Reference Callingham, Ekers and Gaensler2017).
SUMSS J233432-585646 (J233432-585646): This object was identified as a peaked-spectrum radio source by Callingham et al. (Reference Callingham, Ekers and Gaensler2017).
Note: The following three objects, detected during the pilot observations, are already well-studied sources and therefore not included in this paper. These serve as good test cases for our observations:
PKS 1610-77 (J161749-77171): An intervening H i absorption line was detected at
$z=0.450$
in ASKAP commissioning data by Sadler et al. (Reference Sadler, Moss and Allison2020). This object and its environment have been studied in detail by Weng et al. (Reference Weng, Sadler and Foster2022).
PKS 1740-517 (J174425-514442): Allison et al. (Reference Allison, Sadler and Moss2015) detected a strong associated H i absorption line at
$z=0.438$
in ASKAP commissioning data. The field containing this source was observed in the second Pilot Survey as a test, but the poor quality of the data meant that the linefinder results were not reliable.
PKS 1830-21 (J183339-210339): This bright (11 Jy) radio source is a well-studied system in which a
$z=2.51$
quasar is gravitationally lensed by a foreground galaxy at
$z=0.885$
(Subrahmanyan et al. Reference Subrahmanyan, Narasimha, Pramesh Rao and Swarup1990; Wiklind & Combes Reference Wiklind and Combes1996; Lidman et al. Reference Lidman, Courbin and Meylan1999). The radio spectrum shows broad intervening H i and OH absorption in the foreground galaxy (Chengalur et al. Reference Chengalur, de Bruyn and Narasimha1999; Allison et al. Reference Allison, Moss and Macquart2017). The line provides a good test of the 5 MHz beam-forming intervals used for the Pilot 2 observations, and so PKS 1830-211 was used as a ‘quality-gate’ test field at the start of the second FLASH pilot survey.
Appendix C. Additional detection using a redshift prior
Here, we show one example of an H i absorption line that was detected in the Pilot Survey by using a known optical redshift (and previous H i detection) as a prior for the line search. Although this line has a ln (B) value below our detection cutoff (ln (B) = 30), we accept as a genuine detection because of the additional information from the redshift prior. It is likely that other lines of this kind are present in the Pilot Survey data.
Line profile of PKS 0454-23. Blue and red lines are from Gupta et al. (Reference Gupta, Srianand and Petitjean2012). Black is from our observation (SBID 34547_1a). The grey shaded region is the 1
$\sigma$
level of the rms spectral-line noise.
PKS 0454-23 (J045703-232453): This object is a well-studied quasar at redshift
$z=1.003$
(Stickel et al. Reference Stickel, Fried and Kuehr1989) and with a continuum flux density of 3.18 Jy in the ASKAP band. Gupta et al. (Reference Gupta, Srianand and Petitjean2012) detected an intervening H i absorption line at
$z=0.891$
in a targeted observation using the redshift of a known Mg ii absorption system. The line seen by Gupta et al. (Reference Gupta, Srianand and Petitjean2012) is weak and narrow (peak optical depth 0.013, FWHM
$14\pm2$
km s
$^{-1}$
). It was not found in our original linefinder search, but when we use the Gupta et al. (Reference Gupta, Srianand and Petitjean2012) redshift as a prior then the line is marginally detected (ln (B) = 8.53) in our ASKAP spectrum (see Figure A3). We measure a peak optical depth of
$0.009\pm0.002$
and velocity width of
$13.9\pm2.7$
km.s
$^{-1}$
, in reasonable agreement with the Gupta et al. (Reference Gupta, Srianand and Petitjean2012) values.
Appendix D. Individual spectral-line plots
In Figure A4, we plot the line profiles from the co-added spectra for each of the 30 detections listed in Table 3 along with the radio continuum SED fits described in Section 9.2 and based on the methodology of Kerrison et al. (Reference Kerrison, Sadler, Moss and Mahony2024). Each caption includes a brief note on individual objects.
Spectral-line profiles and radio SED fits for: 1. PKS 0011-023 (top), 2. NVSS J002331+010114 (middle), and 3. MRC 0023-482 (bottom). The SED for NVSS J002331+010114 uses LOFAR data at 144 MHz from the LoTSS survey (private communication with J. Callingham) Spectral-line profiles and radio SED fits for: 4. NVSS J014141-231511 (top), 5. NVSS J015516-251423 (middle), and 6. PKS 0253-259 (bottom). Spectral-line profiles and radio SED fits for: 7. PKS 0405-280 (top), 8. SUMSS J045501-423858 (middle), and 9. NVSS J051806-245502 (bottom). Spectral-line profiles and radio SED fits for: 10. MRC 0531-237 (top), 11. NVSS J090331+010846 (middle), and 12. NVSS J090425+012015 (bottom). Note that in the case of NVSS J090331+010846, the SED makes use of an upper limit from LOFAR at 144 MHz (grey arrow) supplied by J. Callingham & T. Shimwell (private communication). The SED for NVSS J090425+012015 uses LOFAR data at 144 MHz from the LoTSS survey supplied by J. Callingham (private communication). Spectral-line profiles and radio SED fits for: 13. NVSS J091256+030021 (top), 14. PKS 0917+18 (middle), and 15. NVSS J092012+161238 (bottom). Spectral-line profiles and radio SED fits for: 16. NVSS J094650-202044 (top), 17. NVSS J100238-195917 (middle), and 18. NVSS J113622+004850 (bottom). Spectral-line profiles and radio SED fits for: 19. NVSS J150506+022927 (top), 20. NVSS J170135-294918 (middle), and 21. PKS 2007-245 (bottom). Spectral-line profiles and radio SED fits for: 22. NVSS 205147+021740 (top), 23. NVSS J215924-241752 (middle), and 24. NVSS J223317-015739 (bottom). Spectral-line profiles and radio SED fits for: 25. J223605-251919 (top), 26. NVSS J223620-222430 (middle), and 27. MRC 2234-254 (bottom). Spectral-line profiles and radio SED fits for: 28. PKS 2311-477 (top), 29. SUMSS J233432-585646 (middle), and 30. NVSS J233702-015209 (bottom).
Appendix E. Individual detection and linefinder results
FLASH Pilot Survey detections and FLASHfinder results; (1) ID; (2) Selavy ID; (3) Component name; (4) Field name; (5) Right Ascension (deg); (6) Declination (deg); (7) Continuum flux density for this component; (8) H i redshift; (9) Peak optical depth; (10) Integrated optical depth; (11) Linewidth; (12) ln (B); (13) Optical redshift; (14) Closest radio source. These spectra were combined into a single spectrum per source as listed in Table 3.
Appendix F. Observation logs with validation flags
Log of observations for FLASH Pilot Survey 1.
aFLASH
${}^{\vphantom{(}}$
305P was observed at an incorrect RA due to a scheduling error and will be reobserved during the Full Survey.
Log of observations for FLASH Pilot Survey 2.
Log of observations for FLASH Pilot Survey 2 re-observed fields.
Open access
