2.1 The solar wind

The solar wind is a supersonic outflow of plasma from the Sun, travelling at speeds ∼400–800 km s⁻¹. The rotation of the Sun means that solar wind structures emanating from a particular point on the Sun’s surface (and the magnetic field lines) describe an Archimedian spiral known as the Parker Spiral (Parker Reference Parker1958). The spiral crosses the Earth’s orbit at an angle of approximately 45° East of the Sun; however as viewed from the Earth, a solar wind feature travelling from the Sun to Earth barely moves on the celestial sphere, since the Earth’s orbital speed is almost negligible compared to typical solar wind speeds.

The density of the radial solar wind would be expected to be proportional to R ⁻² (where R is distance from the Sun). The relevant density for determining how electromagnetic waves propagate through the plasma is the electron number density. The variance of the electron density fluctuations $\delta N_e^2$ removed as a function of distance from the Sun would then be

(1)\begin{equation}\delta N_e^2 \propto R^{-\beta}\end{equation}

with an exponent, β, of 4 (Thompson, Moran, & Swenson Reference Thompson, Moran and Swenson2001). Readhead (Reference Readhead1971) give a power law index of 4.1 ± 0.1 based on IPS observations. Observations at 326 MHz by Manoharan (Reference Manoharan1993) over a range of conditions agree with this value within errors. Much closer to the Sun (2.7°–13.3°), Spangler & Sakurai (Reference Spangler and Sakurai1995) derived a value of β = 3.7 ± 0.3 from VLBI observations of phase scintillations.

The IPS phenomenon itself arises from density fluctuations on scales ∼10–1 000 km. These fluctuations have been studied in detail using a variety of remote sensing techniques, as well as in situ measurements (Coles Reference Coles1978), especially close to the Sun (Coles & Harmon Reference Coles and Harmon1989) where the turbulence is approximately described by a power law, steepening towards smaller scales. This steepening is also seen at longer elongations (Manoharan, Kojima, & Misawa Reference Manoharan, Kojima and Misawa1994). Beyond 1 AU the turbulence continues to evolve towards a Kolmogorov spectrum (Roberts Reference Roberts2010).

The most obvious anisotropy in the solar wind is the existence of the equatorial and polar streams of the solar wind. The polar wind is perhaps 50% faster (Coles et al. Reference Coles1980) and 10–15× less dense than the equatorial solar wind (Coles et al. Reference Coles, Grall, Klinglesmith and Bourgois1995). At solar maximum the volume that the polar wind fills shrinks to just a few percent. However in the declining phase and at solar minimum the polar solar wind is more dominant. The spectrum of density fluctuations is somewhat different for high- and low-speed streams and varies depending on solar activity and over the solar cycle (Manoharan Reference Manoharan1993; Manoharan et al. Reference Manoharan, Kojima and Misawa1994).

In addition to these large-scale structures, there are also transient changes in the solar wind, the most notable of which are coronal mass ejections (CMEs). IPS observations also reveal stochastic changes in scintillation index, implying variations in the mean plasma density. We loosely refer to all of these departures from an isotropic solar wind as ‘space weather’.

2.2 Scintillation

When electromagnetic waves cross an inhomogeneous ionised medium, local density fluctuations will cause the radiation to be deflected. At a sufficient distance, this will increase and decrease the observed flux density, as radiation that has taken slightly different paths to the observer interferes. We give a brief description relevant to IPS here and refer the reader to Narayan (Reference Narayan, Shull and Thronson1993) for further details (see also Thompson et al. Reference Thompson, Moran and Swenson2001 Chapter 13 and the discussion in Section 3.4 of Paper II).

If the scattering is ‘weak’, only radiation crossing the scattering medium within a radius r _F of the point on the line of sight (LoS) from source to observer can contribute to scintillation (where r _F is the Fresnel Scale). The weak regime applies when r _diff ≫ r _F where r _diff is the transverse distance over which the scattering medium produces an rms phase change of one radian. Scintillation in the weak regime is relatively simple to model, even when the scattering occurs at many locations along the LoS, thus most IPS observations are undertaken in the weak regime. The strength of scintillation can be described by the scintillation index m, which is the scintillating flux density (standard deviation) as a fraction of the mean flux density. In weak scintillation m ≪ 1, and m approaches 1 as r _diff approaches r _F. If the source of radiation is in the far field, then θ _F, the angle subtended by r _F at the location of the observer, is given by

(2)\begin{equation}\theta_{\rm F} = \sqrt{\frac{\lambda}{2\pi d}},\end{equation}

where λ is the observed wavelength and d is the distance from the scattering medium to the observer. This gives a value ∼0.3 arcsec for metre-wavelength IPS observations. Sources with a size θ _s < θ _F will exhibit scintillation like a point source. Larger sources have lower scintillation indices than a point source.

In the strong scintillation regime (r _diff ≪ r _F), a sufficiently compact source will show diffractive scintillation with a scintillation index of approximate unity.Footnote ^a However, the critical angular size for a source to be compact is now θ _diff rather than θ _F. Another characteristic of diffractive scintillation is its narrow fractional bandwidth δv/v ∼ (r_diff / r_F)².

2.3 IPS geometry

Figure 1 illustrates the basic geometry of IPS close to the Sun (elongation, ϵ ≲ 70°). The LoS from the source to the Earth passes closer than 1 AU to the Sun. In principle, solar wind turbulence all the way along the LoS contributes to the IPS signal. However, IPS can, to first order, be considered to come from the point of the closest approach of the LoS to the Sun, due to the fact that the turbulence of the solar wind is so much stronger close to the Sun (Little & Hewish Reference Little and Hewish1966). The distance of closest approach to the Sun is the impact parameter p:

(3)\begin{equation}p=\sin\left(\epsilon\right)\,\textrm{AU},\end{equation}

whereas the distance from Earth to the screen d is

(4)\begin{equation}d=\cos\left(\epsilon\right)\,\textrm{AU}.\end{equation}

An empirical relationship between p and the average strength of scintillation was measured by Hewish & Symonds (Reference Hewish and Symonds1969) and has been shown to be valid over a wide range of frequencies and solar elongations (Rickett Reference Rickett1973). For a sufficiently compact source (≲0.3 arcsec), the measured scintillation index m is given by

(5)\begin{equation}m=0.06\lambda^{1.0}p^{-b},\end{equation}

where λ is the wavelength in m, and b ≈ 1.6. This is shown in Figure 2. With a radial solar wind as described by equation (1), b = (β − 1)/2 (Manoharan Reference Manoharan1993). This relationship only applies in the weak scintillation regime. As a source approaches the Sun its scintillation index will increase until finally saturating and turning over as m approaches 1. Thus for a given frequency there is a range of solar elongations which are optimum for IPS observations: too far away and m becomes too small to be easily measurable; too close and the strong scintillation regime is reached. High radio frequencies can be used to probe very close to the Sun (e.g. Tokumaru et al. Reference Tokumaru, Kondo, Mori and Tanaka1994), while at the lower frequencies, IPS can be observed all over the sky (e.g. Bell Reference Bell1968).

Figure 1. Following Bell (Reference Bell1968), the geometry of IPS. Dotted lines show solar radii from 1/16 AU to 1 AU. Solid lines show the LoS for transition into the strong regime at 80 MHz (23°) and 162 MHz (15°) and the approximate longest elongation with well-defined piercepoint (70°). Crosses mark the point of closest approach to the Sun.

Figure 2. Scintillation index m as a function of solar elongation while scintillation remains in weak regime. Dashed line is for 162 MHz; dotted line is for 80 MHz.

The preceding description is only valid for day-side IPS observations close to the Sun. At longer elongations (ϵ ≳ 70°) the distance from the Sun along the LoS does not vary as rapidly, and so the weighting function is not so concentrated on the piercepoint (see Figure 1). A fixed distance of 0.3 AU may be a better estimate for the distance to the scattering screen (Rickett & Coles Reference Rickett and Coles1991). IPS can also be observed on the nightside (e.g. Hewish & Burnell Reference Hewish and Burnell1970; Kaplan et al. Reference Kaplan2015); however, here the impact parameter is even less well defined.

2.4 IPS in the presence of space weather

Equation (5) assumes that the solar wind at a given distance from the Sun does not vary with time or solar latitude, which is not the case. Taking data from two solar cycles, Manoharan (Reference Manoharan1993, Figure 7) find that at solar maximum, the solar wind is indeed radially symmetric on average; however at solar minimum, the contour of average constant scattering describes an ellipsoid which is flattened at the poles. Thus the mean scintillation index would be better described as

(6)\begin{equation}m=0.06\lambda^{1.0}\left(e\,p\right)^{-b},\end{equation}

where e is the elliptical term and

(7)\begin{equation}e=\sqrt{\rho^2\sin^2\phi + \cos^2\phi},\end{equation}

where ρ is the ratio of the equatorial to polar diameters of an ellipsoid of constant scattering (i.e. ρ ≥ 1: 1 at solar maximum, 1.5 at solar minimum) and φ is the latitude of the point on the surface of the Sun directly below the scattering screen.

We reiterate that this elliptical model represents a long-term average. The polar solar winds giving rise to a decrease in scintillation towards the poles occupy different volumes of the heliosphere to the ecliptic solar winds (Coles Reference Coles1996). Even when both streams are contributing to an IPS signal, multi-station IPS observations resolve two distinct peaks in the cross-correlation function (Grail et al. Reference Grail, Coles, Klinglesmith, Breen, Williams, Markkanen and Esser1996). Since the sector of the LoS responsible for IPS is relatively short, especially close to the Sun, the observed scintillation indices close to the boundary between polar and equatorial streams are likely to be quite bimodal, depending on whether the equatorial stream occupies the piercepoint or not (though where both are close to the piercepoint, the much denser equatorial stream will dominate the IPS signal).

In addition to the polar and equatorial solar winds, stochastic changes in solar wind turbulence also cause departures of the observed scintillating index m _obs from the expected scintillation index m (e.g. that predicted by equation 5). These variations are parametrised as the ‘scintillation enhancement factorFootnote ^b’ g, where

(8)\begin{equation}g = \frac{m_{\rm obs}}{m} .\end{equation}

Hajivassiliou (Reference Hajivassiliou1990) (as cited in Hajivassiliou & Duffett-Smith Reference Hajivassiliou1990) characterised the distribution of g values for IPS observations, decomposing them into more extreme events which are easily identified, and less extreme variations which are Gaussian distributed with a standard deviation of approximately 20% at an elongation of 40° from the Sun. This is consistent with the findings presented in Paper II where it was shown that the scintillation indices of flat spectrum sources (which are expected to be compact) followed the relation given in equation (5) with a standard deviation of approximately 20%.

2.5 Optimal detection of compact sources

We now briefly describe our method for detecting compact sources with IPS via interferometric imaging. This is described in detail in Paper I, and the reader is referred to Section 2.3 of that paper for further details. Essentially, the full FoV of the instrument is imaged on a time cadence sufficiently short to critically sample IPS variability. This results in an image cube with time as the third axis. The timeseries corresponding to each spatial pixel is then filtered to emphasise the scintillation timescale of IPS, downweighting variability more rapid than 1 s and slower than 10 s (the latter to filter out ionospheric variability). The rms of each filtered timeseries is then calculated and an image is constructed from these values. This ‘variability image’ has Gaussian noise statistics, and sources detected at a particular level of confidence (e.g. 5–σ) can readily be extracted using standard techniques (e.g. Hancock et al. Reference Hancock, Murphy, Gaensler, Hopkins and Curran2012).

A particular characteristic of searching for signals by their standard deviation alone identified in Paper I is that increased observing time will not lead to a great increase in sensitivity since sensitivity scales with observing time T only as T ^1/4, rather than the usual $\sqrt{T}$. Since greater sensitivity cannot easily be attained by integrating for long, it becomes even more important to observe sources when their scintillation index is maximised. In the remainder of this section we discuss the optimum solar elongation at which sources should be observed.

As discussed above, at a given observing frequency, the scintillation index of a sufficiently compact source will increase as the source approaches the Sun, up to the point where the strong scattering regime is reached. Given that a sufficiently compact source will show diffractive scintillation with a scintillation index of unity, this might suggest that searching for sources in the strong scintillation regime will yield the strongest detections; however, there are a number of reasons why this is not the case. First, as noted in Section 2.2, the critical size for sources to show scintillation is smaller for diffractive scintillation than for weak. If particularly compact sources (e.g. pulsars) are being searched for, this may be a useful extra constraint. However, in general fewer sources will be detectable.

The narrow bandwidth of diffractive scintillation also introduces complications (see Fallows et al. Reference Fallows, Asgekar, Bisi and Breen2013, for a dynamic spectrum of IPS partly in the strong scintillation regime). If the observing bandwidth exceeds the scintillation bandwidth, then it will be necessary to form multiple images across the band to avoid averaging out the scintillations. This is technically feasible for modern correlators; however, it will incur extra processing costs. Additionally, when searching for the IPS signal via its standard deviation, searching over N spectral images reduces sensitivity by N ^1/4, so this should be avoided.

Finally, in the diffractive scintillation regime, the timescale of scintillation will reduce. Our 0.5 s sampling time is only just sufficient to critically sample IPS in the weak regime (the IPS signal remains strong up to at least 0.3 Hz; see Section 2.2.2 of Paper I). Thus faster sampling would be required to fully capture the IPS signature in the strong regime, and this would further reduce sensitivity due to the T ^1/4 problem.

Modelling the scintillation in the transition from weak to strong scattering is difficult, although it has been calculated for the case of thin-screen, isotropic, Kolmogorov turbulence by Goodman & Narayan (Reference Goodman and Narayan2006). For the remainder of this work we take the maximum scintillation index of typical IPS to be 80% as measured empirically in Paper I, and we assume that this value is attained at the solar elongation predicted by equation (6) (i.e. 27° and 18° at 80 and 162 MHz, respectively, for a source on the ecliptic).

In Figures 3 and 4, we combine these limits with equation (6) to show the maximum scintillation index that is reached over the whole year (i.e. the scintillation index at the point of closest approach to the Sun) for a point source for any location on the celestial sphere, both at solar maximum and at solar minimum, for frequencies of 80 and 162 MHz.Footnote ^c

Figure 3. Maximum point-source scintillation index obtained during the year for every point on the celestial sphere for observations at 162 MHz (See equation 6). Equatorial coordinates (RA in hours, Declination in degrees). Inverse hyperbolic sine scale. Top panel is for solar minimum (ρ = 1.5); bottom panel is for solar maximum (ρ = 1.0).

Figure 4. Maximum point-source scintillation index obtained during the year for every point on the celestial sphere for observations at 80 MHz (See equation 6). Equatorial coordinates (RA in hours, Declination in degrees). Inverse hyperbolic sine scale. Top panel is for solar minimum (ρ = 1.5); bottom panel is for solar maximum (ρ = 1.0).

The preceding discussion assumes that we are able to observe a source at whatever solar elongation we wish. This is only the case for sources on the ecliptic. Sources at higher ecliptic latitudes may not approach the Sun closely enough to reach the strong regime, particularly at solar minimum. Furthermore their solar elongation will change relatively slowly over the year compared to a source on the ecliptic. If such regions are surveyed a sufficient number of times, many of the detections will be due to random increases in δN_e causing sources that would usually be below the detection limit to be upscattered. A g-levels of 2.5 (which could increase the scintillation index of a source from 40% to 100%) is possible (see Paper I and Tappin Reference Tappin1986). However, whether or not such occasional detections can lead to an estimate of the compact flux density of the source will depend on the extent to which the scattering along the LoS can be characterised.

2.6 Determining the structure of compact sources

In this section we discuss to what extent the structure of a compact source can be determined from IPS observations. In Paper II the only IPS observable considered was the scintillation index, m. We then used equation (5) to compare the scintillation index of each source to that which would be expected from a point source, and termed this the Normalised Scintillation Index (NSI); thus a source with an NSI of 1 would be compact, whereas a source with NSI of 0.5 would have half its flux density coming from scales smaller than θ _F. Given that variations in the solar wind imposed relatively large errors (≈20%) on the NSI, we classified all of our sources into three broad categories: strong scintillators (NSI ≥ 0.9), which were dominated by a compact source, moderate scintillators (0.4 ≤ NSI < 0.9), and sources with weak or no scintillation (NSI < 0.4).

With multiple observations of each source, it is possible to determine the NSI more accurately. However, there are still a multitude of possible source structures consistent with any given NSI. Some illustrative examples are shown in Figure 5 and the indicative NSI for each is given in Table 1. We define them as follows: a ‘point source’ is a source where all emissions come from scales much smaller than the Fresnel Scale. A ‘slightly extended source’ is one where the emission comes from a scale comparable with the Fresnel scale. An ‘embedded source’ is a point source embedded in much more extended structure. An ‘unresolved double’ is a double source where the double separation is smaller than the Fresnel scale. A ‘resolved double’ is a double source where the double separation is larger than the Fresnel scale. A ‘wide double’ is a double source sufficiently large that its structure can be inferred by other means; for example, The GMRT Sky Survey Alternative Data Release (Intema et al. Reference Intema, Jagannathan, Mooley and Frail2017).

Table 1. NSI for source types illustrated in Figure 5

Real radio sources that could be approximated by each of these examples can be found, so none can be ruled out a priori. Jackson et al. (Reference Jackson2016) used LOFAR (van Haarlem et al. Reference van Haarlem2013) at ∼150 MHz to search for compact calibrators and found a noticeable reduction in correlated flux density on baselines corresponding to spatial scales ∼0.7–2 arcsec. This would suggest that at least some compact sources are slightly extended.

Figure 5. Illustrative source types. Asterisks indicate point-like source, grey scale represents extended emission, and dotted circles indicate Fresnel scale (≈ 0.3 arcsec). Source types are: (a) ‘point source’, (b) ‘slightly extended source’, (c) ‘embedded source’, (d) ‘unresolved double’, (e) ‘resolved double’, and (f) ‘wide double’.

The NSI is not the only observable that an IPS survey would provide. More generally the IPS signal itself is convolved with the source brightness distribution on arcsecond scales, so in principle information on more complex morphologies may be gleaned from IPS. Below we consider various other observables and how they can be used to determine source structure.

2.6.5 Characterisation of source morphologies

In Paper I it was shown that a double structure on scales of a few arcseconds could be detected as a peak in the autocorrelation function (c.f. detection of an emerging double via interstellar scintillation: Macquart & de Bruyn Reference Macquart and de Bruyn2007). The angular separation of source components can be determined if the solar wind speed is known. For sources of the ecliptic, the angle of the solar wind across the source will change with elongation. Figure 6 shows the range of solar wind vectors encountered by a source 18° from the ecliptic while its elongation <70°. Most directions across the source are probed, showing that for most double sources it will be possible to determine their major and minor axes and their position angle. In some cases the solar wind has different turbulence in directions parallel and perpendicular to its velocity. In these cases it may be possible to determine whether the sources are extended parallel or perpendicular to the solar wind velocity. Therefore, in principle it may be possible to reconstruct quite complicated sky brightness distributions.

Figure 6. Polar plot of solar wind vector for a source located at RA 0, Decl. −20° (18° from the ecliptic). Radial distance shows θ _F in arcseconds. Each Black ‘+’ indicates a daily observation while solar elongation <70°. The complex conjugate is plotted in grey since these measurements would give almost identical information. This emphasises that ∼3/4 (rather than ∼3/8) of position angles are probed.

3.1 Sensitivity of the MWA

The MWA is not mechanically steered, instead the signals from static arrays of dipoles are combined with different delays, giving rise to a restricted set of possible pointings where all dipoles are perfectly in phase (these are shown in Figure 7).

Figure 7. Figure showing azimuth and elevation of all MWA ‘sweetspot’ pointings: those for which all 16 tiles are in phase for the location on the sky indicated. Those described in Table 2 are indicated with filled circles.

The sensitivity of the MWA across its full FoV as a function of frequency and pointing and polarisation has been simulated with increasing sophistication and efficiency as described in a series of papers (Sutinjo et al. Reference Sutinjo, O’sullivan, Lenc, Wayth, Padhi, Hall and Tingay2015a, Reference Sutinjo2015b; Sokolowski et al. Reference Sokolowski2017). We draw on the most recent iteration of this modelling to give a general idea of the sensitivity of the MWA as a function of zenith angle in Table 2. The maximum sensitivity of the MWA is achieved when the MWA is pointing at the zenith. At both frequencies, the sensitivity drops off rapidly for pointings at elevations below 45°.

Table 2. Sensitivity as a function of elevation all nine pointings of the MWA along the meridian from zenith to the northern horizon

Due to the fourfold symmetry of MWA tiles, all pointings towards the cardinal points will match with one of these. Other pointings match the general trend with zenith angle within ∼10%. Peak gives the maximum sensitivity of the pointing compared to the centre of the zenith pointing. Area gives the number of square degrees where the sensitivity exceeds half the maximum sensitivity of the zenith pointing.

The system noise of the MWA comes principally from diffuse Galactic emission. This means that the system noise varies by more than a factor of two depending on where on the sky it is pointing (Tingay et al. Reference Tingay2013). Wayth et al. (Reference Wayth2015) describe how the system noise can be estimated for any MWA observation using the beam model and a simple sky model derived from the all-sky single dish survey of Haslam et al. (Reference Haslam, Salter, Stoffel and Wilson1982), scaled to the appropriate frequency.

Thus, with a model for both the beam and the system noise, and knowledge of the state of the array (i.e. how many tiles were active), we are able to predict the sensitivity at each point on the sky for any MWA observation.

3.2 The Sun as seen by the MWA wavelengths

The quiet Sun at MWA wavelengths is a thermal source with a temperature ∼10⁶ K (the inner corona) which appears as an edge-darkened disk somewhat larger than the optical disk of the Sun (Newkirk Reference Newkirk1967; McCauley et al. Reference McCauley, Cairns and Morgan2018). For this wavelength and temperature the Rayleigh–Jeans approximation applies and so the Sun has a power law spectral energy distribution with a spectral index of +2 (i.e. is it brighter at higher frequencies). At 160 MHz the integrated quiet Sun flux density is approximately 50 000 Jy, many orders of magnitude brighter than the astrophysical sources we are probing. Additionally, radio bursts, transient phenomena with complex time and frequency evolution which can last from milliseconds to days, can be many orders of magnitude brighter still than the quiet Sun (Wild, Smerd, & Weiss Reference Wild, Smerd and Weiss1963). McCauley et al. (Reference McCauley, Cairns, Morgan, Gibson, Harding, Lonsdale and Oberoi2017) showcase the complex structure that can be seen on the Sun with the MWA both during a burst and a relatively quiet period.

Two observatories in Australia (as part of an international network) observe the Sun and radio bursts are automatically detected (Lobzin et al. Reference Lobzin, Cairns, Robinson, Steward and Patterson2010). These detections are then curated and archived by the Space Weather Prediction Centre (SWPCFootnote ^d). Thus, we can know which of our observations are affected at least by the brightest radio bursts. See Section 4.2 for further details.

3.3 Observing strategy

As described in the previous section, observations of IPS at our observing frequency are best made at elongations ∼30° from the Sun. Since even the quiet Sun is many orders of magnitude brighter than the astrophysical sources we wish to study, it is imperative that the Sun’s flux density is strongly attenuated if we are to observe IPS without requiring unrealistically high dynamic range imaging.

First we consider placing the Sun at the centre of the FoV. If the Sun were merely a constant source a few times brighter than the astrophysical sources that we are observing, this would be a sensible strategy since placing the source to be subtracted at the centre of the beam gives the best chance of subtracting it cleanly. Unfortunately, with the Sun orders of magnitudes brighter and varying in time and frequency, modelling it well enough to give a dynamic range sufficient to measure IPS on 0.1 Jy sources would in practice be impossible. Lonsdale et al. (Reference Lonsdale2018) have achieved a dynamic range on the Sun of 38 000:1 only with extensive calibration on a single 40 kHz fine channel in a single 0.5 s integration. Replicating this across many thousands of channels and timesteps would be untenable. It is clearly necessary to significantly attenuate the Sun in some way.

Another possibility would be to observe while the Sun is just below the horizon. Table 2 illustrates the beam sensitivity for a selection of pointings at different zenith angles. Reasonable sensitivity is attainable down to approximately 45° from the horizon. This is acceptable but means that at 162 MHz it is not possible to observe sources at the maximum scintillation index.

However, by placing the Sun in one of the primary beam nulls, its effects are minimal, at least in those fields analysed in Papers I and II. Moreover, the typical distance from the Sun to the centre of the field (where the sensitivity is maximum) is typically 25–30°, where the scintillation index is close to maximum.

The major disadvantage of this approach is that it is almost always impossible to null the Sun at two widely spaced frequencies (since the primary beam scales approximately with frequency). However, at lower frequencies the Sun is dimmer, whereas radio telescopes are less sensitive and even compact astrophysical sources (which tend to have flatter spectra than more extended ones) are somewhat brighter. Thus, provided that the Sun is not too active, observations at lower frequencies with the Sun imperfect results still yield useful measurements. In fact, in Paper I, using this approach, we detected almost as many sources in the low band as in the high band (the wider FoV in the low band being a major contributing factor).

3.4 Imaging

MWA images are typically confusion noise limited, which means that the maximum number of sources can be detected with a baseline weighting scheme close to uniform, which will maximise resolution at the expense of increased noise. Briggs −1 (a weighting scheme closer to uniform than natural; Briggs Reference Briggs1995) was used for the GLEAM survey (Hurley-Walker et al. Reference Hurley-Walker2017) and for Papers I&II.

However, as a result of the work presented in Paper I, it is now clear that IPS images are very far from the confusion limit and are limited only by the system noise. This raises the question of whether a more natural weighting scheme might lead to greater sensitivity. Wayth et al. (Reference Wayth2015) suggest a doubling of sensitivity going from Briggs −1 to Briggs +1.

Preliminary work confirms that the system noise in IPS observations does indeed reduce by a factor of two, and that this is a straightforward way to immediately double our sensitivity. There are a few costs to this approach, however. Most obviously, moving to a more natural weighting scheme must mean that the interferometric resolution is decreased. This will not greatly affect the astrometric accuracy, so for an isolated IPS source it should not be any more difficult to match it with a known source (astrometric accuracy scales linearly with both S/N and resolution; see Reid et al. Reference Reid, Schneps, Moran, Gwinn, Genzel, Downes and Roennaeng1988). The reduced resolution will, however, increase the chance that two nearby IPS sources will be confused (manifesting as a reduced scintillation index; see Section 2.6). Furthermore, Briggs +1 MWA images have more pronounced sidelobes than more uniformly weighted MWA images. This is due to the core of the MWA being extremely dense relative to the outer antennas. This leads to areas of very high density in the UV plane. Methods have been developed to distinguish between true scintillating sources and the sidelobes of scintillating sources (see Paper I); however, some increase in noise is unavoidable.

4.1 General observing parameters

Observations commenced on 2015 December 23 and finished on 2016 August 02 when the MWA was shut down for reconfiguration into a new Phase II array (Wayth et al. Reference Wayth2018). Initially a set of observations were made once per week, but this was soon increased to one set of observations per day when the array was available. On occasion, the array was not available for some or all of the day, either due to clashes with other observations or hardware failures. Most notably the array suffered lightning damage in mid-March which could not be repaired for several weeks. Overall, usable observations were made on 91/224 local (UTC+08:00) calendar dates (see Figure 8).

Figure 8. Top panel: Number of observations on each date (UTC+08:00). Greyed observations are not included in further analyses; other coloured observations are coincident with solar radio activity. Middle panel: CMEs per day detected by CACTUS. Bottom panel: Sweep frequency radio bursts per day. Black dots show daily totals; grey lines show moving average.

Observations were scheduled in pairs of consecutive 5-min observations. The Western limb is surveyed over the course of the morning and Eastern limb in the afternoon.

4.2 Space weather context

Figure 9 shows the International Sunspot Number (SILSO World Data Center 2018), the number of CMEs detected by Computer Aided CME Tracking software (CACTus Robbrecht & Berghmans Reference Robbrecht and Berghmans2004) using data from space-based imagers (Brueckner et al. Reference Brueckner1995; Howard et al. Reference Howard2008), and the Radio burst events recorded in the Space Weather Prediction Centre archive.Footnote ^e Note that the latter two are only illustrative since they use the raw numbers from their respective archives: no account is taken of instrument availability, etc.

Figure 9. Top panel: International Sunspot Number (ISSN); Middle panel: black line: CMEs per day detected by CACTUS; grey line: scaled ISSN. Bottom panel: black line: Sweep frequency radio bursts per day; grey line: scaled ISSN. Dotted and dashed lines indicated solar minimum and maximum, respectively. All quantities averaged by calendar month.

These data show that this survey took place when Solar Cycle 24 was in its declining phase, having peaked in April 2014. Nonetheless, there are still a significant number of CMEs (meaning that there will be inhomogeneities in the solar wind) and large numbers of solar radio bursts, which have the potential to interfere with our observations. The number of detected CMEs and solar radio bursts are shown in more detail in Figure 8. We find that 205 (10%) of our observations are concurrent with a burst event; almost all of which are of type III. The majority are actually aggregated into so-called ‘Type VI’ bursts which are defined as a ‘Series of Type III bursts over a period of 10 min or more, with no period longer than 30 min without activity’. So overlap with a Type VI event does not necessarily mean that a burst was coincident with our observation. Moreover, the duration of a Type III burst is just a few seconds, so only a small fraction of an observation would need to be discarded for each one.

4.3 Sky coverage

We now use the methodology laid out in Section 3.1 to estimate the rms noise in a single 0.5 s integration for each point on the sky for each observation, accounting for any antennas that are flagged. From this quantity (denoted μ in Paper I) it is straightforward to calculate the 5–σ limit to which we can detect scintillating flux density (via Paper I Equation 5; see also Paper III Equation 1). We then calculate the compact flux density to which each point on the sky is sensitive using equation (6) with an upper limit on the scintillation index of 0.8 and assuming solar minimum. We further assume that in areas of sky where the scintillation would be in the strong regime (m>1.5 as given by equation 6), no detection is possible. To summarise, we model the observed scintillation m _obs to be

(9)\begin{equation}m_\mathrm{obs} = \left\{ \begin{array}{ll}m, & m \lt 0.8 \\0.8, & 0.8\leq m \lt 1.5 \\0, & \hbox{otherwise,} \\\end{array} \right.\end{equation}

where m is computed via equation (6), with ρ = 1.5, b = 1.6. When we compare our 5–σ limit to that determined for the field analysed in Paper II (which is part of this survey) we find that the Paper II sensitivity was lower by a factor of 1.19 than that expected. Therefore we scale all of our sensitivity maps by this factor. The result of this process is a set of 2 095 all-sky maps, one for each observation. We exclude those maps where the array was not fully functional as described in Section 4.1 and we exclude any observations where the attenuation of the sun was less than a factor of 100. The remaining 1 702 maps are summarised in Figure 10, showing the maximum sensitivity for each point on the celestial sphere, regardless of what observation it lies in. Table 3 shows the area over which we obtain sensitivities ranging from 200 mJy to 4 Jy. Note that while these maps should give a reasonable estimate of our sensitivity, we err on the side of pessimism for the following reasons: (1) we assume the use of the uniform baseline weighting scheme used for Papers I & II since the potential for increasing our sensitivity with a more natural weighting scheme (see Section 3.4) is untested; (2) we assume the largest filling factor (solar minimum) of the fast (polar) solar wind, which is much less dense (see Section 2); (3) rather than allowing scintillation indices to approach or exceed 1, we assume a maximum scintillation index of 0.8 (Section 2.5); (4) we assume that sources cannot be detected once in the strong regime; (5) observations are excluded if the Sun’s primary beam attenuation factor is less than 100; and (6) we assume no upscattering, whereas if a source is observed multiple times, it will most likely be detected below the mean detection limit (Section 2.5).

Figure 10. Maximum sensitivity achieved for each point on the celestial sphere, regardless of which observation it is in for the high band (162 MHz, top) and low band (80 MHz, bottom). Uncoloured areas to the north are below the horizon in all observations.

Table 3. Derived from Figure 10, sky coverage over which we obtain various sensitivities for both the high band (162 MHz) and low band (80 MHz)

Note that the lack of sensitivity around RA 18 h is not due to a lack of observations but a lack of sensitivity of the array due to the presence of the Galactic Plane close to the Galactic Centre in the FoV; a less pronounced lack of sensitivity can also be seen around RA 06 h where our observations cross the Galactic Plane closer to the Galactic Anti-centre (see Section 3.1).

The survey area covers a range of Galactic latitudes, crossing the Galactic Plane close to the Galactic Centre and Anti-centre. Coverage of the Southern Galactic Pole is particularly good, with excellent overlap with well-studied fields such as the MWA EoR-0 field (e.g. Offringa et al. Reference Offringa2016), GAMA (Driver et al. Reference Driver2009), SDSS BOSS (Dawson et al. Reference Dawson2013), and Stripe 82.Footnote ^f

4.4 Expected source counts

Using the coverage maps shown in Figure 10 and the source counts presented in Paper III, it is possible to predict, with some extrapolation down to lower flux densities, the number of sources that we will detect across the full survey. Here we restrict ourselves to the high band, where we have measured source counts, and to areas of sky where our detection limit is 1 Jy or lower. We expect that the total number of compact detections will be 11 000 sources. Of these, 4 500 will be point sources, of which 1 100 will be peaked, 2 100 will be CSS, and 1 200 will be flat-spectrum. This is an increase of over a factor of 50 compared to the single field studied in Paper III and will allow sub-populations to be studied in much greater detail.

These counts do not include pulsars, which will be much more prevalent at lower Galactic latitudes than in the high Galactic latitude field from which these source counts were derived.

4.5 Computational requirements

The steps required to process ∼100 GB of raw data from the MWA archive (the typical amount for an individual 5-min observation) and produce a catalogue of compact sources are discussed in Paper I, Appendix A1. The sizes of various data products are summarised in Table 4. The total computing time needed to process the entire survey is similar to other large MWA projects, around 36 h using the full capacity of a ∼500-node supercomputer such as Galaxy at the Pawsey Supercomputing Centre.Footnote ^g

Table 4. Size of data products

The raw visibilities are currently stored in the MWA data archive. Those in bold are derived data products which will ultimately be made openly available.

5.1 Phase II MWA

The original MWA Phase I configuration used for the survey described in Section 4 was designed to satisfy the requirements of the four main science themes (Bowman et al. Reference Bowman2013) including Epoch of Reionisation studies, which requires high sensitivity on very large spatial scales. As a result, 65% of MWA Phase I baselines are < 750 m in length (corresponding to roughly 10 arcmin resolution at 150 MHz). From mid-2016, a further 128 tiles were added to the MWA (Wayth et al. Reference Wayth2018): some to provide baselines up to 5 km, others to provide further short baselines for EoR studies. It is envisaged that the MWA will switch between just two configurations: a short baseline configuration and a long baseline configuration. Thus any future MWA IPS survey will use one of these two configurations.

The new long-baseline configuration is particularly well suited to IPS observations since it provides extremely uniform coverage of the UV plane. This means that IPS observations with the long baseline Phase II MWA will combine the sensitivity of naturally weighted Phase I images with the exceptional sidelobe suppression of uniformly weighted Phase I images.

The only downsides are that in order to use the very longest baselines it would be necessary to image with higher resolution: increasing the computational requirements and increasing the magnitude of ionospheric effects. However, even if baselines > 3 000 m were excluded (which would reduce the image resolution requirements to those used in Papers I and II) only 15% of baselines would be lost.

Excluding these long baselines does not exclude all baselines to the outer tiles, so the footprint of the array on the ionosphere is significantly larger, increasing ionospheric scintillation and direction-dependent calibration errors. However, it was shown in Paper I that even in very adverse ionospheric conditions the effects on our IPS analysis are minimal.

5.2 Possible future upgrades to the MWA

An obvious next upgrade to the MWA would be to correlate all 256 existing tiles simultaneously. This would result in 3× as many baselines in the range that interests us (350λ–1 500λ), though the dense concentrations of tiles at the core will reintroduce the sidelobes present in the Phase I configuration for naturally weighted images.

Increasing the bandwidth of the MWA may increase the IPS sensitivity slightly; however, the difficulties of keeping the Sun nulled across a wide bandwidth mean that this is unlikely to increase the sensitivity drastically. Perhaps more useful would be an upgrade to the beamformers to allow multiple beams to be observed across the sky, increasing the instantaneous FoV.

If the correlator was upgraded to allow a shorter integration time, this would allow observations further into the strong regime (and improved resolution to detect fine structure for the very brightest sources). As computing power gets cheaper, observing at higher time resolution, and imaging more spectral channels will become more computationally feasible.

5.3 SKA-low

SKA-low is expected to be more than 100× more sensitive than the 128-tile MWA, which would allow scintillating sources to be detected as low as ∼0.2 mJy. The technical requirements which would allow the SKA-low to carry out image-based IPS observations are relatively modest: it is required only that the SKA be able to form images with a cadence of 0.5 s or less and that it can observe within a few tens of degrees of the Sun. However, the value of such observations would be very great: allowing spatial scales well beyond the interferometric resolution of SKA-low to be probed. Papers II–IV of this series showcase some of the extragalactic science which this enables. An additional important science case, using IPS to efficiently discover pulsars, is discussed in Section 5.4 of Paper II.

In contrast to our MWA observations, it cannot be taken for granted that the SKA-low will not be confusion limited. Thus it is useful for us to consider the resolution that would be required for classical confusion to be reduced to the same 0.2 mJy level as the system sensitivity. This requires compact source counts several orders of magnitude in flux density below those we have measured in Paper III, and since no other information on compact sources exist at mJy levels at this frequency, large uncertainties are unavoidable.

In the absence of direct measurements, we calculate the source counts in the relevant flux density range using two different approaches, both of which are illustrated in Figure 11. The simplest approach assumes that the compact source counts have a power law index, γ, exactly matching the standard source counts at 150 MHz (Franzen et al. Reference Franzen2016), but with the scaling factor, k, reduced by a factor of 2.5 to match the lowest flux density counts we were able to measure (i.e. k = 1 800, γ = 1.5).

Figure 11. Euclidean-weighted differential source counts. Grey circles: GLEAM source counts (Franzen et al. in prep. as presented in Paper III); dotted line source counts at 151 MHz as measured by Franzen et al. (Reference Franzen2016); black triangles: source counts of IPS sources as presented in Paper III; dashed line: dotted line scaled to match lowest black triangle; diamonds: 151 MHz IPS source counts predicted by simulation of Wilman et al. (Reference Wilman2008).

The other approach we take is to measure the source counts predicted by the S³ simulation of Wilman et al. (Reference Wilman2008)Footnote ^h. We calculate these simulated counts exactly as described in Section 3.1 of Paper III except that a resolution is 100 arcsec is assumed. As shown in Figure 11, these two approaches agree almost perfectly below 200 mJy, apart from around 1 mJy, where the simulation predicts a population of starforming disks which are sufficiently distant/compact to show IPS.

A full analysis would also take into account the unique way in which noise and confusion manifest when detecting sources via their variability. As described in Paper I (see also Sections 2.5 and 4.3), we search for scintillating sources not in a standard interferometry image but in a ‘variability image’ which has Gaussian noise statistics, but where the value of a pixel corresponding to a scintillating source does not give the variable flux density ΔS, but a related value, which close to the detection limit is proportional to $\sqrt{\Delta S}$ The sources which contribute to confusion are very much in this regime which has the effect of flattening the source counts. Additionally, unless extremely close to each other, the observed flux density will be the flux densities of the individual sources combined in quadrature (see Table 1) which will also have the effect of reducing classical confusion.

Classical confusion is usually considered to be significant when s/b, the number of sources per beam, exceeds 1/50 – 1/10 (e.g. Hogg Reference Hogg2001). Due to the effects above, we adopt the limit of s/b ≤ 1/10.

The SKA Baseline design envisages a compact core with a resolution of 10 arcmin surrounded by three logarithmic spirals with stations out to a radius of 90 km.Footnote ⁱ A more recent proposed configurationFootnote ^j has an even more compact core and a maximum radius of 35 km.

With a resolution of 10 arcmin the classical confusion limit would be reached at approximately 20 mJy.Footnote ^k Clearly, much longer baselines will be required to achieve reasonable sensitivity. At 300 arcsec resolution (optimum sensitivity), the confusion limit is 1.3 mJy. This drops to 0.08 mJy with 150 arcsec resolution. At this resolution the SKA-low configuration referenced above can achieve a sensitivity of approximately 0.5× that of natural weighting with a sidelobe level is ∼1%. With γ = 1.5 there is approximately one source with a S/N ≥ 100 per 100 resolution units. This suggests system noise, classical confusion, and sidelobe confusion are of a similar order of magnitude. However, the main point is that imaging the full 27 deg² FoV of the SKA-low with 150 arcsec resolution is not an onerous task; on the contrary, it requires only moderately longer baselines than the current MWA and produces images with 1/10th of the pixels of a standard MWA image.

Since a 4 -km baseline (corresponding to a resolution of 100 arcsec at 150 MHz) is very small compared to the spatial scales of the turbulence responsible for IPS, we should not need to consider the loss of spatial coherence across the array due to IPS. However, if the SKA data processor allows it, the very longest SKA baselines might be used commensally for multi-station IPS, using beamforming on each of the brighter IPS sources. A 64-km baseline is sufficient to measure the speed (Dennison & Hewish Reference Dennison and Hewish1967; Fallows et al. Reference Fallows, Bisi, Forte, Ulich, Konovalenko, Mann and Vocks2016); however, a baseline of 180 km or more would be necessary to resolve the high- and low-speed streams (Grail et al. Reference Grail, Coles, Klinglesmith, Breen, Williams, Markkanen and Esser1996).