The astronomical context

Observational astronomers always struggle, and often fail, to characterize celestial populations in an unbiased fashion. Many surveys are flux-limited (or, as expressed in traditional optical astronomy, magnitude-limited) so that only the brighter objects are detected. As flux is a convolution of the object's intrinsic luminosity and the (often uninteresting) distance to the observer according to Flux = L/4πd2, this produces a sample with a complicated bias in luminosity: high-luminosity objects at large distances are over-represented and lowluminosity objects are under-represented in a flux-limited survey. This and related issues with nondetections have confronted astronomers for nearly 200 years.

A blind astronomical survey of a portion of the sky is thus truncated at the sensitivity limit, where truncation indicates that the undetected objects, even the number of undetected objects, are entirely missing from the dataset. In a supervised astronomical survey where a particular property (e.g. far-infrared luminosity, calcium line absorption, CO molecular line strength) of a previously defined sample of objects is sought, some objects in the sample may be too faint to detect. The dataset then contains the full sample of interest, but some objects have upper limits and others have detections. Statisticians refer to upper limits as left-censored data points.

Multivariate problems with censoring and truncation also arise in astronomy. Consider, for example, a study of howthe luminosity function of active galactic nuclei (AGN) depends on covariates such as redshift (as a measure of cosmic time), clustering environment, host galaxy bulge luminosity and starburst activity.

The astronomical context

Our astronomical knowledge of planets, stars, the interstellarmedium, galaxies or accretion phenomena is usually limited to a few observables that give limited information about the underlying conditions. Our astrophysical understanding usually involves primitive models of complex processes operating on complex distributions of atoms. In light of these diffi- culties intrinsic to astronomy, there is often little basis for assuming particular statistical distributions of and relationships between the observed variables. For example, astronomers frequently take the log of an observed variable to reduce broad ranges and to remove physical units, and then assume with little justification that their residuals around some distribution are normally distributed. Few astrophysical theories can predict whether the scatter in observable quantities is Gaussian in linear, logarithmic or other transformation of the variable.

Astronomersmay commonly use a simple heuristic model in situationswhere there is little astrophysical foundation, or where the underlying phenomena are undoubtedly far more complex. Linear or loglinear (i.e. power-law) fits are often used to quantify relationships between observables in starburst galaxies, molecular clouds or gamma-ray bursts where the statistical model has little basis in astrophysical theory. In such cases, the mathematical assumptions of the statistical procedures are often not established, and the choice of a simplistic model may obfuscate interesting characteristics of the data.

Feigelson (2007) reviews one of these situations where hundreds of studies over seven decades modeled the radial starlight profiles of elliptical galaxies using simple parametric models, see king to understand the distribution of mass within the galaxies.

Introduction

The Byrd Green Bank Telescope is the largest fully steerable filled-aperture radio telescope, with a size of 100 × 110 meters. The runners-up are the Effelsberg telescope, with a diameter of 100 meters, and the Jodrell Bank Lovell Telescope, 76 meters in diameter. The collecting areas of these telescopes are awesome, but their angular resolutions are poor: at λ = 21 cm, λ/D for a 100-m telescope is about 7 arcmin. These enormous telescope structures are difficult to keep in accurate alignment and are subject to huge forces from wind. Proposals for larger telescopes (e.g., the Jodrell Bank Mark IV and V telescopes at 305 and 122 m respectively) pose major engineering challenges for modest improvements in resolution and have also proven to press the limits of what other humans are willing to purchase for astronomers. The Arecibo disk circumvents some of these issues by being fixed in the ground and pointing by moving its feed; this concept allows a diameter of 259 m. However, the dish must be spherical, limiting the optical accuracy of the telescope, and it can reach only a fraction of the sky (roughly ± 20° from the zenith). The angular resolution is still modest, a bit worse than 2 arcmin at 21 cm (1.4 GHz). This resolution is only equivalent to that of the unaided human eye!

Even if we could build larger telescopes, they would be limited by source confusion: in very deep observations, the background of distant galaxies is so complex to these relatively large beams that it becomes impossible to distinguish an object from its neighbors (see Section 1.5.4).

Multivariate analysis discussed in Chapter 8 seeks to characterize structural relationships among the p variables that may be present in addition to random scatter. The primary structural relations may link the subpopulations without characterizing the structure of any one population.

In such cases, the scientist should first attempt to discriminate the subpopulations. This is the subject of multivariate clustering and classification. Clustering refers to situations where the subpopulations must be estimated from the dataset alone whereas classification refers to situationswhere training datasets of known populations are available independently of the dataset under study. When the datasets are very large with well-characterized training sets, classification is a major component of data mining. The efforts to find concentrations in a multivariate distribution of points are closely allied with clustering analysis of spatial distribution when p = 2 or 3; for such low-p problems, the reader is encouraged to examine Chapter 12 along with the present discussion.

The astronomical context

Since the advent of astrophotography and spectroscopy over a century ago, astronomers have faced the challenge of characterizing and understanding vast numbers of asteroids, stars, galaxies and other cosmic populations. A crucial step towards astrophysical understanding was the classification of objects into distinct, and often ordered, categories which contain objects sharing similar properties. Over a century ago, A. J. Cannon examined hundreds of thousands of low-resolution photographic stellar spectra, classifying them in the OBAFGKM sequence of decreasing surface temperature.

Astronomical context

Astronomers fit data both to simple phenomenological relationships and to complex non-linear models based on astrophysical understanding of the observed phenomenon. The first type often involves linear relationships, and is common in other fields such as social and biological sciences. Examples might include characterizing the Fundamental Plane of elliptical galaxies or the power-law index of solar flare energies. Astrophysicists may have some semi-quantitative explanations for these relationships, but they typically do not arise from a well-established astrophysical process.

But the second type of statistical modeling is not seen outside of the physical sciences. Here, providing the model family truly represents the underlying phenomenon, the fitted parameters give insights into sizes, masses, compositions, temperatures, geometries and other physical properties of astronomical objects. Examples of astrophysical modeling include:

• Interpreting the spectrum of an accreting black hole such as a quasar. Is it a nonthermal power law, a sum of featureless blackbodies, and/or a thermal gas with atomic emission and absorption lines?

• Interpreting the radial velocity variations of a large sample of solar-like stars. This can lead to discovery of orbiting systems such as binary stars and exoplanets, giving insights into star and planet formation. Is a star orbited by two planets or four planets?

• Interpreting the spatial fluctuations in the Cosmic Microwave Background radiation. What are the best-fit combinations of baryonic, darkmatter and dark energy components? Are Big Bang models with quintessence or cosmic strings excluded?

The goals of astronomical modeling also differ from many applications in social science or industry.

Introduction

The submillimeter and millimeter-wave regime – roughly λ = 0.2 mm to 3 mm – represents a transition between infrared and radio (λ > 3 mm) methods. Because of the infinitesimal energy associated with a photon, photodetectors are no longer effective and we must turn to the alternative two types described in Section 1.4.2. Thermal detectors – bolometers – are useful at low spectral resolution. For high-resolution spectroscopy (and interferometers), coherent detectors are used. Coherent detectors – heterodyne receivers – dominate the radio regime for both low and high spectral resolution (Wilson et al. 2009).

As the wavelengths get longer, the requirements for optics also change. The designs of the components surrounding bolometers and submm- and mm-wave mixers must take account of the wave nature of the energy to optimize the absorption efficiency. “Pseudo-optics” are employed, combining standard lenses and mirrors with components that concentrate energy without necessarily bringing it to a traditional focus. In the radio region, non-optical techniques are used to transport and concentrate the photon stream energy. For example, energy can be conveyed long distances in waveguides, hollow conductors designed to carry signals through resonant reflection from their walls. At higher frequencies, strip lines or microstrips can be designed to have some of the characteristics of waveguides; they consist of circuit traces on insulators and between or over ground planes.