The astronomical context

The goal of density estimation is to estimate the unknown probability density function of a random variable from a set of observations. In more familiar language, density estimation smooths collections of individual measurements into a continuous distribution, smoothing dots on a scatterplot by a curve or surface.

The problem arises in a wide variety of astronomical investigations. Galaxy or lensing distributions can be smoothed to trace the underlying dark matter distribution. Photons in an X-ray or gamma-ray image can be smoothed to visualize the X-ray or gamma-ray sky. Light curves from episodically observed variable stars or quasars can be smoothed to understand the nature of their variability. Star streams in the Galaxy's halo can be smoothed to trace the dynamics of cannibalized dwarf galaxies. Orbital parameters of Kuiper Belt Objects can be smoothed to understand resonances with planets.

Astronomical surveys measure properties of large samples of sources in a consistent fashion, and the objects are often plotted in low-dimensional projections to study characteristics of (sub)populations. Photometric color-magnitude and color-color plots are well-known examples, but parameters may be derived from spectra (e.g. emission-line ratios to measure gas ionization, velocity dispersions to study kinematics) or images (e.g. galaxy morphology measures). In these situations, it is often desirable to estimate the density for comparison with astrophysical theory, to visualize relationships between variables, or to find outliers of interest.

Concepts of density estimation

When the parametric form of the distribution is known, either from astrophysical theory or from a heuristic choice of some simple mathematical form, then the distribution function can be estimated by fitting the model parameters.

Imagers

Imagers capture the two-dimensional pattern of light at the telescope focal plane. They consist of a detector array along with the necessary optics, electronics, and cryogenic apparatus to put the light onto the array at an appropriate angular scale and wavelength range, to collect the resulting signal, and to hold the detector at an optimum temperature while the signal is being collected. Imaging is basic to a variety of investigations, but is also the foundation for the use of other instrument types that need to have their target sources located accurately. In this chapter we discuss the basic design requirements for imagers in the optical and the infrared and guidelines for obtaining good data and reducing it well. We finish with a section on astrometry, a particularly demanding and specialized use of images.

Optical imager design

A simple optical imager consists of a CCD in a liquid nitrogen dewar or cryostat with a window through which the telescope focuses light onto the CCD (see Figure 4.1). Broad spectral bands are isolated with filters, mounted in a wheel or slide to allow different ones to be placed conveniently over the window. Although this imager is conceptually simple, good performance requires attention to detail. For example, if the filters are too close to the CCD, any imperfections or dust on them will produce large-amplitude artifacts in the image.

Basic properties of photodetectors

For nearly a century, photography was central to huge advances in astronomy. Photographic plates were the first detectors that could accumulate long integrations and could store the results for in-depth analysis away from the telescope. They had three major shortcomings, however: (1) they have poor DQE; (2) their response can be nonlinear and complex; and (3) it is impossible to obtain repeated exposures with the identical detector array, an essential step toward quantitative understanding of subtle signals. The further advances with electronic detectors arise largely because they have overcome these shortcomings.

Modern photon detectors operate by placing a bias voltage across a semiconductor crystal, illuminating it with light, and measuring the resulting photo-current. There are a variety of implementations, but an underlying principle is to improve the performance by separating the region of the device responsible for the photon absorption from the one that provides the high electrical resistance needed to minimize noise. Nearly all of these detector types can be fabricated in large-format two-dimensional arrays with multiplexing electrical readout circuits that deliver the signals from the individual detectors, or pixels, in a time sequence. Such devices dominate in the ultraviolet, visible, and near- and mid-infrared. Our discussion describes: (1) the solid-state physics around the absorption process (Section 3.2); (2) basic detector properties (Section 3.3); (3) infrared detectors (Section 3.4); (4) infrared arrays and readouts (Section 3.5); and charge coupled devices (CCDs – Section 3.6). This chapter also describes image intensifiers as used in the ultraviolet, and photomultipliers (Section 3.7). Heritage detectors that operate on other principles are discussed elsewhere (e.g., Rieke 2003, Kitchin 2008).

Preface

Progress in astronomy is fueled by new technical opportunities (Harwit, 1984). For a long time, steady and overall spectacular advances in the optical were made in telescopes and, more recently, in detectors. In the last 60 years, continued progress has been fueled by opening new spectral windows: radio, X-ray, infrared (IR), gamma ray. We haven't run out of possibilities: submillimeter, hard X-ray/gamma ray, cold IR telescopes, multi-conjugate adaptive optics, neutrinos, and gravitational waves are some of the remaining frontiers. To stay at the forefront requires that you be knowledgeable about new technical possibilities.

You will also need to maintain a broad perspective, an increasingly difficult challenge with the ongoing explosion of information. Much of the future progress in astronomy will come from combining insights in different spectral regions. Astronomy has become panchromatic. This is behind much of the push for Virtual Observatories and the development of data archives of many kinds. To make optimum use of all this information requires you to understand the capabilities and limitations of a broad range of instruments so you know the strengths and limitations of the data you are working with.

Uncertainty in observational science

Probability theory models uncertainty. Observational scientists often come across events whose outcome is uncertain. It may be physically impossible, too expensive or even counterproductive to observe all the inputs. The astronomer might want to measure the location and motions of all stars in a globular cluster to understand its dynamical state. But even with the best telescopes, only a fraction of the stars can be located in the two dimensions of sky coordinates with the third distance dimension unobtainable. Only one component (the radial velocity) of the three-dimensional velocity vector can be measured, and this may be accessible for only a few cluster members. Furthermore, limitations of the spectrograph and observing conditions lead to uncertainty in the measured radial velocities. Thus, our knowledge of the structure and dynamics of globular clusters is subject to considerable restrictions and uncertainty.

In developing the basic principles of uncertainty, we will consider both astronomical systems and simple familiar systems such as a tossed coin. The outcome of a toss, heads or tails, is completely determined by the forces on the coin and Newton's laws ofmotion. Butwe would need to measure too many parameters of the coin's trajectory and rotations to predict with acceptable reliability which face of the coin will be up. The outcomes of coin tosses are thus considered to be uncertain even though they are regulated by deterministic physical processes.

The astronomical context

Whenever an astronomer is faced with a dataset that can be presented as a table — rows representing celestial objects and columns representing measured or inferred properties — then the many tools of multivariate statistics come into play. Multivariate datasets also arise in other situations. Astronomical images can be viewed as tables of three variables: right ascension, declination and brightness. Here the spatial variables are in a fixed lattice while the brightness is a random variable. An astronomical datacube has a fourth variable that may be wavelength (for spectro-imaging) or time (for multi-epoch imaging). High-energy (X-ray, gamma-ray, neutrino) detectors give tables where each row is a photon or event with columns representing properties such as arrival direction and energy. Calculations arising from astrophysical models also produce outputs that can be formulated as multivariate datasets, such as N-body simulations of star or galaxy interactions, or hydrodynamical simulations of gas densities and motion.

For multivariate datasets, we designate n for the number of objects in the dataset and p for the number of variables, the dimensionality of the problem. In traditional multivariate analysis, n is large compared to p; statistical methods for high-dimensional problems with p > n are now under development. The variables can have a variety of forms: real numbers representing measurements in any physical unit; integer values representing counts of some variable; ordinal values representing a sequence; binary variables representing “Yes/No” categories; or nonsequential categorical indicators.

We address multivariate issues in several chapters of this volume. The present chapter on multivariate analysis considers datasets that are commonly displayed in a table of objects and properties.

The astronomical context

Spatial data consists of data points in p dimensions, usually p = 2 or 3 dimensions, which can be interpreted as spatial variables. The variables might give locations in astronomical units or megaparsecs, location in right ascension and declination, or pixel locations on an image. Sometimes nonspatial variables are treated as spatial analogs; for example, stellar distance moduli based on photometry or galaxy redshifts based on spectra are common proxies for radial distances that are merged with sky locations to give approximate threedimensional locations.

The methods of spatial point processes are not restricted to spatial variables. They can be applied to any distribution of astronomical data in low dimensions: the orbital distributions of asteroids in the Kuiper Belt; mass segregation in stellar clusters; velocity distributions across a triaxial galaxy or within a turbulent giant molecular cloud; elemental abundance variations across the disk of a spiral galaxy; plasma temperatures within a supernova remnant; gravitational potential variations measured from embedded plasma or lensing distortions of background galaxy shapes; and so forth.

The most intensive study of spatial point processes in astronomy has involved the distribution of galaxies in the two-dimensional sky and in three-dimensional space. One approach, pioneered by Abell (1958), is to locate individual concentrations or “clusters” of galaxies. The principal difficulty is the overlapping of foreground and background galaxies on a cluster, diluting its prominence in two-dimensional projections. The greatest progress is made when spectroscopic redshifts are obtained that, due to Hubble's law of universal expansion, allows the third dimension of galaxy distances to be estimated with reasonable accuracy.

The astronomical context

Time-domain astronomy is a newly recognized field devoted to the study of variable phenomena in celestial objects. They arise from three basic causes. First, as is evident from observation of the Sun's surface, the rotation of celestial bodies produces periodic variations in their appearance. This effect can be dramatic in cases such as beamed emission from rapidly rotating neutron stars (pulsars).

Second, as is evident from observation of Solar System planets and moons, celestial bodies move about each other in periodic orbits. Orbital motions cause periodic variations in Doppler shifts and, when eclipses are seen, in brightness. One could say that the birth of modern time series analysis dates back to Tycho Brahe's accurate measurement of planetary positions and Johannes Kepler's nonlinear models of their behavior.

Third, though less evident from naked eye observations, intrinsic variations can occur in the luminous output of various bodies due to pulsations, explosions and ejections, and accretion of gas from the environment. The high-energy X-ray and gamma-ray sky is particularly replete with highly variable sources. Classes of variable objects include flares from magnetically active stars, pulsating stars in the instability strip, accretion variations from cataclysmic variable and X-ray binary systems, explosions seen as supernovae and gamma-ray bursts, accretion variations in active galactic nuclei (e.g. Seyfert galaxies and quasi-stellar objects, quasars and blazars), and the hopeful detection of gravitational wave signals. A significant fraction of all empirical astronomical studies concerns variable phenomena; see the review by Feigelson (1997) and the symposium New Horizons in Time Domain Astronomy (Griffin et al. 2012).

Introduction

Spectrometers divide the light centered at wavelength λ into narrow spectral ranges, Δλ. If the resolution R = λ/Δλ > 10, the goals of the observation are generally different from those in photometry, including both measuring spectral lines and characterizing broad features.

There are three basic ways of measuring light spectroscopically:

1. Differential-refraction-based, in which the variation of refractive index with wavelength of an optical material is used to separate the wavelengths, as in a prism spectrometer.

2. Interference-based, in which the light is divided so a phase-delay can be imposed on a portion of it. When the light is re-combined, interference among components is at different phases depending on the wavelength, allowing extraction of spectral information. The most widely used examples are diffraction grating, Fabry–Perot, and Fourier spectrometers. Heterodyne spectroscopy also falls into this category, but we will delay discussing it until we reach the submillimeter and radio regimes in Chapter 8.

3. Bolometrically, in which the signal is based on the energy of the absorbed photon. This method is applied in the X-ray, for example, using CCDs or bolometers, and will be discussed in Chapter 10.