The role of statistics in astronomy

Astronomy and astrophysics

Today, the term “astronomy” is best understood as shorthand for “astronomy and astrophysics”. Astronomy (astro = star and nomen = name in ancient Greek) is the observational study of matter beyond Earth: planets and bodies in the Solar System, stars in the Milky Way Galaxy, galaxies in the Universe, and diffuse matter between these concentrations of mass. The perspective is rooted in our viewpoint on or near Earth, typically using telescopes on mountaintops or robotic satellites to enhance the limited capabilities of our eyes. Astrophysics (astro =star and physis =nature) is the study of the intrinsic nature of astronomical bodies and the processes by which they interact and evolve. This is an indirect, inferential intellectual effort based on the (apparently valid) assumption that physical processes established to rule terrestrial phenomena – gravity, thermodynamics, electromagnetism, quantum mechanics, plasma physics, chemistry, and so forth – also apply to distant cosmic phenomena. Figure 1.1 gives a broad-stroke outline of the major fields and themes of modern astronomy.

The fields of astronomy are often distinguished by the structures under study. There are planetary astronomers (who study our Solar System and extra-solar planetary systems), solar physicists (who study our Sun), stellar astronomers (who study other stars), Galactic astronomers (who study our Milky Way Galaxy), extragalactic astronomers (who study other galaxies), and cosmologists (who study the Universe as a whole).

History and scope of R/CRAN

The development of R (R Development Core Team, 2010) as an independent public-domain statistical computing environment was started in the early 1990s by two statisticians at the University of Auckland, Ross Ihaka and Robert Gentleman. They decided to mimic the Ssystem developed at AT&T during the 1980s by John Chambers and colleagues. By the late 1990s, R development was expanded to a larger core group, and the Comprehensive R Archive Network (CRAN) was created for specialized packages. The group established itself as a non-profit R Foundation based in Vienna, Austria, and began releasing the code biannually as a GNU General Public License software product (Ihaka & Gentleman 1996).

R grew dramatically, both in content and in widespread usage, during the 2000s. CRAN increased exponentially with ∼100 packages in 2001, ∼600 in 2005, ∼2500 in 2010, and ∼3,300 by early 2012. The user population is uncertain but was estimated to be ∼2 million people in 2010.

R consists of a collection of software for infrastructure analysis, and about 25 important packages providing a variety of important data analysis, applied mathematics, statistics, graphics and utilities packages. The CRAN add-on packages are mostly supplied by users, sometimes individual experts and sometimes significant user communities in biology, chemistry, economics, geology and other fields. Tables B.1 and B.2 give a sense of the breadth of methodology in R as well as CRAN packages (up to mid-2010).

As demonstrated throughout this volume, astronomical statistical problems are remarkably varied, and no single dataset can exemplify the range of methodological issues raised in modern research. Despite the range and challenges of astronomical data analysis, few astronomical datasets appear in statistical texts or studies. The Zurich (or Wolff) sunspot counts over ∼200 years showing the 11 year cycle of solar activity is most commonly seen (Section C.13).

We present 20 datasets in two classes drawn from contemporary research. Thirteen datasets are used for R applications in this volume; they are listed in Table C.1 and described in Sections C.1–C.13. The full datasets are available on-line at http://astrostatistics.psu.edu/MSMA formatted for immediate use in R. Six additional datasets that, as of this writing, are dynamically changing due to continuing observations are listed in Table C.2 and described in Sections C.14–C.19. Most of these are time series of variable phenomena in the sky.

Tables C.1–C.2 provide a brief title and summary of statistical issues treated in each dataset. Here Nd is the number of datasets, n is the number of datapoints, and p is the dimensionality. In the sections below, for each dataset we introduce the scientific issues, describe and tabulate a portion of the dataset, and outline appropriate statistical exercises.

The datasets presented here can be used for classroom exercises involving a wide range of statistical analyses. Some problems are straightforward, others are challenging but within the scope of R and CRAN, and yet others await advances in astrostatistical methodology and can be used for research purposes.

Introduction

For many years, X-ray astronomy depended on gaseous detectors: basically, capacitors or series of capacitors with a voltage across them and filled with gas. Depending upon the value of the voltage, these devices: (1) just collect the charge freed when an energetic particle interacts with the gas (ionization chamber); or (2) provide gain [in order of increasing voltage, exciting the gas to produce ultraviolet light (scintillation proportional counter); creating modest-sized ionization avalanches to provide gain but with signals still in proportion to the original number of freed electrons (proportional counter); providing gain to saturation (Geiger counter); yielding a visible spark along the path of the avalanche (spark chamber)]. The absorbing gas is typically argon or xenon, for which high absorption efficiency in the 0.1–10 keV range requires a path of order 1 cm. Very thin windows are required to admit the X-rays to the sensitive volume – for example, 1 μm of polypropylene to provide > 80% transmission for energies above 0.9 keV. Proportional counters with multiple anode wires provide spatial resolution of a few hundreds of microns.

Because the atmosphere is opaque to them, X-rays and gamma rays require telescopes and detectors to operate from balloons, or more commonly from space (with the exception of the highest-energy gamma rays). Initially, the detectors were used without collecting optics; the large detector areas then resulted in high spurious detection rates due to cosmic rays. Anti-coincidence counters were required to identify charged particles coming from random directions, allowing probable X-ray events to be isolated. The necessity to operate at the top of or above the atmosphere plus these requirements on the detector systems were very limiting in terms of the angular resolution and sensitive areas that could be achieved.

Introduction

Most of what we know about astronomical sources comes from measuring their spectral energy distributions (SEDs) or from taking spectra. We can distinguish the two approaches in terms of the spectral resolution, defined as R = λ/Δλ, where λ is the wavelength of observation and Δλ is the range of wavelengths around λ that are combined into a single flux measurement. Photometry refers to the procedures for measuring or comparing SEDs and is typically obtained at R ~ 2–10. It is discussed in this chapter, while spectroscopy (with R ≥ 10) is described in the following one.

In the optical and near-infrared, nearly all the initial photometry was obtained on stars, whose SEDs are a related family of modified blackbodies with relative characteristics determined primarily by a small set of free parameters (e.g., temperature, reddening, composition, surface gravity). Useful comparisons among stars can be obtained relatively easily by defining a photometric system, which is a set of response bands for the [(telescope)-(instrument optics)-(defining optical filter)-(detector)] combination. Comparisons of measurements of stars with such a system, commonly called colors, can reveal their relative temperatures, reddening, and other parameters. Such comparisons are facilitated by defining a set of reference stars whose colors have been determined accurately and that can be used as transfer standards from one unknown star to another. This process is called classical stellar photometry. It does not require that the measurements be converted into physical units; all the results are relative to measurements of a network of stars. Instead, its validity depends on the stability of the photometric system and the accuracy with which it can be reproduced by other astronomers carrying out comparable measurements.

The astronomical context

Statistical inference helps the scientist to reach conclusions that extend beyond the obvious and immediate characterization of individual datasets. In some cases, the astronomer measures the properties of a limited sample of objects (often chosen to be brighter or closer than others) in order to learn about the properties of the vast underlying population of similar objects in the Universe. Inference is often based on a statistic, a function of random variables. At the early stages of an investigation, the astronomermight seek simple statistics of the data such as the average value or the slope of a heuristic linear relation. At later stages, the astronomer might measure in great detail the properties of one or a few objects to test the applicability, or to estimate the parameters, of an astrophysical theory thought to underly the observed phenomenon.

Statistical inference is so pervasive throughout these astronomical and astrophysical investigations that we are hardly aware of its ubiquitous role. It arises when the astronomer:

– smooths over discrete observations to understand the underlying continuous phenomenon

– seeks to quantify relationships between observed properties

– tests whether an observation agrees with an assumed astrophysical theory

– divides a sample into subsamples with distinct properties

– tries to compensate for flux limits and nondetections

– investigates the temporal behavior of variable sources

– infers the evolution of cosmic bodies from studies of objects at different stages

– characterizes and models patterns in wavelength, images or space

and many other situations.

Motivation and goals

For many years, astronomers have struggled with the application of sophisticated statistical methodologies to analyze their rich datasets and address complex astrophysical problems. On one hand, at least in the United States, astronomers receive little or no formal training in statistics. The traditional method of education has been informal exposure to a few familiar methods during early research experiences. On the other hand, astronomers correctly perceive that a vast world of applied mathematical and statistical methodologies has emerged in recent decades. But systematic, broad training in modern statistical methods has not been available to most astronomers.

This volume seeks to address this problem at three levels. First, we present fundamental principles and results of broad fields of statistics applicable to astronomical research. The material is roughly at a level of advanced undergraduate courses in statistics. We also outline some recent advanced techniques that may be useful for astronomical research to give a flavor of the breadth of modern methodology. It is important to recognize that we give only incomplete introductions to the fields, and we guide the astronomer towards more complete and authoritative treatments.

Second, we present tutorials on the application of both simple and more advanced methods applied to contemporary astronomical research datasets using the R statistical software package. R has emerged in recent years as the most versatile public-domain statistical software environment for researchers in many fields.