Hostname: page-component-77c89778f8-vpsfw Total loading time: 0 Render date: 2024-07-20T15:59:51.026Z Has data issue: false hasContentIssue false

Fast Transient Detection as a Prototypical “Big Data” Problem

Published online by Cambridge University Press:  20 April 2012

Dayton L. Jones
Affiliation:
Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91109, USA email: dayton.jones@jpl.nasa.gov
Kiri Wagstaff
Affiliation:
Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91109, USA email: dayton.jones@jpl.nasa.gov
David Thompson
Affiliation:
Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91109, USA email: dayton.jones@jpl.nasa.gov
Larry D'Addario
Affiliation:
Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91109, USA email: dayton.jones@jpl.nasa.gov
Robert Navarro
Affiliation:
Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91109, USA email: dayton.jones@jpl.nasa.gov
Chris Mattmann
Affiliation:
Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91109, USA email: dayton.jones@jpl.nasa.gov
Walid Majid
Affiliation:
Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91109, USA email: dayton.jones@jpl.nasa.gov
Umaa Rebbapragada
Affiliation:
Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91109, USA email: dayton.jones@jpl.nasa.gov
Joseph Lazio
Affiliation:
Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91109, USA email: dayton.jones@jpl.nasa.gov
Robert Preston
Affiliation:
Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91109, USA email: dayton.jones@jpl.nasa.gov
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

The detection of fast (< 1 second) transient signals requires a challenging balance between the need to examine vast quantities of high time-resolution data and the impracticality of storing all the data for later analysis. This is the epitome of a “big data” issue—far more data will be produced by next generation-astronomy facilities than can be analyzed, distributed, or archived using traditional methods. JPL is developing technologies to deal with “big data” problems from initial data generation through real-time data triage algorithms to large-scale data archiving and mining. Although most current work is focused on the needs of large radio arrays, the technologies involved are widely applicable in other areas.

Type
Contributed Papers
Copyright
Copyright © International Astronomical Union 2012

References

D'Addario, L. 2011 Square Kilometre Array Memo, 133, http://www.skatelescope.org/Google Scholar
Brisken, W., et al. , 2011, Proc. National Radio Science Meeting, Boulder, CO, 2011 January 6Google Scholar
Thompson, D. R., et al. , 2011, ApJ, 735, 98Google Scholar
Wayth, R. B., et al. , 2011, ApJ, 735, 97CrossRefGoogle Scholar