Philosophical Foundations of Causality

When discussing causality and causal inference we must first distinguish between the thing itself and how to recognize it. Most scientific work on causality involves developing methods for providing evidence for causal relationships, while work in philosophy addresses what it means for something to be a cause. This philosophical work is not immediately applicable to practical problems, but it provides a necessary starting point for work by computer scientists, epidemiologists, and economists. This section introduces readers not familiar with the philosophical literature to how philosophers have conceptualized causality and why this problem is still unsolved after centuries of work. I begin with a review of the primary ways philosophers have addressed causality leading up to more recent probabilistic methods. The review is not an unbiased survey of causality, but rather a discussion of its philosophical foundations through the lens of researchers aiming to build inference methods upon them. As a result, I omit large bodies of work such as process-based theories (Dowe, 2000; Salmon, 1994) and mechanistic models (Glennan, 1996; Machamer et al., 2000) because knowledge of these is not required to understand the later sections. I also raise concerns (such as computational complexity) that differ from those of philosophers but are important when translating these methods to practice.

While Aristotle is often credited with the first formal theory of causality in his Physics and Metaphysics, the most influential modern discussion of causality comes from David Hume in the 18th century. Hume attempted to define both what a cause is and what is meant by the term; as well as how we can come to possess causal knowledge and what is needed to infer it from observations. The core of Hume’s work is arguing that we come to know of causal relationships by inferring them from observations, so they may also be subjective due to beliefs and perception.

Distributed and parallel processing of very large datasets has been employed for decades in specialized, high-budget settings, such as financial and petroleum industry applications. Recent years have brought dramatic progress in usability, cost effectiveness, and diversity of parallel computing platforms, with their popularity growing for a broad set of data analysis and machine learning tasks.

The current rise in interest in scaling up machine learning applications can be partially attributed to the evolution of hardware architectures and programming frameworks that make it easy to exploit the types of parallelism realizable in many learning algorithms. A number of platforms make it convenient to implement concurrent processing of data instances or their features. This allows fairly straightforward parallelization of many learning algorithms that view input as an unordered batch of examples and aggregate isolated computations over each of them.

Increased attention to large-scale machine learning is also due to the spread of very large datasets across many modern applications. Such datasets are often accumulated on distributed storage platforms, motivating the development of learning algorithms that can be distributed appropriately. Finally, the proliferation of sensing devices that perform real-time inference based on high-dimensional, complex feature representations drives additional demand for utilizing parallelism in learning-centric applications. Examples of this trend include speech recognition and visual object detection becoming commonplace in autonomous robots and mobile devices.