Hostname: page-component-5d84bcc8dc-d774m Total loading time: 0 Render date: 2026-09-06T13:40:33.294Z Has data issue: false hasContentIssue false

A Quantitative Method for Substantive Robustness Assessment

Published online by Cambridge University Press:  23 September 2014

Abstract

Empirical political science is not simply about reporting evidence; it is also about coming to conclusions on the basis of that evidence and acting on those conclusions. But whether a result is substantively significant––strong and certain enough to justify acting upon the belief that the null hypothesis is false––is difficult to objectively pin down, in part because different researchers have different standards for interpreting evidence. Instead, this article advocates judging results according to their “substantive robustness,” the degree to which a community with heterogeneous standards for interpreting evidence would agree that the result is substantively significant. This study illustrates how this can be done using Bayesian statistical decision techniques. Judging results in this way yields a tangible benefit: false positives are reduced without decreasing the power of the test, thus decreasing the error rate in published results.

Information

Type
Original Articles
Copyright
Copyright © The European Political Science Association 2014 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 613 Bytes
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 8.1 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 4.5 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 6.4 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 11.3 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 8.1 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 8.1 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 101 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 8.1 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 8.1 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 690 Bytes
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 11.4 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 7.8 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 7.6 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 9.9 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 25.5 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 102.3 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 1.1 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 9.5 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 10.7 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 2.5 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 9.5 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 8.3 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 8.5 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 6 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 1 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 5.4 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 1.2 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 1.2 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 2.6 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 6.2 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 1 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 5.6 KB
Supplementary material: File

Esarey and Danneman Supplementary Material

Supplementary Material

Download Esarey and Danneman Supplementary Material(File)
File 1.3 KB