Hostname: page-component-848d4c4894-x5gtn Total loading time: 0 Render date: 2024-05-02T04:57:57.369Z Has data issue: false hasContentIssue false
  • English
  • Français

Quality in Measurement Matters: Adjusted American Bar Association Ratings and Circuit Court Confirmation Hearing Word Choice

Published online by Cambridge University Press:  10 January 2024

James A. Sieja
James A. Sieja*
Affiliation:
Assistant Professor of Government, St. Lawrence University, Canton, NY, USA
*
Email: jsieja@stlawu.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Though widely used in studies of judicial politics, American Bar Association (ABA) ratings have a partisan bias. As a result, when researchers include ABA ratings and ideology in a model together, the results may be biased toward non-findings with respect to the effect of ideology, qualifications, or both. This study leverages new data on the ABA rating process to create a valid and reliable new measure for the qualifications of nominees to the US Courts of Appeals. In an empirical example, I test the new measure against alternative specifications to demonstrate its potential. The empirical example also presents a new data set on circuit court confirmation hearing speech. The findings contrast with well-established conclusions from previous studies.

Keywords

judicial politicsmeasurementqualificationsnominationsconfirmation hearings
Type
Research Article
Information
Journal of Law and Courts , First View , pp. 1 - 22
Copyright
© The Author(s), 2024. Published by Cambridge University Press on behalf of the Law and Courts Organized Section of the American Political Science Association

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

1

All data to reproduce the tables and figures in this publication are available on the author’s website and in the Journal’s Dataverse archive.

References

ABA Watch. 2006. A Focus On: The ABA Standing Committee on Federal Judiciary. ABA Watch (August). Washington, DC: The Federalist Society for Law and Public Policy Studies.Google Scholar
Allison, Garland. 1996. “Delay in Senate Confirmation of Federal Judicial Nominees.” Judicature 80: 8–15.Google Scholar
American Bar Association. 2009. “Standing Committee on the Federal Judiciary: What It Is and How It Works.” http://www.americanbar.org/content/dam/aba/uncategorized/GAO/Backgrounder.authcheckdam.pdf.Google Scholar
American Bar Association. 2020. “Standing Committee on the Federal Judiciary: What It Is and How It Works.” https://www.americanbar.org/content/dam/aba/administrative/government_affairs_office/backgrounder-9-21-2020.pdf.Google Scholar
Benesh, Sarah, Armstrong, David, and Wallander, Zachary. 2020. “Advisors to Elites: Untangling Their Effect.” Journal of Law and Courts 8 (1): 51–73.CrossRefGoogle Scholar
Binder, Sarah, and Maltzman, Forrest. 2002. “Senatorial Delay in Confirming Federal Judges, 1947–1998.” American Journal of Political Science 46: 190–199.CrossRefGoogle Scholar
Binder, Sarah, and Maltzman, Forrest. 2009a. “Advice and Consent during the Bush Years: The Politics of Confirming Federal Judges.” Judicature 92: 320–329.Google Scholar
Binder, Sarah, and Maltzman, Forrest. 2009b. Advice and Dissent: The Struggle to Shape the Federal Judiciary. Washington, DC: Brookings.Google Scholar
Black, Ryan, and Boyd, Christina. 2011. “The Role of Law Clerks in the U.S. Supreme Court’s Agenda-Setting Process.” American Politics Research 40: 147–173.CrossRefGoogle Scholar
Black, Ryan, Madonna, Anthony, and Owens, Ryan. 2014. “Qualifications or Philosophy? The Use of Blue Slips in a Polarized Era.” Presidential Studies Quarterly 44 (2): 290–308.CrossRefGoogle Scholar
Black, Ryan, and Owens, Ryan. 2013. “Bargaining and Legal Development in the United States Courts of Appeals.” American Politics Research 41: 1071–1101.CrossRefGoogle Scholar
Cameron, Charles, Cover, Albert, and Segal, Jeffrey. 1990. “Senate Voting on Supreme Court Nominees: A Neoinstitutional Model.” American Political Science Review 84: 525–534.CrossRefGoogle Scholar
Choi, Stephen, Gulati, Mitu, and Posner, Eric. 2011. “What Do Federal District Judges Want? An Analysis of Publications, Citations, and Reversals.” Journal of Law, Economics, and Organization 28 (3): 518–549.CrossRefGoogle Scholar
Choi, Stephen, Gulati, Mitu, and Posner, Eric. 2013. “How Well Do Measures of Judicial Ability Predict Judicial Performance?: A Case Study Using Securities Class Actions.” International Review of Law and Economics 33: 37–53.CrossRefGoogle Scholar
Christensen, Robert, and Szmer, John. 2012. “Examining the Efficiency of the U.S. Courts of Appeals: Pathologies and Prescriptions.” International Review of Law and Economics 32: 30–37.CrossRefGoogle Scholar
Collins, Paul, and Ringhand, Lori. 2013. Supreme Court Confirmation Hearings and Constitutional Change. New York: Cambridge University Press.CrossRefGoogle Scholar
Dancey, Logan, Nelson, Kjersten, and Ringsmuth, Eve. 2014. “Individual Scrutiny or Politics as Usual? Senatorial Assessment of U.S. District Court Nominees.” American Politics Research 42 (5): 784–814.CrossRefGoogle Scholar
Dancey, Logan, Nelson, Kjersten, and Ringsmuth, Eve. 2020. It’s Not Personal: Politics and Policy in Lower Court Confirmation Hearings. Ann Arbor, MI: University of Michigan Press.CrossRefGoogle Scholar
Diamond, Alexis, and Sekhon, Jasjeet. 2013. “Genetic Matching for Estimating Causal Effects: A General Multivariate Matching Method for Achieving Balance in Observational Studies.” Review of Economics and Statistics 95 (3): 932–945.CrossRefGoogle Scholar
Fabozzi, Frank, Focardi, Sergio, Rachev, Svetlozar, and Arshanapalli, Bala. 2014. The Basics of Financial Econometrics: Tools, Concepts, and Asset Management Applications. Hoboken, NJ: Wiley.CrossRefGoogle Scholar
Fenno, Richard. 1978. Home Style: House Members in Their Districts. Boston: Little, Brown, and Company.Google Scholar
Franco, Annie, Malhotra, Neil, and Simonovits, Gabor. 2014. “Publication Bias in the Social Sciences: Unlocking the File Drawer.” Science 345 (6203): 1502–1505.CrossRefGoogle ScholarPubMed
Giles, Michael, Hettinger, Virginia, and Peppers, Todd. 2001. “Picking Federal Judges: A Note on Policy and Partisan Selection Agendas.” Political Research Quarterly 54: 623–641.CrossRefGoogle Scholar
Handler, Samantha. 2022. “Black Women Lawyers Still Sparsely Seen at Federal Circuit.” Bloomberg Law. https://news.bloomberglaw.com/ip-law/black-women-lawyers-still-sparsely-seen-at-federal-circuit.Google Scholar
Hendershot, Marcus. 2010. “From Consent to Advice and Consent: Cyclical Constraints within the District Court Appointment Process.” Political Research Quarterly 63: 328–342.CrossRefGoogle Scholar
Hettinger, Virginia, Lindquist, Stefanie, and Martinek, Wendy. 2006. Judging on a Collegial Court: Influences on Federal Appellate Decision-Making. Charlottesville, VA: University of Virginia Press.Google Scholar
Ho, Daniel, Imai, Kosuke, King, Gary, and Stuart, Elizabeth. 2011. “MatchIt: Nonparametric Preprocessing for Parametric Causal Inference.” r.iq.harvard.edu/docs/matchit/2.4-20/matchit.pdf.CrossRefGoogle Scholar
Iyengar, Shanto. 1987. Is Anyone Responsible? How Television Frames Political Issues. Chicago: University of Chicago Press.Google Scholar
Kahneman, Daniel, and Tversky, Amos. 1984. “Choices, Values, and Frames.” American Psychologist 39: 341–350.CrossRefGoogle Scholar
Kamenar, Paul. 1990. “The Role of the ABA in the Judicial Selection Process.” In Judicial Selection: Merit, Ideology, and Politics, edited by Abraham, Henry, Bell, Griffin, Grassley, Charles, Hickok, Eugene, Kern, John III, Markman, Stephen, and Reynolds, Wm. Bradford, 93–102. Washington, DC: National Legal Center for the Public Interest.Google Scholar
Landes, William, Lessig, Lawrence, and Solimine, Michael. 1998. “Judicial Influence: A Citation Analysis of Federal Courts of Appeals Judges.” The Journal of Legal Studies 27: 271–332.CrossRefGoogle Scholar
Leflar, Robert. 1960. “The Quality of Judges.” Indiana Law Journal 35: 289–305.Google Scholar
Little, Laura. 2001. “The ABA’s Role in Prescreening Federal Judicial Candidates: Are We Ready to Give Up on the Lawyers?” William and Mary Bill of Rights Journal 10: 37–73.Google Scholar
Lott, John. 2005. “The Judicial Confirmation Process: The Difficulty with Being Smart.” Journal of Empirical Legal Studies 2: 407–447.CrossRefGoogle Scholar
Lott, John. 2006. “Pulling Rank.” New York Times, January 25, 2006. http://www.nytimes.com/2006/01/25/opinion/25Lott.htmlGoogle Scholar
Lott, John. 2013. “ABA Ratings: What Do They Really Measure.” Public Choice 156 (1–2): 139–161.CrossRefGoogle Scholar
Martinek, Wendy, Kemper, Mark, and Van Winkle, Steven. 2002. “To Advise and Consent: The Senate and Lower Federal Court Nominations, 1977–1998.” Journal of Politics 64 (2): 337–361.CrossRefGoogle Scholar
Mayhew, David. 2004. Congress: The Electoral Connection. New Haven, CT: Yale University Press.Google Scholar
Merritt, Deborah Jones, and Brudney, James. 2001. “Stalking Secret Law: What Predicts Publication in the United States Courts of Appeals.” Vanderbilt Law Review 54: 71–121.Google Scholar
Mott, Rodney. 1948. “Measurement of Judicial Personnel.” New York University Quarterly Law Review 23: 262–277.Google Scholar
NOLA.com. 2017. “Watch Trump Judicial Pick Stumble over Sen. John Kennedy’s Basic Legal Questions.” YouTube video. December 15, 2017. https://www.youtube.com/watch?v=wd-eYl-LiPs.Google Scholar
Poole, Keith. 1998. “Recovering a Basic Space from a Set of Issue Scales.” American Journal of Political Science 42: 954–993.CrossRefGoogle Scholar
Savchak, Elisha, Hansford, Thomas, Songer, Donald, Manning, Kenneth, and Carp, Robert. 2006. “Taking It to the Next Level: The Elevation of District Court Judges to the U.S. Courts of Appeals.” American Journal of Political Science 50: 478–493.CrossRefGoogle Scholar
Scherer, Nancy, Bartels, Brandon, and Steigerwalt, Amy. 2008. “Sounding the Fire Alarm: The Role of Interest Groups in the Lower Federal Court Confirmation Process.” Journal of Politics 70: 1026–1039.CrossRefGoogle Scholar
Sen, Maya. 2014. “How Judicial Qualification Ratings May Disadvantage Minority and Female Candidates.” Journal of Law and Courts 2 (1): 33–65.CrossRefGoogle Scholar
Sieja, James A. 2015. “Do They Know Them When They See Them? The ABA Standing Committee on the Federal Judiciary and Qualifications for the Federal Bench.” PhD diss., University of Wisconsin-Madison.Google Scholar
Sieja, James A. 2023a. “How You Rate Depends on Who Investigates: Partisan Bias in ABA Ratings of US Courts of Appeals Nominees, 1958–2020.” Political Research Quarterly 76 (4): 1723–1735.CrossRefGoogle Scholar
Sieja, James A. 2023b. “It’s Time for the American Bar Association to Admit Its Bias and Work to Correct It.” NorthJersey.com July 31. https://www.northjersey.com/story/opinion/2023/07/31/time-for-the-aba-to-admit-its-bias-and-work-to-correct-it/70477803007/Google Scholar
Silver, Nate. 2012. The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t. New York: Penguin.Google Scholar
Slotnick, Elliot. 1983. “The ABA Standing Committee on Federal Judiciary: A Contemporary Assessment—Part 1.” Judicature 66 (8): 349–362.Google Scholar
Smelcer, Susan, Steigerwalt, Amy, and Vining, Richard. 2012. “Bias and the Bar: Evaluating the ABA Ratings of Federal Judicial Nominees.” Political Research Quarterly 65 (4): 827–840.CrossRefGoogle Scholar
Steigerwalt, Amy. 2010. Battle over the Bench: Senators, Interest Groups, and Lower Court Confirmations. Charlottesville, VA: University of Virginia Press.Google Scholar
Stratmann, Thomas, and Garner, Jared. 2004. “Judicial Selection: Politics, Biases, and Constituency Demands.” Public Choice 118: 251–270.CrossRefGoogle Scholar
Taha, Ahmed. 2004. “Publish or Paris? Evidence of How Judges Allocate Their Time.” American Law and Economics Review 6: 1–27.CrossRefGoogle Scholar
Tober, Stephen. 2006. “Statement of Stephen L. Tober on Behalf of the Standing Committee on the Federal Judiciary of the American Bar Association Concerning the Nomination of Brett Michael Kavanaugh to Be Judge of the United States Court of Appeals for the District of Columbia Circuit before the Committee on the Judiciary, United States Senate.” May 8, 2006. http://www.americanbar.org/content/dam/aba/migrated/scfedjud/statements/kavanaugh.authcheckdam.pdfGoogle Scholar
Voeten, Eric. 2013. “The ABA Prefers White Men.” The Monkey Cage. February 28, 2013. http://themonkeycage.org/2013/02/28/the-aba-prefers-white-men/.Google Scholar
Whelan, , Ed. 2010. “Re: The ABA and Ninth Circuit Nominee Goodwin Liu.” Bench Memos. February 25, 2010. http://www.nationalreview.com/bench-memos/49265/re-aba-and-ninth-circuit-nominee-goodwin-liu-ed-whelan.Google Scholar
Supplementary material: File

Sieja supplementary material
Download undefined(File)
File 39.6 KB