Scientists’ discoveries have a worldwide impact. Over time, these discoveries alter our understanding of ourselves and our relationships with one another. They provide a basis for innovation—for new technology—for tools and infrastructure that continually change how we live. In times of crisis, science provides both the promise and actuality of substantial improvements to quality of life.
Because science is known to have these powers, people seek scientific research as a means to achieve aspirations and overcome challenges. People often look to science, scientists, and scientific organizations for guidance on how to transform the world we have into the world we want. What is the value of this guidance to others?
Many people ask variants of this question. Consider, for example, questions about how replicability and reproducibility influence science’s public value (National Academies of Sciences, Engineering, and Medicine 2019). In a number of high-profile cases, scientific claims once thought to be generally true did not replicate more broadly (Open Science Collaboration 2015). Similarly, in areas of study in which researchers have the ability to choose one of possibly many statistical frameworks to interpret relationships among variables, we now know that many chose statistical models based on whether they produced “statistically significant” results rather than on the fidelity of the models’ correspondence to a well-defined theory or problem (Gerber and Malhotra Reference Gerber and Malhotra2008; Lupia Reference Lupia2016, chaps. 16–17; Simonsohn, Nelson, and Simmons Reference Simonsohn, Nelson and Simmons2014). As a result of these cases, there are now larger questions about what, if anything, nonreplicable or “p-hacked” research claims actually teach us about the phenomena in question.
In response, open science movements—that is, movements that seek ethical and sustainable ways to increase transparency, data sharing, reproducibility, replication, and many related concepts—have the potential to help researchers in all fields produce scholarship with substantially higher value to other researchers and the public at large. Open science can empower people to interpret research claims with greater accuracy and rigor. When empowered in this way, prospective beneficiaries of research know more about how claims do, or do not, relate to their circumstances. With these potential benefits in mind, entrepreneurs around the world are developing infrastructure and incentive systems that make a more open science technologically possible and incentive compatible (Nosek et al. Reference Nosek, Alter, Banks, Borsboom, Bowman, Breckler and Buck2015).
Some people ask why a more open science is important. The answer is that there are strong ethical reasons to conduct science in ways that are more transparent and more honest than in the past. For example, many potential users of research, particularly nonscientists, are not well positioned to understand elements of research-production processes that affect the correspondence between what researchers claim and what these claims mean for others. As a result, many potential users would not know about researchers’ professional pressures to publish papers with “statistically significant” results—or the strong incentive for them to suppress “null results.”
Some people ask why a more open science is important. The answer is that there are strong ethical reasons to conduct science in ways that are more transparent and more honest than in the past.
The research community is uniquely positioned to recognize these dynamics and then do something about them. In what follows, I argue that the public value of scientific work—and, indeed, the substantial trust that many elements of society place in science—requires that we make science more open.
At the same time, there are important constraints to sharing information. Sometimes, sharing data can put people in great danger. Protecting human subjects and safeguarding the integrity of environmental materials and human institutions are actions with strong ethical components. With these considerations in mind, I contend that when data, code, and contextual materials can be shared in ways that make meaning easier to discern—and that do so without harming people or slowing the progress of science to an unreasonable degree, then making a commitment to open science is an ethical necessity (see also Lupia and Elman Reference Lupia and Elman2014).
The article continues as follows. First, I offer mathematical proofs as an analog for how open science practices convey value to others. Next, I contend that the changing communication technologies pose new challenges for researchers who seek to have their work seen as credible and legitimate. I then explain how researchers can use open science practices to increase the credibility and legitimacy that science needs to maintain its cultural authority in challenging times.
After demonstrating that open science practices can help researchers convey value in credible and legitimate ways, I turn to circumstances of people who rely on our research for knowledge. I contend that the same open science practices can be the difference between helping and harming those who seek our guidance.
USING PROOFS TO EMPOWER OTHERS
My views on open science, like everyone else’s, are grounded in my training. In the spirit of being transparent about my approach to this issue, I describe some of my own “code.”
My training involved a lot of mathematics. Early in my career, I published articles and books that relied heavily on math. In that world, if you want to make the claim, “A causes B,” you are expected to prove it. You can’t stand before a room of similarly trained people and say, “Hey, I found that people who don’t know very much about a referendum can vote as though they knew a lot” and expect the audience to believe you. If you make the claim without a proof, attendees wonder what you are doing there—and then complain about you in the hallway.
I love proofs because they can give readers access to dynamic and unexpected logical sequences. A well-written proof empowers readers to assemble, combine, deconstruct, and construct combinations of ideas and concepts—at their own pace and in their own time. Indeed, readers can take an entire logical journey without the prover being physically present. The proof becomes the reader’s proof—knowledge that they can understand and validate on their own.
My exposure to proofs informs my approach to open science. Greater transparency offers others greater access to the logic, evidence, and inference that underlie scientific claims. This information can help people more accurately interpret these claims. With greater accuracy, people can empower themselves to make better decisions and more effectively improve the quality of life for others.
Indeed, when researchers are sufficiently open about the process that generated their claims, their identities become increasingly irrelevant to the claim’s meaning. When science is sufficiently open, claims become the reader’s claims—knowledge that they can understand and validate on their own.
USING OPEN SCIENCE TO INCREASE CREDIBILITY AND LEGITIMACY
One reason that we need greater openness today than in years past is that the context in which science is evaluated and used continues to evolve. There are new challenges—challenges for which greater openness is a key to generating greater public value. To see this change, let’s first step back in time.
For most of recorded history, few people could credibly claim to have scientific expertise in any subject. The few people whose expertise was so recognized often became accustomed to their claims having cultural authority. Science’s cultural authority was so widespread that many scientists came to believe that their views were naturally entitled to a degree of deference—particularly when compared to nonscientific claims. This was certainly the expectation conveyed to me as a younger scholar.
The world has changed since then. Fast-evolving communication technologies allow many people to present themselves as “experts” on many subjects that scientists once studied in isolation. In these modern communicative environments, scientists cannot assume that their claims will be given deference. Instead, we now have a greater incentive to demonstrate that our claims are credible and legitimate. How do we do this?
Many scientific fields study how to build credibility and legitimacy. I convey a few aggregate lessons here. A claim is credible if it has attributes that make it believable or trustworthy. What are those attributes? Two conditions for credibility that are individually necessary and collectively sufficient are perceived common interest and perceived relative expertise (Lupia and McCubbins Reference Lupia and McCubbins1998). By perceived common interests, I mean that within the domain of the topic being discussed, readers perceive that a researcher has a goal that is consistent with their own reason for reading the paper—in particular, a search for a more accurate understanding of relationships between variables. By perceived relative expertise, I mean that readers must believe that the researcher knows more about critical aspects of the domain in question than they do.
Consider a credibility assessment in the absence of these attributes. For example, if a reader believes that a researcher will substitute an illusory “statistical-significance” claim for a more true “null result” without informing her, she may doubt that she and the researcher share a common interest in her quest for clarity. Similarly, if the reader comes to doubt that the researcher knows how to provide the information that she needs, she may question the value of the researcher’s alleged expertise. As a result, when open science practices enhance readers’ belief that researchers have the incentive and the ability to report what they actually observe, then these practices increase credibility.
As a result, when open science practices enhance readers’ belief that researchers have the incentive and the ability to report what they actually observe, then these practices increase credibility.
A claim is legitimate if it is constructed in accordance with recognized or accepted principles. Legitimacy influences the public value of science because it allows us to convert initial disagreement into cumulative knowledge gains. To see why this is true, suppose that two researchers make different claims but each can understand how the other came to their conclusion. When they have this understanding, they gain constructive contexts for learning from one another and resolving disagreements. When open science practices provide readers greater access to the contexts from which scientific claims originate, then these practices offer a stronger basis for legitimacy.
As knowledge of statistical inference evolves, and as technology makes it easier and less expensive to share data and code, researchers in all fields can give one another new abilities to evaluate one another’s claims. Platforms and incentives for greater openness provide new opportunities for sharing contextual information that supports the credibility and legitimacy of scientific claims. When researchers share enough data and context so that others can see that the results are what anyone would obtain if they committed to the scientific method and executed it with sufficient skill, then stronger bases for credibility and legitimacy emerge.
EXCEPT WHEN IT DOES NOT: OPEN SCIENCE AS AN ETHICAL MATTER
The move toward open science in recent years shows that researchers can be more purposeful in sharing materials that can help readers more accurately understand the context and meaning of the scientific claims. Yet, some scholars ask why they should make this move voluntarily. This section discusses an ethical reason for doing so.
A scientific claim can follow one of two paths. It can be ignored after it is made or it can be a claim that others pay attention to or use. Some researchers are content to be ignored—or they claim that the value of their work is somehow inherent and independent of its usefulness to others. This is a highly entitled and, in my view, indefensible claim for most publicly funded scholars to make, but I will respect their wishes and ignore them in what follows.
In many cases, researchers produce scientific claims because they want others to pay attention to or use the claims. In many of these cases, researchers know much more about the claim’s meaning than does a prospective reader. The researcher knows about her design choices. She knows about choices that she made when choosing which data to collect or evidence to record. She knows about choices she made when categorizing her observations, analyzing her categories, and interpreting her analyses. She may even know that she would not have been able to make her claim if she had made different choices along certain steps in this path.
However, when she publishes a paper, she cannot explain all of this—at least not all at once. In a paper, she puts a few words in her title but has to leave out many other words. She puts a few words in her abstract but has to leave out many other details. She has to decide which parts of her work to put in the text of the paper, which parts to put in footnotes, which parts to put in appendices, and which parts to omit. Of the facts and descriptions that she does include, she must make decisions about the order in which to present them.
Every researcher makes these decisions every time they present their work. Because many audiences for scientific information have only so much time and energy to devote to a paper or a presentation, how a researcher presents this information can have a significant impact on the audience’s beliefs.
When I was younger, I did not think about these things—at least not in the way that I do now. In the academic career and rewards ecosystem that existed at that time, we all “knew” that we might be able to publish statistically significant results if we had a strong theory, a strong empirical design, and got “the right” reviewers. We also knew that submitting a paper with null results was a “death sentence” for its publication prospects—and definitely not a way to secure a job or subsequent career. We were told that this incentive system, and the publication bias that it generated, was a normal part of science. So, we knew that statistically significant findings were helpful to and perhaps necessary for professional survival. We did not question the incentives—and we never discussed the downstream consequences for the people who might read our claims.
That was a mistake.
Many of our readers did not know that we had a strong incentive for substituting illusory “statistical-significance” claims for more accurate “null results.” As a result, the mass production of significance claims that ensued in many fields of research misled readers who wanted our guidance.
One of the most riveting books on this point is Rigor Mortis by Richard Harris (Reference Harris2017). He describes how mass production of false positives (resulting from publication bias, p-hacking, and related practices) affected medical literatures. The main effect is not simply that single claims in individual research papers are misleading; it also is the production of unreliable scientific consensuses.
Imagine, for example, that researchers conduct 1,000 studies on a topic—and that only 75 produce a positive result. If publication bias and p-hacking are sufficiently powerful, it may be that only the 75 positive findings are published. Harris (Reference Harris2017) documents these dynamics in cancer-related research. The most devastating consequences of this practice, however, do not occur on the pages of academic journals. Instead, they occur in hospital rooms all over the world.
When only positive results are published and readers believe the resulting “consensus,” people then seek to convert the consensus into practice. Thousands of people participate in clinical trials. Millions of dollars are spent.
Then, the trials do not work as the consensus proposed. When the desire for progress is sufficiently great, there is a tendency to question the fidelity of the unsuccessful trials rather than the veracity of the published literature. Believing that the new trials are the problem, more dollars are spent. More patients enter trials. More time is wasted. More people die. The significant human costs of limited openness are revealed.
Researchers in the social sciences may rue this outcome but doubt that the analogy applies to us. In response to this reaction, I ask that we think about whether we can make this ethical challenge disappear so quickly. If we contend that social science does not really affect people’s lives in a serious way, then it is reasonable for others to question why governments, universities, and others should pay for it.
I happen to believe that social science has transformative effects on quality of life for people across the world. It has helped to raise people out of poverty. It has helped to rescue people from violence. It has helped institutions of all types to provide critical services to vulnerable populations. It has helped us to understand prejudice and how to limit its worst manifestations. It has done this and so much more (National Academies of Sciences, Engineering, and Medicine 2017).
Except when it does not.
If the rationale for public support of any type of research is its potential to improve the quality of life for others, then it follows that incentive systems that create misleading findings are more than a methodological annoyance. They are an ethical problem.
There are real people at the end of our research workflows. There are real people who come to our research seeking ways to better reconcile their actions and their aspirations. Contemplating end users’ plight when making decisions about how to conduct research can be the difference between pursuing science as way of unintentionally doing bad things to other people and pursuing science as an intentional way of helping people transform the world they have into the world they want.
Contemplating end users’ plight when making decisions about how to conduct research can be the difference between pursuing science as way of unintentionally doing bad things to other people and pursuing science as an intentional way of helping people transform the world they have into the world they want.
Although increasing numbers of people are taking on critical leadership responsibilities and building essential infrastructure and incentives (see, e.g., Elman, Kapiszewski, and Lupia Reference Elman, Kapiszewski and Lupia2018 and Wuttke Reference Wuttke2018 for descriptions), we as researchers cannot count on them to change how we affect others. Most people who seek our guidance are not endowed with extensive information about how we derive our scientific findings. If we want to tell ourselves that we are pursuing science to serve others, then we must serve them well—and show proper respect for them by sharing as much of that information as we can. What happens next is up to us.
Open access
