Among the key constructs of biomedical research (random error [chance], risk, and bias in the search for causation), bias (or systemic error) is the most formidable source of inefficient and wasteful research, leading to incorrect or exaggerated results. The cause of most disease is complex, owed to many inherent (genetic) and environmental risk factors. It is in studying the interplay of these, each incurring modest risk, that many biases come into play.

Much biomedical research is funded by governments. The examples of abortion, gun safety, and some breast cancer research are offered to demonstrate political bias in prioritizing research. Limited evidence for bias during funding review and bias in citation are discussed.

Bias in reporting study outcomes, so as to favor statistically significant results, is a relatively recently documented phenomenon, which conflates hypothesis testing with hypothesis generation. The biased publication of significant results is enhanced by the preference for authors to submit publications with statistically “positive” results, and journal editors to publish them. Citation bias has been shown to occur throughout the scientific process, from grant writing to final manuscripts. Peer review, from grant submission to manuscripts, is also subject to bias, favoring the status of the authors. All of this contributes to the “winners curse,” the propensity for the first studies declaring an association to be wrong.

Tuskegee, the early twentieth-century eugenics movement, and Roma studies, are examples of major research efforts that were biased and misdirected. More recent examples from the study of electro-magnetic field exposure and childhood cancer, as influenced by interviewer bias, are provided.

Here, we discuss more formal approaches for identifying the influence of possible bias on a study design, or in the conduct of a study, what is called quantitative bias assessment. Then we consider the even more difficult problem of managing systemic bias in researchers themselves, and in research teams. Investigators find it particularly difficult to recognize cognitive and psychological bias in themselves and suggestions for facilitating this are made. The importance of creating a “bias calculation” in grant proposals is emphasized. A hierarchical structure to research bias is recognized.

Here we sail into an analysis of what is random error (chance) and what is systemic error (bias), the focus of this present work. Bias produces research whose results are the most destructive; the result appears to be precise (chance does not exert a large effect), but it is wrong. Early descriptions of bias list 35 types but later catalogues describe well over 200. Documenting an association between risk factor and disease is the sine qua non in a causal analysis, and this is where bias most fully operates.

Control and comparison groups are essential for most biomedical research, but who is employed fundamentally influences risk estimation. Historical and hospital control groups are especially susceptible to bias. Natural experiments can prove useful if conducted carefully. Pharmaco-epidemiology has to manage the, not quite, intractable problem of indication bias: Are treatment effects a result of the drug or the disease being treated? Some control groups may be appropriate for detecting and ameliorating bias: respectively, negative and sibling controls.

Early methods of determining the truth simply relied on the size of association. Modern approaches used Koch’s postulates for infectious disease and the Bradford-Hill criteria for chronic disease. In the latter, documentation of an association between risk factor and disease is a “gateway” criterion, but it is in identifying an association that bias can be most disruptive. The most recent additions to searching for truth are derived from paradigms of evidence-based medicine: systematic reviewing and meta-analysis, and most recently a GRADE assessment. Current levels for determining statistical significance are not fit for purpose and a stricter level is proposed.

Participating in research incurs its own risk for bias in research subjects themselves. The classic Milgram studies on obedience to authority are discussed as classic studies of response bias. Methods for unbiased questionaire design, especially for sensitive topics, are reviewed. The iconic Hawthorne studies are considered and the impact of litigation on study subjects is described. Pretest sensitization occurs when preliminary screening tests for study eligibility interact with the study’s primary intervention. Informative presence bias results from research subjects having multiple contacts with the health care system from which they are recruited.

Animal studies demonstrate particular difficulty in conducting bias-free research, investigators rarely synthesize existing research using modern methods, and have the added problem of translating results in one species to another (humans). Recent guidelines are described that attempt to improve the quality of animal research.

Bias in scientists themselves is discussed – is it in our DNA? I argue that it is; it had survival value early in our hominin history but is now a negative force. These cognitive biases must be acknowledged, understood in the context of our professional lives, and mitigated. Bias also operates within research teams, as groupthink, and this too must be managed. Confirmation bias is a dominant force but are we “lumpers or splitters.” How are data grouped or categorized, and how does “framing” influence our response to data, a total of 35 cognitive biases are described. Scientific fraud is rare and an entirely different problem but the impulse for it may share some of the same psychological roots.

The “play of chance” is described and sources of bias in how data are prepared for analysis (grouping and categorization) is a common source of bias. Multiple comparison bias is another frequently observed bias but problems with a companion bias – subgroup analysis – are less well known. Statistical analysis depends on having independent observations, or some method of accounting for dependency, and this too is a commonplace error and source of bias; it especially threatens meta-analyses. Bias is introduced when the wrong statistical unit is analyzed because “clustering” in the data has been ignored, reducing independency.