Some of the practices that are believed to enhance the quality of science may produce bias. Studies with unexciting results may never be published, or results are selectively reported to highlight positive outcomes. Investigators often measure multiple outcomes while only reporting those with statistically significant findings. The best remedy for this problem is to require prospective declaration of study plans through study registration, such as the primary and secondary outcome variables and data analysis plans. Failure to report results of completed studies remains a serious problem. Further, results from many studies remain unpublished and the probability of publication is higher for positive results, leading to overestimates of treatment benefit. It is possible that some encouraging clinical trial findings are actually false positive results. For US Food and Drug Administration evaluations, data from a significant portion of relevant completed trials remain undisclosed at the time the pharmaceutical products are under evaluation.

Clinical research is expensive: In 2024, the US National Institutes of Health will spend about $49 billion on research projects. Requesting sufficient resources to conduct a high-quality investigation must be balanced against a desire to use public funds prudently. Most studies are underbudgeted. In addition to funds for study personnel and the costs of evaluation and treatment, there may be costs associated with regulatory and scientific oversight, such as a research ethics committee, community advisory boards, information technology, study registration, and funds for study dissemination. Clinical research is a heterogeneous enterprise that usually requires personnel with a range of complementary expertise. This chapter offers guidance on constructing realistic budgets. In addition, we address the complicated issue of paying study participants, which raises important ethical issues. It is important to compensate participants for their time and discomfort. We review models on which to base participant compensation.

This book is about the science and ethics of clinical research and healthcare. We provide an overview of each chapter in its three sections. The first section reviews foundational knowledge about clinical research. The second section provides background and critique on key components and issues in clinical research, ranging from how research questions are formulated, to how to find and synthesize the research that is produced. The third section comprises four case studies of widely used evaluations and treatments. These case examples are exercises in critical thinking, applying the questions and methods outlined in other sections of the book. Each chapter suggests strategies to help clinical research be more useful for clinicians and more relevant for patients.

In the middle of the last century, Archie Cochrane, one of the founding fathers of evidence-based medicine, argued that understanding healthcare treatments required the consideration of three questions: “Can it work?”, “Does it work?” and “Is it worth it?” Each of these questions addresses a different aspect of the problem and requires different assumptions and different research methodologies. Understanding if a treatment can work establishes proof of principle derived from efficacy studies that control who takes the treatment, how it is administered, and how outcomes are measured. The question “Does it work?” is about effectiveness that is evaluated under conditions of the usual care. Randomized controlled trials, which form the core of efficacy research, are difficult to employ in the evaluation of effectiveness. Even if interventions are shown to be efficacious and effective, people need to decide if accepting the treatment is worth it. Healthcare can be expensive, inconvenient, painful, and sometimes of little value. This introductory chapter reviews the three questions and prepares the reader for the in-depth discussion of these issues in the following 16 chapters.

In order to examine our three questions, we need objective research methods. Estimating whether a treatment can work, does work, and has value requires a wide range of research strategies. Evidence establishing that a treatment works under controlled conditions does not necessarily assure benefits when the intervention is applied in clinical practice. This chapter considers the development of a research protocol, and biases that might be attributable to participant recruitment, enrollment, retention, and dissemination of findings. In practice, establishing the value of a treatment should consider an examination of the existing literature, development of thorough research plans, recognition of the strengths and weaknesses of the chosen research methods, and integration of study results within a wider body of knowledge. We challenge beliefs in a hierarchy of methods that assumes some methods, such as the RCT, are free from bias.

Antidepressant medications are widely prescribed for depression and other uses. They are considered a first-line treatment for major depressive disorder. We examine the lack of support for the mechanistic idea that neurotransmitters affect and are affected by these medications. Few people experience significant benefit from their use when compared with the effects of placebos. We consider several ethical issues associated with antidepressants, including conflicts of interest among the committees recommending their use, and examine a study that suffered from spin and other issues of integrity. The chapter examines potential alternatives to antidepressant medications for those with depression.

We use healthcare in an effort to live longer or feel better. Yet many evaluations do not consider these outcomes, which are of high importance to patients. Instead, they concentrate on variables that are considered surrogates for what treatment is attempting to achieve. Prevention of heart disease, for example, might be estimated from changes in LDL cholesterol levels. These surrogate markers are often poorly correlated with the outcomes of most importance to patients. Understanding the basic biological mechanisms is valuable, but sometimes irrelevant. The chapter reviews patient-reported outcomes that are becoming more commonly used to evaluate health care. These measures are used to create indexes that combine how long people live with the quality of life during the years that precede death. The measures are generic and can be used to compare the value of investing in interventions that have different specific objectives. Cost-effectiveness analysis can directly compare health gain associated with treatments as different as exercise training versus organ transplantation. The public policy implications associated with these metrics are discussed.

Gastroesophageal reflux disease is a common condition that can be controlled with proton pump inhibitors such as omeprazole. We examine randomized controlled trials (RCTs) of omeprazole and find stronger evidence of efficacy among RCTs with industry support than without. The participants in these trials were unlike most people who take proton pump inhibitors, raising questions about the external validity of RCTs. Furthermore, use of these medicines is associated with short- and longer-term adverse effects. Healthy behavior change, such as weight loss, holds promise as an alternative to proton pump inhibitors.

The criteria for evaluating research studies often include large sample size. It is assumed that studies with large sample sizes are more meaningful than those that include a fewer number of participants. This chapter explores biases associated with the traditional application of null hypothesis testing. Statisticians now challenge the idea that retention of the null hypothesis signifies that a treatment is not effective. A finding associated with an exact probability value of p = 0.049 is not meaningfully different from one in which p = 0.051. Yet the interpretation of these two studies can be dramatically different, including the likelihood of publication. Large studies are not necessarily more accurate or less biased. In fact, biases in sampling strategy are amplified in studies with large sample sizes. These problems are of increasing concern in the era of big data and the analysis of electronic health records. Studies that are overpowered (because of very large sample sizes) are capable of identifying statistically significant differences that are of no clinical importance.

Ethical decisions must be made at every phase of a research study. Codes of ethics provide guidance on behaviors that are permissible or nonpermissible for research investigators. In contemporary science, investigators are required to have regular training on the responsible research conduct relevant to studies involving human subjects and animals. Despite this training, ethical lapses occur. This chapter explores some of the basic issues, including ethical mandates on what should be done, what must be done, and what must not be done. We consider the history of serious ethical concerns, such as the Tuskegee experiment. The chapter also reviews historical milestones such as the Belmont report, the Declaration of Helsinki, and the establishment of the Common Rule that is applied for research funded by US federal agencies. Further, the chapter explores challenges relevant to the reporting of conflicts of interests, imperfections in institutional review boards (IRBs), and ethical challenges in studies that use placebos. Among a range of research methods, randomized controlled trials tend to encounter the greatest number of ethical concerns.

Screening for breast cancer using mammography is one of the most common medical tests for women aged 50 and older. In the United States, many protocols initiate mammography at ages 40 or 45. Although cancer screening tests are widely advocated, some systematic reviews find little evidence supporting the most common screening tests. Cancer screening clearly identifies lesions at an earlier stage. Yet, when evaluated against cancer-specific or all-cause mortality, screening is less likely to be associated with longer life of higher quality of life. This chapter reviews a series of biases, including lead time bias and length bias, that may explain the discrepancy between enthusiasm for cancer screening and clinical trials that have consistently failed to show benefit. We also review potential harms of screening, such as false positive results, unnecessary biopsies, and anxiety. We conclude that more studies are needed, particularly investigations that include a heterogeneous mix of studies participants.

After reviewing a wide range of topics, we conclude that good science requires greater efforts to manage biases and to promote the ethical conduct of research. An important problem is the belief that randomized controlled trials (RCTs) are exempt from systematic bias. Throughout the book, we acknowledge the importance of RCTs, but also emphasize that they are not immune from systematic bias. A second lesson concerns conflict of interest, which must always be taken seriously. Most large RCTs are sponsored by for-profit pharmaceutical companies. We identify leverage points to address these problems. These include cultivating equipoise – the position that research investigators enter a study with the understanding that either a positive, negative, or null result is of value. We return to several other themes prominent throughout this book, including the reporting of research findings and serious problems with our system of peer review. The book concludes with recommendations for reducing conflicts of interest, improving transparency, and reimagining the peer review system.

Scientific knowledge is abundant, but this abundance has created challenges. What can be synthesized from the research is limited because of the inconsistent use of terms and classification systems. For example in clinical research, literature reviews, such as meta-analyses, are critical in the development of clinical practice guidelines and recommendations. And the problem is especially acute in the behavioral sciences, where the lack of an agreed-upon classification system for research terms means this knowledge is less likely to be synthesized and interpreted in a manner that can affect clinical care and public policies. This chapter examines the gap between what is known and the capacity to act on that knowledge. We discuss strategies to make research more replicable, better organized, and more easily retrieved.

Research is about asking and answering questions. One of the most important investments of time for a research investigator should occur before the study starts. This chapter considers the importance of well-defined research questions that have clear boundaries and scope. The specifics of the research methodologies such as sample size and data analysis are essential for high-quality research. Yet less emphasis is placed on the importance of the research question, the feasibility of the study, and the social impact of the investigation. This chapter argues that clinical research should be person- and community-centered. The population, intervention, comparator, outcome, and timeframe (PICOT) framework encompasses content that may be informative for those who use health care. The feasible, interesting, novel, ethical, and relevant (FINER) framework comes closer to focusing on questions and outcomes of importance to study participants. We offer a BASES (biases, awareness, social, equilibrium, specificity) model that builds on the FINER and PICOT systems to place greater emphasis on social context.

HMG-CoA reductase inhibitors, also known as statin medications, are used to reduce cholesterol levels in efforts to prevent heart attacks and strokes. Extensive evidence justifies the use of statins. As an exercise, we take a skeptical look at the evidence and raise concerns about the consistency, patient-centeredness, and potency of benefit. Much of the justification for statins focuses on LDL cholesterol as a surrogate for heart disease. Only one major clinical trial has demonstrated that statins (versus placebos) result in longer life expectancy. Subject populations evaluated in statin trials tend to be highly selected. Older adults, a group that almost universally uses the medications, have been studied only rarely. Assuming that lower LDL levels reflect better health, a recent campaign promotes lowering LDL cholesterol values to below 50 mg/dl. The campaign is based on the assumption that the relationship between LDL cholesterol and mortality is linear. Inspection of the data reveals that the relationship is log linear; there is more benefit for initiating treatment among people who are initially at high LDL levels in comparison with those who are initially at lower risk.

In hierarchies of research evidence, the randomized controlled trial (RCT) usually appears near the top of the pyramid. RCTs are usually considered to be free of bias; systematic reviews of the literature may exclude studies that are not RCTs. Although controlled trials are excellent methods for establishing causation, they do not assure freedom from systematic bias. This chapter explores biases that are common among RCTs. In particular, we report on the practice of systematically excluding study participants who do not meet specific criteria. It is not uncommon for 90 percent of potential volunteers to be turned away. Common grounds for exclusion include comorbidities, even though living with multiple chronic conditions is almost universal among older adults. The selection of the control group can also increase bias. Control groups might be selected specifically because they increase the likely difference between treated and control conditions. The implications of biases in our RCTs are discussed.