Hostname: page-component-8448b6f56d-c47g7 Total loading time: 0 Render date: 2024-04-23T12:56:12.662Z Has data issue: false hasContentIssue false

Learning to Regulate Learning Healthcare Systems

Published online by Cambridge University Press:  22 May 2019

Rights & Permissions [Opens in a new window]

Abstract:

It is commonplace to observe that science often outstrips the ability of society to monitor, supervise and regulate it. A recent challenge in this regard concerns Learning Healthcare Systems, an initiative to collect data and test hypotheses across clinical settings, and therefore to a larger degree than before. Some argue that the line between research and clinical practice is becoming blurred, and that existing regulation seems to obstruct low risk research. They propose the creation of a new ethical framework for Learning Healthcare Systems, to speed up research. This paper opposes that view, and argues that Learning Healthcare Systems do not blur or remove the line between research and practice, and will not result in a single set of regulations for all kinds of biomedical research. The authors suggest that a large proportion of Learning Healthcare System activities resemble public health surveillance to a significant extent. Therefore, they propose that these activities should be organized in accordance with the same ethical principles as public health surveillance, specifically that Learning Healthcare Systems should rest on the principles of comprehensiveness, transparency, and public accountability.

Type
Departments and Columns
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
Copyright © Cambridge University Press 2019

1. Background

Healthcare practice requires progress driven by evidence. Randomized clinical trials (RCTs) are often considered the gold standard for gathering evidence in medicine. Unfortunately, RCTs have their own limitations and sometimes even statistically-significant results lack clinical significance.Footnote 1 Strict inclusion and exclusion criteria limit access to RCTs for patients with chronic diseases, comorbidities, the elderly, women and children.Footnote 2 As a consequence, one can observe variations and disparities in care and a lack, inconclusiveness or poor quality of evidence for clinical practice.Footnote 3 The Learning Healthcare System (LHS) is offered as an answer to these problems. The Institute of Medicine defines an LHS as a system in which “science, informatics, incentives, and culture are aligned for continuous improvement and innovation, with best practices seamlessly embedded in the delivery process and new knowledge captured as an integral by-product of the delivery experience.”Footnote 4 As indicated by Charles P. Friedman, the essence of the LHS is an information loop in which data are assembled, analyzed, interpreted and then come back to a system in form of an innovative practice.Footnote 5 The LHS uses medical records and computing power to test hypotheses and find causative factors of health and disease. In their report, Thomas Foley and Fergus Fairmichael differentiate five areas where the idea of the LHS could be applied: (1) comparative effectiveness studies, (2) so-called “positive deviants” analysis, (3) real-time surveillance, (4) predictive models, and (5) clinical decision support systems.Footnote 6 In this paper, we will refer chiefly to the third kind of application of the LHS, namely, real-time surveillance, but our argument could be applied to all five areas.

The existing ethical and regulatory framework is not adjusted to the LHS. In many developed countries, researchers encounter problems with launching and carrying out research projects.Footnote 7 Some researchers and bioethicists share a belief that the existing regulations overprotect research subjects, and by doing so, deprive them of better healthcare.Footnote 8 For instance, an ethics review can significantly impede multicenter research projects and generate immense costs.Footnote 9 There are discrepancies in the procedures of ethical review and interpretations of existing regulations.Footnote 10 Also, obtaining informed consent in large-scale projects is sometimes infeasible. In other cases, the pursuit of informed consent can skew the research sample, because even if a research project poses very low risk, significantly lower numbers of people respond to a call for participation, and a recruited group does not result in a representative sample.Footnote 11

The LHS is sometimes considered to be a disruptive factor in medicine. Some authors argue that the LHS and the development of medicine have changed the very nature of both medical practice and research involving human subjects, blurring the difference between treating patients and conducting research.Footnote 12 Therefore, goes the argument, there should not be a distinction between practice and research but rather, the new category of a learning activity (LA) needs to be adopted. This change in nature of science should be followed by regulations and ethical guidelines which, according to this view, should become more permissive to facilitate research projects.

It seems plausible to adjust existing regulations and ethical guidelines in a way that facilitates low-risk safety and quality studies. However, the LHS—which is a part of a wider process of digitalization in medicineFootnote 13—raises myriad ethical questions. For instance, should a patient give her informed consent for a particular kind of research, or she should only be given an opt-out option? Who can gather, retain and process the health data of patients: healthcare facilities, doctors, insurers, or researchers? Should access to health data be limited (and if so, how)? What is, and should be, the role of the government in creating the LHS? Below, we address some of these questions and want to underline that we do not refer to any particular healthcare system, country, or legal system. Instead, we aim at formulating general principles and guidelines that could be applicable to all countries affirming human rights, the rule of law, and democracy.

In the following sections, we show that the LHS does not abolish the difference between treatment and research, but rather adopts a new category of an LA. Further we argue that some learning activities have similar characteristics to public health surveillance (PHS). We propose that these LAs and PHS should be similarly regulated, and should not require previous ethical review and informed consent. Rather, it should be conducted by a publicly-controlled and supervised agent in accordance with national legal regulations in a transparent and socially-accountable manner.

2. LHS and Public Health Surveillance

2.1. Dissolving a Treatment- Research Distinction?

There are five characteristics usually ascribed to research to distinguish it from practice: (i) research is mainly designed to develop generalizable knowledge; (ii) research requires systematic investigation; (iii) research is thought to present less net benefit and pose more risk and burden than practice; (iv) in the research context a, subject is exposed to more risk and burden than in the clinical setting; and finally, (v) in practice, a patient is treated in accordance with the personal care standard, meaning that a physician adjusts the dosage to the individual needs of a patient and can carefully monitor the effects of the treatment. In contrast, a researcher is strictly limited by a research protocol, as a dosage is fixed and often no adjunctive treatment is permitted.Footnote 14

Nancy Kass, et al.Footnote 15 ague that progress in medicine, and especially the development of the LHS, have led us to a point where medical practice and biomedical research pose similar risks to patients. According to these authors, there is no longer a significant conceptual and moral difference between practice and research. Instead, the new category of a learning activity (LA) should be adopted. An LA dissolves the distinction between practice and research because it can, by definition, be curing, diagnosing or preventing, and at the same time, be a part of systematic collection of scientific evidence and a tool for testing scientific hypotheses.

According to Kass and her colleagues, these five characteristics can be ascribed to medical practice within the framework of the LHS. Medical records from the healthcare computer system are collected and analyzed in a systematic manner to produce generalizable knowledge that is then promptly applied in practice. Research does not always present less net benefit, and more risk and burden, than practice. For instance, in pediatric anticancer trials research can be considered, at least in some instances,Footnote 16 the best available treatment.Footnote 17 Moreover, when it is the case that an existing treatment is unproven, but still standard care, it is possible that during therapy a patient is exposed to unnecessary risk and burden. Finally, as well, the standard of personal care is limited to the framework of existing healthcare systems, due to the fact that physicians are constrained in their professional judgment by reimbursement policies and the patient’s ability to pay. Moreover, new research protocols allow adjustments for experimental intervention, according to patient’s needs.

Given the above, we perhaps do not need two different ethical frameworks, but only one set of ethical guidelines that should be applied to research protocols and medical practice.Footnote 18 However, it is our position that this proposal does not provide a satisfactory solution.

We argue that different activities require different procedural safeguards, although these procedural safeguards can be justified using the same set of ethical principles. For example, we may have one uniform ethical framework, such as the one proposed by Ruth Faden, et al.,Footnote 19 but this ethical framework could be translated into different procedural requirements in different circumstances. However, in order to adapt a set of ethical principles to certain circumstances we need a rich conceptual framework that will allow us to differentiate and discriminate among varieties of activities. In this context, the difference between research and practice appears to be important.

Let us consider three different learning activities: an RCT, an observational study, and medical practice. All three can be considered LAs, but are significantly different. RCTs often carry more risk than observational studies. A double-blind, placebo-controlled clinical trial assessing a new agent for general anxiety disorder, when a proven treatment exists, poses more net risk to a participant than would an observational research study. But even an observational study can involve additional diagnostic procedures, for instance a set of intrusive questionnaires, and multiple blood tests that can interfere with the routine therapy of a patient diagnosed with general anxiety. Participation in an observational research project requiring additional diagnostic procedures may not be in the patient’s best interests, although a study’s activities might be arranged in a way that make them difficult to distinguish from a routine diagnostic process. In routine medical practice, a physician may discuss with a patient different options of pharmacotherapy, including pharmacotherapy coupled with psychotherapy, or even off-label pharmacotherapy. All these options could be associated with different risk levels, but in the process of shared decision making, a patient and a psychiatrist can arrive at the best possible option from the patient’s perspective.Footnote 20

In all three cases, different ethical problems can arise and researchers and physicians may try to resolve them by referring to the same ethical framework. A researcher may want to avoid imposing nonclinical risk and burdens, and would try to limit them to a necessary list.Footnote 21 The same principle would be followed by the physician. But it seems clear that because of the very nature of these activities, all three would require different procedural safeguards. In the first case, informed consent is required, and a full ethics review is needed to ensure that the study does not pose excessive risk to research subjects. In the case of an observational study, at least abbreviated review would be necessary, followed by obtaining the patient’s informed consent. In the case of routine practice, an ethics review may be unnecessary, and the process of informed consent has different goals. Therefore, although ethical problems concerning all activities can be addressed within one ethical framework for an LHS, we still need different procedural and regulatory approaches: one for research, and one for medical practice. Moreover, the claim that the very nature of these activities is the same—namely, learning—may impede properly differentiating them and adjusting corresponding procedural safeguards.

An argument could be made that off-label use is a special case that does not fall into either the research or practice category. If there is no evidence that an off-label drug is efficient and safe, then it does not meet one of the defining features of practice, in that it does not guarantee a reasonable expectation of success.Footnote 22 In that instance, a broader category of learning activity would be useful. But this argument seems to miss the point. First, not all medical activities have to be either research, or practice. Second, off-label use is not always without evidence. It seems that off-label use might fall into at least three categories: supported by evidence, supposedly safe as well as effective, and investigational.Footnote 23 Only a suppositional off-label drug use, when only weak evidence of safety and efficacy exists, could pose a problem. Moreover, a new category of LA is not especially helpful in this case. We have only to decide what kind of standards should be applied to this type of off-label use: those of practice, or those of research, or perhaps a third category with specially-adjusted regulations. By discriminating between different research activities, we can allow, for instance, a waiver of informed consent requirement or abbreviation of ethics review. This differentiation is done case-by-case, because even if two studies fall into the same category, for example, pragmatic clinical trial, they can pose different risk and benefit ratios to their participants.Footnote 24

The category of LA does not dissolve the practice-research distinction, because it does not resolve inherent tensions within a research setting. A researcher can sometimes face a conflict between the best interests of her patient and a willingness to develop new therapies.Footnote 25 This kind of conflict arises from the divergent goals of treating and conducting research. There is a relation between a risk for a participant and the moral significance of the tensions between divergent goals of treating and research. As a result, special safeguards are needed to manage this tension when it increases. Within the LHS, a new category of low-risk observational studies is possible. This is brought by technological advances—the ability to collect and analyze large amounts of data. This technological transformation does not imply a major change in methodology of RCTs or other studies. LHS research studies are in fact observational research studies, and that makes them similar to epidemiological and public health studies. LHS makes observational studies easier due to the application of computers and appropriate software. If one considers an example of safety and quality improvement studies within the LHS, it becomes clear that they have the same characteristics and similar risk-benefit ratio as public health surveillance. Therefore, in our view, this is not the distinction between practice and research that is dissolved within LHS, but rather a distinction between some LAs and public health surveillance (PHS) activities.

2.2. Public Health Practice, Research and Learning Healthcare

The term “public health” can be understood in several ways. However, it seems that when we limit the meaning of “public health” to the narrow definition proposed by Mark A. Rothstein, the transformation of healthcare systems into learning healthcare systems could be understood analogically to be public health interventions.Footnote 26 According to Rothstein, public health intervention involves public officials with specific legal authority that balances private and public interests, aims to protect public health, and sometimes uses coercive measures. Marcel Verweij and Angus Dawson modify the definition of public health and propose that a public health intervention can be undertaken by nongovernmental bodies, but that it has to be an action involving the entire population that though engaging individuals, is not necessarily beneficial for everyone.Footnote 27 In that sense, the LHS becomes a part of the public health surveillance system: within the LHS, data are collected and analyzed to protect public health or enable early intervention. Lawrence O. Gostin and Lesley Stone seek to delineate public health law and claim that public health law embraces five elements: government, population, service, power to enforce certain behavior, and limitation of this power.Footnote 28 The power of the government can be used to directly regulate the behavior of persons and professionals. Taking these characteristics into account, we can attempt to build a definition of public health intervention, as: an activity mandated by law or a government agency embedded in the legal system, that aims to protect and promote public health and prevent disease. This activity might be mandatory, and is not always in the best interests of an individual.

Public health surveillance (PHS) is defined as “the systematic, ongoing collection, management, analysis, and interpretation of data followed by the dissemination of these data to public health programs to stimulate public health action.”Footnote 29 Because of its systematic character that leads to generalization, there is an ongoing debate as to whether PHS should be understood as research or as practice.Footnote 30 As Amy Fairchild concludes, public health surveillance is “simultaneously both research and practice.”Footnote 31 That fits the definition of an LA. Moreover, the definition of PHS given by Stephen B. Thacker and Ruth Berkelman is similar to the core characteristic of the LHS, namely, a bidirectional feedback loop, as surveillance comprises not only gathering data, but also implantation of certain polices. Therefore, it seems that rather than focusing on the blurred line between research and practice, we should be paying attention to similarities between the LHS and PHS. From this essential and descriptive recognition, we can draw a normative conclusion: to the extent that the LHS is similar to PHS, it should be governed by similar ethical principles, and similarly regulated.

3. How to Regulate Learning Activities?

3.1. Public Health and Research

Research ethics and research regulations significantly differ from public health ethics and public health laws. The main goal of research ethics is to protect individual interests, weighing them against social benefit. Public health has an opposite goal: to protect the community and weigh public good against individual freedoms.Footnote 32 Research requires informed consent and ethics review. Public health activities are mandatory and are usually launched without prior ethics review. It seems puzzling that the two sometimes very similar activities are regulated and guided by such different ethical principles and legal rules.

Research was strictly regulated because it was rightly thought that a substantial number of research projects pose risks to participants and deprive them of the benefits of personalized care. Moreover, it was recognized that the development of medicine is only one of many different goods and there is no moral obligation to participate in medical research.Footnote 33 The requirements of informed consent and an ethics review are then justified by the characteristics of some research projects. Public health practice is regulated in a different way, because the risks and benefits for the society and individuals are differently distributed in the context of the health protection and promotion, as well as disease prevention. Systematic gathering of health data is necessary to detect health risk factors and prevent disease from spreading. Protection of the common good is a governmental responsibility, and participation in public health activities could be considered a moral obligation. Evading participation either poses a risk or undermines a protection system, and in both cases, endangers the common good. Moreover, PHS does not pose significant risks to the subjects, and is conducted by state institutions that execute their duties as imposed by legislation. Therefore, PHS has not only legal but also moral aspects. A state’s legal actions are not just expressions of arbitrary will, but are an embodiment of the common will and political agreement between the citizens (in the case of a democratic state that respects human rights and rule of law) and as such, they have moral significance.

3.2. The Learning Healthcare System and Public Health Surveillance

Some activities in the LHS have the same characteristics as PHS. The list of such activates consists of databases tracking the safety of new drugs, safety-monitoring of innovative medical practices, databases of clinical research, and comparative effectiveness studies using medical records. The goal of these activities is exactly the same as PHS: to protect public health and prevent harm. Additionally, in the case of PHS, the risks for the patients are almost negligible, and primarily consist of infringement of privacy. The fact that all these activities are conducted by state agencies, or in close cooperation with state agencies, also provides these activities with public legitimization.

3.3. What Makes Public Health Surveillance and the Learning Healthcare System Ethical?

The main and persistent problem of public health ethics is that we do not know where to draw a line between individual freedom and public good. This basic dilemma can be described in different ethical terms, but we have not, as yet, arrived at a commonly accepted solution.Footnote 34 In liberal democracies, public opinion requires moral justification of any limitation of personal freedom.Footnote 35 We propose that a LA which meets three conditions should not be the subject of an ethics review and should not require informed consent. These conditions are the following:

A. A comprehensive healthcare system condition.

Research has to target the whole healthcare system (its safety, efficacy, efficiency, or all patients suffering a certain condition). Public health is a common good, and the protection of public health may require reasonable limitations of individual liberty. Research using medical records to determine the safety, efficacy, or prevalence of a certain condition falls into the category of reasonable limitation of individual liberty. An individual is not exposed to any additional harm. The possibility of infringement of privacy in a properly-working system seems less than in the case of standard healthcare and administrative procedures. It is also hard to imagine that research involving consented-to treatment, its efficacy, safety, and efficiency, could be wrongful to a participant. The condition of comprehensiveness also implies that research does not discriminate, or target any specific population or community.

B. Research that is publicly transparent and open to public scrutiny.

In the LHS, two levels of transparency are possible: (1) patients’ representatives can be directly involved in oversight of all research conducted in a healthcare system. This information about research can be open to public scrutiny; and (2) the LHS can enable individual patients to see and control the research studies in which they are involved. Within the LHS, it is possible to create patients’ interfaces.Footnote 36 A patient can thus have access to her electronic health record (EHR), follow which projects her data was used in, and find sufficient information about these research projects.

C. Conducted by state health system or institution in collaboration with state.

State and governmental institutions have public accountability, as well as strictly-defined goals and procedures. They therefore have legal and moral legitimacy to act. Also, to the extent that the goals of these institutions are in accordance with common interests, there is low risk of conflict of interests. Of course, there still is a risk of corruption or exceeding the legal authority.

We can briefly mention some examples of research that fall out of the scope of what is clearly ethically justified in the LHS. Research targeting a specific population. Some research targeting health problems of only one specific population or group should be subject to an ethics review. This kind of research is not unethical in principle, but may require additional ethical safeguards: an ethics review and informed consent. Research within the LHS that does not meet the condition of comprehensiveness can also pose a risk of group stigmatization and discrimination. By the same token, research conducted by an external sponsor, though not in principle unethical, requires at least an ethics review. The ethics review and IRB/REC (Institutional Research Board/Research Ethics Committee) body is built into the democratic system, and plays an important role of public control over the research enterprise.Footnote 37

Conclusion

It has become clear that the transformation of healthcare into a learning system does not dissolve an ethically-significant distinction between research and practice. Moreover, one ethical framework for all LAs does not exclude different regulatory requirements for different types of research. Therefore, as argued, RCTs, which pose more risk to participants should be subject to an ethics review in which participants have given their informed consent, whereas other kinds of study do not necessarily require such strict regulations. We demonstrated that some LAs in fact blur the difference between research and public health, and therefore should be regulated in a similar way. Finally, we formulated three conditions of ethically-justified learning activities in health: a comprehensive healthcare condition, a public transparency condition, and the requirement of collaboration with a state health agency.

Footnotes

Acknowledgements: This project was funded by the National Science Centre, Poland, 2015/19/D/HS1/00991. We thank Phyllis Zych Budka for linguistic edits.

This section features original work on ethical, legal, policy, and social aspects of the use of computing and information technology in health, biomedical research, and the health professions. For submissions, contact Kenneth Goodman at: kgoodman@med.miami.edu

References

Notes

1. Pincus, T, Sokka, T. Evidence-based practice and practice-based evidence. Nature Clinical Practice Rheumatolology 2006;2(3):114–5.CrossRefGoogle ScholarPubMed Foley, T, Fairmichael, D. The Potential of Learning Healthcare Systems, Newcastle: Learning Healthcare Project; 2015.Google Scholar

2. See note 1, Foley, Fairmichael 2015; Fleurence, R, Selby, JV, Odom-Walker, K, Hunt, G, Meltzer, D, Slutsky, JR, et al. How the Patient-Centered Outcomes Research Institute is engaging patients and others in shaping its research agenda. Health Affairs (Millwood) 2013;32(2):393400.CrossRefGoogle ScholarPubMed

3. Abernethy, AP, Etheredge, LM, Ganz, PA, Wallace, P, German, RR, Neti, C, et al. Rapid-learning system for cancer care. Journal of Clinical Oncology 2010;28(27):4268–74.CrossRefGoogle ScholarPubMed

4. Institute of Medicine. Integrating Research and Practice. Health System Leaders Working Toward High Value Care. Workshop Summary; Washington D.C., The National Academies Press; 2015, at xi.

5. Friedman CP, Macy J. Toward Complete & Sustainable Learning Systems; 2014; avaiable at: https://medicine.umich.edu/sites/default/files/2014_12_08-Friedman-IOM%20LHS.pdf (last accessed 16 Aug 2018); Morain, SR, Kass, NE, Grossmann, C. What allows a health care system to become a learning health care system: Results from interviews with health system leaders. Learning Health Systems. 2017;1(1):e10015– doi: 10.1002/lrh2.10015 .CrossRefGoogle Scholar

6. See note 1, Foley 2015.

7. Claudot, F, Alla, F, Fresson, J, Calvez, T, Coudane, H, Bonaiti-Pellie, C. Ethics and observational studies in medical research: Various rules in a common framework. International Journal of Epidemiololgy 2009;38(4):1104-8;CrossRefGoogle Scholar Beauchamp, TL. Viewpoint: Why our conceptions of research and practice may not serve the best interest of patients and subjects. Journal of Internal Medicine 2011;269(4):383-7;CrossRefGoogle Scholar Vaughan, G, Pollock, W, Peek, MJ, Knight, M, Ellwood, D, Homer, CS, et al. Ethical issues: The multi-centre low-risk ethics/governance review process and AMOSS. Australian and New Zealand Journal of Obstetrics Gynaecology 2012;52(2):195-203;CrossRefGoogle ScholarPubMed Vezyridis, P, Timmons, S. Dissenting from care. Data: An analysis of opt-out forms. Journal of Medical Ethics 2016;42(12):792–6.CrossRefGoogle ScholarPubMed

8. See note 7, Beauchamp 2011.

9. See note 7, Vaughan 2012; Tully, J, Ninis, N, Booy, R, Viner, R. The new system of review by multicentre research ethics committees: Prospective study. BMJ 2000;320(7243):1179–82.CrossRefGoogle ScholarPubMed

10. Gefenas, E, Dranseika, V, Cekanauskaite, A, Hug, K, Mezinska, S, Peicius, E, et al. Non-equivalent stringency of ethical review in the Baltic States: a sign of a systematic problem in Europe? Journal of Medical Ethics 2010;36(7):435–9.CrossRefGoogle Scholar

11. See note 7, Vezyridis 2016, Hewison, J, Haines, A. Overcoming barriers to recruitment in health research. British Medical Journal 2006;333(7562):300–2.CrossRefGoogle ScholarPubMed

12. Kass, NE, Faden, RR, Goodman, SN, Pronovost, P, Tunis, S, Beauchamp, TL. The research-treatment distinction: A problematic approach for determining which activities should have ethical oversight. Hastings Center Report 2013;43(s1):S4S15.CrossRefGoogle Scholar

13. Wachter R. The Digital Doctor: Hope, Hype, and Harm at the Dawn of Medicine’s Computer Age. New York, NY: McGraw-Hill Education; 2015.

14. See note 12, Kass 2013.

15. See note 12, Kass 2013.

16. Waligora, M, Bala, MM, Koperny, M, Wasylewski, MT, Strzebonska, K, Jaeschke, RR, et al. Risk and surrogate benefit for pediatric Phase I trials in oncology: A systematic review with meta-analysis. Public Library of Science. Medicine 2018;15(2):e1002505.Google ScholarPubMed

17. Dekking, SA, van der Graaf, R, de Vries, MC, Bierings, MB, van Delden, JJ. Discriminating Between Research and Care in Paediatric Oncology—Ethical Appraisal of the ALL–10 and 11 Protocols of the Dutch Childhood Oncology Group (DCOG). In: Strech, D, Mertz, M, eds. Ethics and Governance of Biomedical Research. Springer; 2016:5972.Google Scholar

18. Faden, RR, Kass, NE, Goodman, SN, Pronovost, P, Tunis, S, Beauchamp, TL. An ethics framework for a learning health care system: A departure from traditional research ethics and clinical ethics. Hastings Center Report 2013;Spec No:S1627.CrossRefGoogle ScholarPubMed

19. See note 18, Faden 2013.

20. Elwyn, G, Frosch, D, Thomson, R, Joseph-Williams, N, Lloyd, A, Kinnersley, P, et al. Shared decision making: A model for clinical practice. Journal of General Internal Medicine 2012;27(10):1361–7.CrossRefGoogle ScholarPubMed

21. See note 18, Faden 2013.

22. National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. The Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research, Washington D.C., 1979.Google Scholar

23. Largent, EA, Miller, FG, Pearson, SD. Going off-label without venturing off-course: Evidence and ethical off-label prescribing. Archives of Internal Medicine 2009;169(19):1745–7.CrossRefGoogle ScholarPubMed

24. Kim, SY. Ethical issues in pragmatic trials of “standard-of-care” interventions in learning health care systems. Learning Health Systems 2018;2(1).CrossRefGoogle Scholar

25. Brody, H, Miller, FG. The clinician-investigator: Unavoidable but manageable tension. Kennedy Institute Ethics Journal 2003;13(4):329–46.CrossRefGoogle ScholarPubMed

26. Rothstein, MA. Rethinking the meaning of public health. The Journal of Law, Medicine & Ethics 2002;30(2):144–9.CrossRefGoogle ScholarPubMed

27. Verweij, M, Dawson, A. The meaning of ‘public’ in ‘public health.’ In: Dawson, A, Verweij, M, editors. Ethics, Prevention, and Public Health. Issues in Biomedical Ethics. Oxford: Oxford University Press; 2007:1329.Google Scholar

28. Gostin, LO, Stone, L. Health of the people: The highest law. In: Dawson, A, Verweij, M, editors. Ethics, Prevention, and Public Health. Issues in Biomedical Ethics. Oxford: Oxford University Press; 2007:5977.Google Scholar

29. Thacker, SB, Berkelman, RL. Public health surveillance in the United States. Epidemiologic Review 1988;10:164–90.CrossRefGoogle ScholarPubMed

30. Fairchild, AL, Bayer, R. Public health. Ethics and the conduct of public health surveillance. Science 2004;303(5658):631–2.CrossRefGoogle ScholarPubMed Willison, DJ, Ondrusek, N, Dawson, A, Emerson, C, Ferris, LE, Saginur, R, et al. What makes public health studies ethical? Dissolving the boundary between research and practice. BMC Medical Ethics 2014;15(1):61.CrossRefGoogle ScholarPubMed

31. Fairchild, AL. Dealing with Humpty Dumpty: Research, practice, and the ethics of public health surveillance. Journal of Law Medicine and Ethics 2003;31(4):615–23.CrossRefGoogle ScholarPubMed

32. See note 31, Fairchild 2003, Fairchild, AL, Johns, DM. Beyond bioethics: Reckoning with the public health paradigm. American Journal of Public Health 2012;102(8):1447–50.CrossRefGoogle ScholarPubMed

33. Jonas, H. Philosophical reflections on experimenting with human subjects. Daedalus 1969:219–47.Google Scholar

34. Holland, S. Public Health Ethics. Cambridge: John Wiley & Sons; 2015.Google Scholar

35. See note 28, Gostin 2007.

36. See note 13, Wachter 2015.

37. Piasecki, J, Dranseika, V, Waligora, M. Should epidemiological studies be subject to an ethics review? Public Health Ethics 2018;11(2):213–22.CrossRefGoogle Scholar