Chapter 1 The true nature of sleep loss-induced “neurocognitive performance deficits”: a critical appraisalThomas J. Balkin
Over the past 100-plus years, a plethora of studies have been performed to determine the effects of sleep loss on cognitive and psychomotor performance . From an “applied science” standpoint, many of these efforts have proven useful, yielding results with clear, readily generalizable implications for real-world application. For example, simulator studies over the past several decades have produced data useful for quantifying the relationship between sleep, the circadian rhythms of alertness, and driving performance – providing the basis for informed decision-making in operational environments (e.g., with respect to work/rest scheduling) and for crafting legislation to generally improve highway safety . Similarly, results from studies of medical personnel have revealed the extent to which extended work hours and resulting sleep loss contribute to medical errors and accidents  – findings that (a) have prompted the Accreditation Council for Graduate Medical Education (ACGME) to limit resident duty hours to 80 hours per week, and (b) may ultimately result in increased government (e.g., Occupational Health and Safety Administration (United States) – OSHA) oversight .
For those of us who conduct such studies, it is gratifying to see these findings applied to improve human health, performance, productivity, safety, and well-being. However, as scientists, our motivation for conducting such studies also has included a desire to address soul-satisfying basic research questions – i.e., to make discoveries that meaningfully contribute to the store of scientific knowledge. Accordingly, perhaps our fondest collective daydream has been that results of these sleep deprivation studies would ultimately reveal some profound scientific truth about the nature of sleep.1 Thus far, despite 100-plus years of assiduous sleep deprivation/performance research, this epiphany remains elusive. Why is this?
Sleep is a unique process
The logic behind sleep deprivation studies is basically sound. Biologists long ago determined that an excellent way to discover the function of an organ is to ablate that organ and see what subsequently goes wrong. However, there are several reasons why ablation strategies have not proven particularly useful for unlocking sleep’s greatest mysteries.
The first (and perhaps most obvious) problem is that sleep is a process that not only occurs in, and is mediated by, the brain, but is also a process that undoubtedly confers unique benefits to the brain itself. This makes sleep far less accessible and less amenable to ablation techniques than actual organs such as the kidney. The kidneys are, of course, the site of glomerular filtration, a process by which waste products are removed from the blood. Glomerular filtration serves to maintain the health and functioning of virtually every living cell in the body. It does not provide any unique benefits to the kidney, per se. This makes it possible to ablate the kidneys (and thereby ablate the process of glomerular filtration) in a manner that allows meaningful observation and measurement of consequences for the organism – consequences that allow definitive inferences regarding kidney function.
Clearly, it would make no sense to ablate the brain in an attempt to discover the functions of sleep. So instead of eliminating sleep by ablating the brain, the strategy has been to eliminate (ablate) sleep while (presumably) leaving the brain intact – and to then look for deficits in brain function, as manifested behaviorally on neurocognitive performance tests.2
The grand idea (or perhaps it is more accurately described as a “fond hope”) behind this endeavor has been that the accrual of information gleaned from such studies would eventually coalesce into a gestalt in which sleep’s most elusive secrets are revealed. That is, the hope has been that findings from each sleep loss study would serve as “puzzle pieces” that incrementally add to our knowledge regarding sleep function, and achievement of the ultimate goal – a coherent “big picture” revealing the process(es) by which sleep subserves waking brain function – would be achieved when a sufficient number of such puzzle pieces had been added.
Serious doubts about the ultimate utility of this approach date back to at least 1976, when Dr. Paul Naitoh commented that sleep deprivation studies had served only “to confirm a truism: (sleep deprivation) makes animals and humans sleepy” . Nevertheless, a perfunctory search of the recent scientific literature suggests that hope persists. Recent examples include papers describing the effects of sleep loss on neurocognitive functions such as impulsivity [6–7], risk-taking , moral reasoning , humor appreciation , working memory , and the ability to recognize human emotions [12–13], to name but a few.
This is not meant to imply that such papers are devoid of scientific value. Clearly, such studies can be of considerable scientific interest in and of themselves, and can produce results with clear implications for predicting performance in military and civilian operational environments. But it is the thesis of the present chapter that findings from such studies will always be of limited value in the larger quest to unlock the mysteries surrounding the nature of sleep. Here’s why:
Measuring of the effects of sleep loss on specific neurocognitive abilities requires some “leaps of faith”
In the parlance familiar to those who (like the present author) are afflicted with a degree in experimental psychology and have thus, at some point in their training, been subjected to a course in “The Philosophy of Science,” the scientific paradigm under which sleep deprivation research is conducted contains some conceptual gaps.
The difficulty is as follows: extended (e.g., 24 hours of) continuous wakefulness is an antecedent condition (i.e., the “cause” in a cause/effect relationship) that leads to a predictable, observable outcome: decremented performance (e.g., slowed reaction time on the Psychomotor Vigilance Test, PVT).
From a strict operationalist3 viewpoint, this empirical relationship is clear and (at least potentially) utilitarian. One does not necessarily need to explain how or why a cause/effect relationship works in order to apply it (e.g., as the basis of a mathematical performance prediction model to inform development of work/rest schedules in an industrial setting). And the logic behind operationalism is irrefutable – if one never “goes beyond the data” by offering conjecture about the unseen mechanisms by which observed variables relate to each other, one can never be “wrong.”
However, strict application of operationalism is also clearly antithetical to the advancement of science, since it eliminates the opportunity to posit possible explanatory mechanisms – i.e., to generate hypotheses regarding the nature of the unseen processes that underlie observed relationships between variables . Therefore, science affords its practitioners some reasonable leeway – hypotheses involving unseen forces and mechanisms are generated, but with the requirement that such hypotheses be parsimonious and plausible (i.e., in a sense, that such hypotheses be produced while maintaining some mindfulness of the logic of operationalism). As described by Marx and Hillix ,
|Image not available in HTML version|
Accordingly, as sleep researchers, we are afforded the opportunity to engage in a modicum of such speculation, but with the implied caveat that this speculation must be both minimal and reasonable.4 In that spirit we have, as a research and clinical community, adopted a scientific paradigm that includes some practical hypothetical constructs.5 As indicated in Figure 1.1, “sleep deprivation” is one. In this example, sleep deprivation is a hypothesized (not directly observable) physiological state reflecting the extent to which unspecified sleep-dependent homeostatic processes mediate neurocognitive performance capacity (more specifically in this example, “vigilance”).
Also, in this paradigm “sleepiness” is the manifestation of “sleep deprivation” – it is the “intervening variable”6 invoked to explain the causal relationship between sleep deprivation (i.e., extended wakefulness) and deficits in vigilance (e.g., performance on the PVT). In addition, it is important to note that “neurocognitive performance” and “vigilance” are themselves hypothetical constructs – in this case, hypothetical constructs that are operationally defined in terms of PVT performance.
So it can be seen that in a typical sleep deprivation/neurocognitive performance study, some period of extended continuous wakefulness is applied to produce “sleep deprivation” (a hypothetical construct that is operationally defined by the intervention) that is, in turn, manifested as reduced vigilance (itself a hypothetical construct that is operationally defined as decremented performance on an objective neurocognitive test, such as the PVT), with “sleepiness” as the underlying, not-directly-observable intervening variable that is invoked to “explain” the causal relationship. Thus, a typical study to determine the effects of sleep loss on neurocognitive performance can be conceptualized as an effort to determine how one operationally defined hypothetical construct impacts another operationally defined hypothetical construct, via a not-directly-observable intervening variable.
Within the realm of sleep research, it is generally considered reasonable to operationally define “sleep deprivation” as a brain state induced by 24 hours of continuous wakefulness – based, for example, on what is known about how much sleep is typically obtained by humans, the effect that this duration of continuous wakefulness has on subjective measures of alertness, etc. Likewise, it is reasonable to operationally define “vigilance” as mean response time on the PVT, since most would agree that “ability to attend to a task and respond to the appearance of intermittently presented stimuli” is central to the concept of vigilance. Thus, the “leaps of faith” between the hypothetical construct of “sleep deprivation” and its operational definition of “24 hours of continuous wakefulness” – and between the hypothetical construct of “vigilance” and its operational definition as “performance of the PVT” – are reasonably small. This is important because the extent to which studies reveal “scientific truth” depends, in part, on the extent to which such operational definitions faithfully reflect the “essence” of the hypothetical constructs they have been assigned to represent.
Accordingly, it is reasonable and appropriate to conduct an experiment in which the duration of continuous EEG-defined wakefulness is extended (i.e., duration of wakefulness is the “independent variable”), and the effect of this manipulation is measured on an instrument such as the PVT (the “dependent variable”). Then, based on the outcome, it may be reasonable to conclude from such a study that “sleep deprivation” results in “impaired vigilance” – with implications for real-world outcomes such as “increased risk of highway accidents.”
However, it is quite another matter to look at the results of such a study and draw conclusions about the nature (e.g., physiology, adaptive significance, etc.) of sleep itself. In part, this is because such conclusions require greater distention of the principles of operationalism, with the meaning of “sleep deprivation” expanded from its original, relatively straightforward operational definition (e.g., 24 hours of EEG-defined wakefulness) to something that is at least one step further removed from the “operation” of maintaining EEG-defined wakefulness for 24 hours – i.e., changes in brain physiology that result from 24 hours of continuous EEG-defined wakefulness. And the dependent variable, PVT performance, not only represents the hypothetical construct of vigilance, it must likewise be considered a direct reflection of the brain processes that underlie vigilance.
The price paid for widening the gap (lengthening the “leap of faith”) between hypothetical constructs and their operational definitions is steep: to the extent that independent and dependent variables represent increasingly nebulous hypothetical constructs (that are less directly and firmly tied to the operational definitions), conclusions from such studies likewise become more tenuous and nebulous, with the ability to uncover scientific truths accordingly diminished.
Neurocognitive performance deficits are non-specific
Even if, for the sake of argument, the logical problems associated with bridging the gaps between hypothetical constructs and operational definitions are solved, it would still be impossible to work backward from the results of sleep deprivation/neurocognitive studies to make significant discoveries regarding the nature of sleep. This is because although neurocognitive performance varies as a function of sleep debt, it does not vary only as a function of sleep debt.
For example, even vigilance performance – a neurocognitive capability that is widely known to vary as a function of alertness/sleepiness – is recognized to be a function of multiple factors. This is illustrated in the equation below (adapted from ):Pv=f(M,S,U,B,C)EBSin which Pv is performance on a vigilance test. Pv is a function of several factors including “signal modality” (M, the sensory modality of the signal being tested); salience (S, the meaningfulness of the signal being presented, e.g., the subject’s name vs. a neutral tone); the uncertainty of the signal (U, which depends on the mathematical likelihood of a signal presentation; the “background events density” (B, for example the density of “competing signals” that are presented and must be distinguished from the target signal); and “signal complexity” (C, reflecting, for example, the amount of mental processing required to identify the signal).
To make matters more complicated, in addition to interacting with each other, each of these factors can interact with a potentially infinite variety of environmental factors (E, e.g., environmental noise, ambient temperature, vibration) – at least as potential distracters, if not as factors that more directly influence vigilance performance.
And finally, the influence of each of these factors is potentially mediated by “brain state” (BS) – i.e., the variable readiness and capacity of the brain to perceive, process, and react to all of the factors that reside in the numerator of the depicted equation. For the purpose of the present discussion, BS could refer specifically to the brain’s level of “sleep debt” (although it could also refer to severity of intoxication, hypoxia, neuronal pathology, etc.).
Obviously, because there are numerous unknowns in this equation, and because there are numerous potential interactions among these unknowns, it would be absurd to actually try to solve this equation for “brain state” (BS), as follows:BS=f(M,S,U,B,C)EPvThe hope that findings from studies of sleep deprivation loss effects on neurocognitive performance (Pv in the present equation in which “v” is replaced by the neurocognitive performance task at hand) can be utilized to reveal the nature of sleep/sleep deprivation (BS in the present equation) is essentially to hope that this unsolvable equation can be solved.
And to hope that the accretion of findings from additional sleep deprivation studies [e.g., on “moral reasoning” (Pmr), “emotional intelligence” (Pei), “arithmetic problem solving” (Paps) etc.] will somehow result in an improved understanding of the nature of sleep is to compound the error. Such optimism amounts to hoping that the scientific truths that remain hidden in the current store of unsolvable equations will somehow emerge with the accrual of additional unsolvable equations, i.e., that the basic problem is not that the equations are unsolvable but that the number of unsolvable equations is inadequate.
A given task’s sensitivity to sleep deprivation varies as a function of that task’s parameters
Finally, even if bridging the gaps between hypothetical constructs and operational definitions presented no difficulty, and even if there was some aspect of neurocognitive performance that varied only as a function of sleep deprivation (i.e., non-specificity was not a problem), attempting to discern the nature of sleep based on findings from sleep deprivation/neurocognitive performance studies would still be a fool’s errand.
This is because the sensitivity (to sleep loss) of neurocognitive tests depends not only on the content of the tests (i.e., the neurocognitive abilities that these tests purportedly reflect) but also on the parameters of the tests. In a classic series of studies in the 1960s, Wilkinson  showed that the sensitivity of behavioral measures to sleep deprivation varies as a function of (a) test duration/time-on-task (with longer tests being more sensitive); (b) the extent to which the task is inherently interesting (with more interesting/rewarding tasks generally less affected by sleep loss); (c) the amount of feedback provided (tests for which no feedback is provided are generally more sensitive to the effects of sleep loss); and (d) “task complexity” (with those tasks that are more “complex” and thus require more effort being relatively more sensitive to the effects of sleep loss). In addition, “task sequence” can affect a given task’s sensitivity: tasks administered toward the end of a block of tests appear to be more sensitive to sleep loss when in reality decrements on these tasks reflect residual fatigue, boredom, cognitive resource depletion, etc.
Adapted from Balkin et al. , Figure 1.2 depicts the relative sensitivity of various measures to sleep loss during a sleep restriction study. In this study, mean sleep latency was found to be most sensitive, followed by PVT speed, standard deviation of lane position on a simulated driving task, mean speed on a 4-choice reaction time test, etc. Based on such findings, one might be tempted to conclude that the primary function of sleep is to maintain wakefulness (i.e., prevent frank sleep onset during waking hours), followed by facilitation of brain processes that mediate vigilance, reaction time, etc. However, considering the fact that the relative sensitivity of the various measures depends not only on the aspect of neurocognitive performance being measured but also on content-independent test parameters (such as test duration), it becomes clear that such interpretations are not possible. This is because, for
|Image not available in HTML version|
In short, while it is appropriate to conclude that sleep loss impacts “ability to maintain wakefulness,” vigilance, reaction time, etc., it is not possible, based on such studies, to conclude that sleep loss impacts one neurocognitive ability to a greater extent than it impacts another – a logical limitation that substantially delimits the extent to which such studies can be used to reveal the nature or function of sleep.
Summary and conclusions
Results from studies conducted to determine the effects of sleep loss on various neurocognitive abilities have proven useful for informing policy- and decision-making in a variety of operational and regulatory environments – and the utilitarian value of such studies for testing work/rest schedules, drug effects, etc. remains high. However, the value of such studies for addressing basic questions regarding the nature and function of sleep is severely limited because (a) logical “leaps of faith” are required to bridge gaps between operational definitions and hypothetical constructs; (b) the specificity of neurocognitive performance is low (i.e., neurocognitive performance is affected by a potentially infinite number of factors); and (c) the sensitivity of a neurocognitive test to sleep loss depends as much upon test parameters as it does on the specific neurocognitive ability reflected by that test.
This material has been reviewed by the Walter Reed Army Institute of Research, and there is no objection to its presentation and/or publication. The opinions or assertions contained herein are the private views of the author and are not to be construed as official or as reflecting the position of the Department of the Army or the Department of Defense.
1 Typically, in this daydream, we are poring over the data from our most recent sleep deprivation study when, suddenly, the “scales fall from our eyes” and, by dint of viewing the world through the new perspective afforded by the fresh findings, we realize how all of the pieces of the puzzle fit together to solve sleep’s most perplexing mysteries (“Eureka!”)
2 Sleep loss effects have also been measured on many other aspects of performance and physiology. But for the purposes of the present paper, discussion will be limited to the effects of sleep loss on neurocognitive performance.
3 To a strict operationalist, it is not logical to make inferences regarding how or why the antecedent condition (e.g., extended wakefulness) results in the observed outcome (e.g., decremented neurocognitive performance) – because the how and why are not directly observable. Logic only permits one to describe the empirically demonstrated relationship, without engaging in conjecture regarding unobserved forces or variables.
4 In this case, “minimal and reasonable” speculation should generally be considered that which is parsimonious –the minimum (in both amount and complexity) required to fill logical gaps in observed cause-effect relationships.
5 A hypothetical construct is an explanatory variable that cannot be observed directly, but is nevertheless invoked broadly to explain phenomena that are observable.
6 As used here, an intervening variable is a hypothetical internal state that is invoked specifically to help “explain” the causal relationship between antecedent conditions and outcomes.
 Durmer JS, Dinges DF Neurocognitive consequences of sleep deprivation. Semin Neurol 2005; 25(1): 117–29.
 Balkin TJ, Thorne D, Sing H, et al. Effects of Sleep Schedules on Commercial Motor Vehicle Driver Performance. FMCSA Technical Report No. DOT-MC-00–133. Washington, D.C: U.S. Department of Transportation; 2000.
 Landrigan CP, Rothschild JM, Cronin JW, et al. Effect of reducing interns’ work hours on serious medical errors in intensive care units. N Engl J Med 2004; 351(18): 1838–48.
 Citizen.org (Internet). Washington, DC: Public Citizen. c2010 (updated 2010 Sep 2). Available from: http://www.citizen.org/Page.aspx?pid=4294
 Naitoh P. Sleep deprivation in human subjects: A reappraisal. Waking and Sleeping 1976; 1: 53–60.
 Acheson A, Richards JB, de Wit H. Effects of sleep deprivation on impulsive behaviors in men and women. Physiol Behav 2007; 91(5): 579–87. Epub 2007 Mar 31.
 Anderson C, Platten CR. Sleep deprivation lowers inhibition and enhances impulsivity to negative stimuli. Behav Brain Res 2010; Oct 1 [Epub ahead of print].
 Killgore WD. Effects of sleep deprivation and morningness-eveningness traits on risk-taking. Psychol Rep 2007; 100(2): 613–26.
 Olsen OK, Pallesen S, Eid J. The impact of partial sleep deprivation on moral reasoning in military officers. Sleep 2010; 33(8): 1086–90.
 Killgore WD, McBride SA, Killgore DB, Balkin TJ. The effects of caffeine, dextroamphetamine, and modafinil on humor appreciation during sleep deprivation. Sleep 2006; 29(6): 841–7.
 Gohar A, Adams A, Gertner E, et al. Working memory capacity is decreased in sleep-deprived internal medicine residents. J Clin Sleep Med 2009; 5(3): 191–7.
 Killgore WD, Kahn-Greene ET, Lipizzi EL, et al. Sleep deprivation reduces perceived emotional intelligence and constructive thinking skills. Sleep Med 2008; 9(5): 517–26.
 van der Helm E, Gujar N, Walker MP. Sleep deprivation impairs the accurate recognition of human emotions. Sleep 2010; 33(3): 335–42.
 Benjamin, AC. Operationalism. Springfield: Charles C. Thomas; 1955.
 Marx MH, Hillix WA. Systems and Theories in Psychology, 2nd ed. New York: McGraw-Hill Book Company; 1973.
 Wilkinson RT. Sleep deprivation. In: Edholm OG, Bacaharach AL, editors. The Physiology of Human Survival. London: Academic Press; 1965: 399–430.
 Balkin TJ, Bliese PD, Belenky G, et al. Comparative utility of instruments for monitoring sleepiness-related performance decrements in the operational environment. J Sleep Res 2004; 13: 219–27.
 Warm JS, Vigilance and target detection. In: Wickens CD, Huey BM, editors. Workload Transition: Implications for Individual and Team Performance. Washingdon, DC: National Research Council; 1993: 139–170.
© Cambridge University Press