Hostname: page-component-7c8c6479df-8mjnm Total loading time: 0 Render date: 2024-03-27T08:47:47.689Z Has data issue: false hasContentIssue false

The development, and day-to-day variation, of a Military-Specific Auditory N-Back Task and Shoot-/Don’t -Shoot Task

Subject: Psychology and Psychiatry

Published online by Cambridge University Press:  24 August 2022

Christopher Vine*
Affiliation:
Occupational Performance Research Group, Institute of Sport, Nursing and Allied Health, University of Chichester, Chichester, United Kingdom
Sarah Coakley
Affiliation:
Occupational Performance Research Group, Institute of Sport, Nursing and Allied Health, University of Chichester, Chichester, United Kingdom Faculty of Sport, Allied Health and Performance Science, St Mary’s University, Twickenham, United Kingdom
Stephen Myers
Affiliation:
Occupational Performance Research Group, Institute of Sport, Nursing and Allied Health, University of Chichester, Chichester, United Kingdom
Sam Blacker
Affiliation:
Occupational Performance Research Group, Institute of Sport, Nursing and Allied Health, University of Chichester, Chichester, United Kingdom
Oliver Runswick
Affiliation:
Occupational Performance Research Group, Institute of Sport, Nursing and Allied Health, University of Chichester, Chichester, United Kingdom Institute of Psychiatry, Psychology & Neuroscience, Kings College London, London, United Kingdom
*
*Corresponding author. Email: c.vine@chi.ac.uk

Abstract

During military operations, soldiers are required to successfully complete numerous physical and cognitive tasks concurrently. Understanding the typical variance in research tools that may be used to provide insight into the interrelationship between physical and cognitive performance is therefore highly important. This study assessed the inter-day variability of two military-specific cognitive assessments: a Military-Specific Auditory N-Back Task (MSANT) and a Shoot-/Don’t-Shoot Task (SDST) in 28 participants. Limits of agreement ±95% confidence intervals, standard error of the mean, and smallest detectable change were calculated to quantify the typical variance in task performance. All parameters within the MSANT and SDST demonstrated no mean difference for trial visit in either the seated or walking condition, with equivalency demonstrated for the majority of comparisons. Collectively, these data provided an indication of the typical variance in MSANT and SDST performance, while demonstrating that both assessments can be used during seated and walking conditions.

Type
Research Article
Information
Result type: Novel result
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
© The Author(s), 2022. Published by Cambridge University Press

Introduction

During military operations, personnel are required to maintain performance in both their role-specific physical tasks (e.g., load carriage) and corresponding cognitive tasks (e.g., decision making and communication) (Crawford et al., Reference Crawford, Teo, Lafferty, Drake, Bingham, Gallon, O’connell, Chittum, Arzola and Berry2017; Scribner, Reference Scribner2016). Failure to maintain performance, in either domain, can result in reduced combat readiness and decreased operational performance (Crawford et al., Reference Crawford, Teo, Lafferty, Drake, Bingham, Gallon, O’connell, Chittum, Arzola and Berry2017; Vrijkotte et al., Reference Vrijkotte, Roelands, Meeusen and Pattyn2016). Consequently, there is growing interest in the relationship between military-specific physical activity and cognitive performance within military operators (Armstrong et al., Reference Armstrong, Smith, Risius, Doyle, Wardle, Greeves, House, Tipton and Lomax2022; Bhattacharyya et al., Reference Bhattacharyya, Pal, Chatterjee and Majumdar2017; Eddy et al., Reference Eddy, Hasselquist, Giles, Hayes, Howe, Rourke, Coyne, O’Donovan, Batty and Brunyé2015; Giles et al., Reference Giles, Hasselquist, Caruso and Eddy2019; Kobus et al., Reference Kobus, Brown, Wu, Robusto and Bartlett2010; Nibbeling et al., Reference Nibbeling, Oudejans, Ubink and Daanen2014; Son et al., Reference Son, Hyun, Beck, Jung and Park2019, Reference Son, Jung, Hwang, Beck and Park2022; Vine et al., Reference Vine, Myers, Coakley, Blacker and Runswick2021). Despite this interest, the methodologies and approaches used to investigate this relationship have differed considerably, particularly concerning the assessment of cognitive performance.

Based on the assessment tools used to date, and the visual and auditory requirements of soldiers, two assessment tools were developed: A Military Specific Auditory N-Back Task (MSANT), and a Shoot-/Don’t-Shoot (SDST). The former used phonetically described pairs of letters and represented aspects of military radio communications, while the latter represented aspects of any military scenario where visual search and inhibition are required (e.g., assaulting an enemy position or operations in builtup areas). The current study, therefore, aimed to detail the methodology of the MSANT and SDST, along with quantifying the typical day-to-day variability of both assessment tools under seated and walking condition. The investigation did not seek to investigate the influence of physical fatigue or dual tasking on the performance of these assessment tools.

Methods

The full methods for this project are available in the Supplementary Material, with the raw data available at: https://osf.io/jekv8/. Briefly, the study comprised of two elements. First, the day-to-day variability of the MSANT and SDST was assessed in a seated condition on three separate occasions (Part 1). This was chosen due to the large variability in potential application of these assessment tools in future projects. Second, within a sub-sample of the study population, the day-to-day variability of the MSANT and SDST was assessed during a 10-min walking activity, on three separate occasions (Part 2). While a matched study population for this part of the study would have been optimal, given the time required for this portion of the study (a result of the necessity to reach a physiological steady state before conducting the test, and the recovery period required between each walking bout to prevent the onset of fatigue), a sub-sample approach was instead chosen. Physiological steady state refers to the stabilization in the physiological responses to exercise (e.g., increases in heart rate). Without this stability, variability in cognitive performance could be induced as a consequence of adapting to the exercise stimulus opposed to just reflecting the typical variation in test performance.

All laboratory visits were separated by a minimum of 48 hr, and participants were required to arrive in a fed and hydrated state having avoided caffeine for a minimum of 3 hr. Study visits were completed at approximately the same time of day (±2 hr) to control for the potential effect of circadian rhythm on test performance. All participants were recruited from the university population (all were students or from academic positions), spoke fluent English, and had self-declared normal or corrected to normal vision.

Twenty-eight participants volunteered for Part 1 of the study (14 male, 14 female, age [mean ± SD] 27.3 ± 4.3 year) and 12 participants for Part 2 (6 male, 6 female, age 28.4 ± 3.5 year). Sample size for Part 1 was calculated using an a priori power calculation (G Power; version 3.1.9.4) (Prajapati et al., Reference Prajapati, Dunne and Armstrong2010). For the seated portion of the investigation, 28 participants were required to a moderate effect size (f = 0.25), with a statistical power of 80% and an alpha level of 0.05, based upon a correlation coefficient of r = 0.5 (identified from initial pilot testing). A moderate effect size (Cohen, Reference Cohen1988) was selected based on the combination of effect sizes reported in previous investigations, utilizing similar cognitive assessment tools (Eddy et al., Reference Eddy, Hasselquist, Giles, Hayes, Howe, Rourke, Coyne, O’Donovan, Batty and Brunyé2015), and the anticipated smallest effect size of interest to military policymakers. The sub-sample size was designed to represent the typical size (and therefore likely variation) of study populations within this research area (e.g., Bhattacharyya et al., Reference Bhattacharyya, Pal, Chatterjee and Majumdar2017; Crowell et al., Reference Crowell, Krausman, Harper, Faughn and Sharp1999; Eddy et al., Reference Eddy, Hasselquist, Giles, Hayes, Howe, Rourke, Coyne, O’Donovan, Batty and Brunyé2015). Ethical approval was provided by the Institution’s Research Ethics Committee, with written consent obtained from all participants.

Cognitive assessments

The MSANT involved identifying a pair of phonetically described letters two previous to an auditory tone (i.e., 2-back). During the seated condition, participants recorded their answers, while during walking trials, participants were required to relay their answers verbally which were recorded on their behalf. The SDST was designed to be a visual search and inhibition task similar to those tasks previously employed within the literature (Armstrong et al., Reference Armstrong, Smith, Risius, Doyle, Wardle, Greeves, House, Tipton and Lomax2022; Eddy et al., Reference Eddy, Hasselquist, Giles, Hayes, Howe, Rourke, Coyne, O’Donovan, Batty and Brunyé2015; Kobus et al., Reference Kobus, Brown, Wu, Robusto and Bartlett2010). The assessment involved responding appropriately to targets and non-targets. Participants were instructed to place equal importance on both response time and accuracy. For the SDST, there was a 2:1 ratio between targets and non-targets.

For Part 1, during the first visit, participants were familiarized (two full trial completions of each assessment) with the MSANT and SDST, in a randomized counterbalanced order. For the second, and third visits, participants completed the MSANT and SDST in the same randomized counterbalanced order. For Part 2, a sub-sample of 12 participants completed 3 additional laboratory visits completing the SDST and MSANT while walking on a treadmill. Again the MSANT and SDST were completed in a randomized order. All tests were completed with 10 min of seated rest between trials to negate the influence of physical fatigue. To enable a physiological steady state to occur, participants completed 5 min of walking before the commencement of the cognitive assessments. For all walking trials, participants walked on a motorized treadmill (6.5 km·h−1, 1% gradient) at a load carriage speed representing a typical “enemy contact” speed (Armstrong et al., Reference Armstrong, Ward, Lomax, Tipton and House2019).

Statistical analysis

Data were principally analyzed using JASP (JASP, 2020; v0.14.1). For normally distributed data, a one-way ANOVA was employed to identify whether a likely main effect of assessment time point was apparent. Effect sizes are presented as Omega squared (Ѡ 2) (Levine & Hullett, Reference Levine and Hullett2002). For non-normally distributed data, a Friedman’s test was employed with effect sizes presented using Kendall’s W. Holm-Bonferroni adjusted pairwise comparisons, and pairwise comparisons using Conover’s test were made post-hoc as appropriate. For key assessment variables, equivalency between trials was calculated using the two one-sided test approach (Lakens et al., Reference Lakens, Scheel and Isager2018). Based upon the a priori sample size calculation, d = 0.5 was employed as the smallest effect size of interest. To describe the typical variation in assessment parameters between trials, Limits of agreement ± 95% confidence intervals, standard error of the mean, and smallest detectable change values were calculated (Hopkins, Reference Hopkins2000; Ludbrook, Reference Ludbrook2010; van Kampen et al., Reference van Kampen, Willems, van Beers, Castelein, Scholtes and Terwee2013).

Results

Descriptive statistics are presented in Table 1, with day-to-day variation descriptors reported in Table 2. One participant was removed from the analysis, due to being more than two SDs outside the remainder of the data set.

Table 1. Descriptive statistics for cognitive assessments (mean ± SD [range]) during seated (S) and walking (W) conditions

Note. Blank cells denote data that were not collected due to the seated condition acting as the familiarization for the walking condition.

Abbreviations: ASTO, accuracy-speed trade-off; CR, correct response; FAM, familiarization; MSANT, Military-Specific Auditory N-Back Task; RT, response time; S, seated; SDST, Shoot-/Don’t-Shoot Task; W, walking.

Table 2. Descriptive statistics for cognitive assessments (mean ± SD) during seated (S) and walking (W) conditions

Abbreviations: ASTO, accuracy-speed trade-off; CI, confidence intervals; CR, correct response; MSANT, Military-Specific Auditory N-Back Task; RT, response time; S, seated; SDC, smallest detectable change; SDST, Shoot-/Don’t-Shoot Task; SEM, standard error of the mean; W, walking.

Seated performance

Military-Specific Auditory N-Back Task

There was no likely main effect for time for total correct response (χ 2(4) = 4.531, p = .361, Kendall’s W = 0.492) or combined correct responses (χ 2(4) = 3.856, p = .426, Kendall’s W = 0.488); however, a likely main effect for time was evident for partial correct responses (χ 2(4) = 11.846, p = .019, Kendall’s W = 0.426). For the key variable of combined correct responses, the comparison between trial 1 versus trial 2 was both statistically equivalent (W (25) = 64, p = .002) and not statistically different (W (25) = 70, p = .938). Similarly, trial 2 versus trial 3 were both statistically equivalent (W (25) = 20, p = .06) and not statistically different. Likewise, trial 1 versus trial 3 were also both statistically equivalent (W (25) = 50, p = .032) and not statistically different.

Shoot-/Don’t-Shoot Task

There was no likely main effect for time on either shoot correct (χ 2(4) = 4.00, p = .406, Kendall’s W = 0.175), don’t-shoot correct (χ 2(4) = 3.069, p = .546, Kendall’s W = 0.482), total correct (χ 2(4) = 3.375, p = .497, Kendall’s W = 0.471), and average response time (F (2.981,77.515) = 1.035, p = .382, Ѡ 2 = 0.001). There was, however, a main effect for time in the accuracy-speed trade-off (ASTO) parameter (F (4,104) = 7.037, p < .001, Ѡ 2 = 0.089). Importantly, the sole noteworthy difference occurred between familiarization 1 and trial 3 (t (26) = 4.855, p < .001, d = 0.756) suggesting no discernible difference was likely between performances in the three experimental trials, following two familiarization trials. For the key variable of total correct responses, trial 1 versus trial 2, trial 1 versus trial 3, and trial 2 versus trial 3 were both statistically equivalent (1 vs. 2: W (26) = 93, p = .011; 1 vs. 3: W (26) = 61, p = .047; 2 vs. 3: W (26) = 41, p = .040) and not statistically different. For the other key variable of ASTO, all comparisons were likely neither statistically equivalent (1 vs. 2: t (26) = −1.701, p = .050; 2 vs. 3: t (26) = −0.127, p = .45; 1 vs. 3: t (26) = 0.287, p = .612) nor statistically different.

Walking performance

Military-Specific Auditory N-Back Task

As with seated MSANT performance, there was no likely effect of time on total correct responses (χ 2(2) = 1.000, p = .607, Kendall’s W = 0.568), partial correct responses (χ 2(2) = 1.280, p = .527, Kendall’s W = 0.541), and combined correct responses (χ 2(2) = 1.000, p = .607, Kendall’s W = 0.582). For the key variable of combined correct responses, trials 1 versus 2 and trials 2 versus 3 were statistically equivalent (1 vs. 2: W (11) = 12, p = .017; 2 vs. 3: W (11) = 13, p = .020) and not statistically different. Conversely, trial 1 versus 3 was neither statistically equivalent nor statistically different.

Shoot-/Don’t-Shoot Task

Again there were no likely effects of time on shoot correct responses (χ 2(2) = 4.800, p = .091, Kendall’s W = 0.449), don’t-shoot correct responses (χ 2(2) = 2.480, p = .289, Kendall’s W = 0.672), total correct responses (χ 2(2) = 3.161, p = .206, Kendall’s W = 0.741), response times (F (2,22) = 2.880, p = .077, Ѡ 2 = 0.018), and ASTO (F (2,22) = 2.713, p = .088, Ѡ 2 = 0.042). For the key variable of total correct responses, all comparisons were neither statistically equivalent (1 vs. 2: W (11) = 6, p = .096; 1 vs. 3: W (11) = 0, p = .093; 2 vs. 3: W (11) = 14, p = .084) nor statistically different. Similarly, for the other key variable of ASTO all comparisons were likely neither statistically equivalent (1 vs. 2: t (11) = 0.127, p = .549; 2 vs. 3: t (11) = −1.205, p = .127; 1 vs. 3: t (11) = 0.787, p = .776) nor statistically different.

Discussion

This study has described the methods of two military-specific cognitive assessment tools (MSANT and SDST) and quantified their typical day-to-day variability. These data provide typical magnitudes of variance for the key assessment parameters. While no likely performance differences were observed across the experimental measurement points, not all walking comparisons were statistically equivalent, suggesting additional data are required before this assertion is made, for the given equivalency bounds. It should however be noted that borderline statistically significant results may become non-significant where correction for multiple testing is utilized. The current investigation has also demonstrated the suitability of these assessment tools for use during military-specific physical activity within a laboratory setting.

Before this investigation, the day-to-day performance variation in any military-specific cognitive assessments had not been quantified. This is an issue for several reasons, including the translational ability of research findings to the “real world” (Close et al., Reference Close, Kasper and Morton2019) and also for methodological decision making (e.g., sample size calculations). Moreover, with military operations rarely conducted in isolation, information on inter-test performance is highly relevant to research investigating sequential or repeated bout performance. The comparison between seated and walking performance was not a research question of interest in the current study, particularly given that deficits in cognitive performance are typically observed after ~30 min of military activity (e.g., Eddy et al., Reference Eddy, Hasselquist, Giles, Hayes, Howe, Rourke, Coyne, O’Donovan, Batty and Brunyé2015; Giles et al., Reference Giles, Hasselquist, Caruso and Eddy2019). However, observationally, the typical variation in performance between trials appears similar between seated and walking conditions.

Familiarizing participants with assessment tools is critical for research, particularly when time limitations may inhibit access to study participants (e.g., military populations). Collectively, the current study’s data demonstrate that beyond two full seated trials, a continued improvement in performance was not likely apparent, suggesting this familiarization length is sufficient to minimize possible learning effects.

Several limitations exist with the current investigation, including the use of a civilian population, and the limited walking sub-sample size. As acknowledged previously, the smaller sub-sample size was chosen for largely practical reasons, although it does match many studies within this area, highlighting issues with underpowered investigations. Future research should attempt to pair reliable and applied cognitive tasks (such as those described herein) with operationally relevant and appropriate physical activity. This in turn will support enhanced applied research as well as enabling a greater focus to be placed on developing mitigation strategies where the greatest mission impact can be obtained.

Acknowledgment

The authors would like to thank the participants for their participation in the current project.

Supplementary Materials

To view supplementary material for this article, please visit http://doi.org/10.1017/exp.2022.11.

Authorship contributions

Research concept and study design: C.V., S.C., S.M., S.B., O.R; Data collection, data and statistical analyses, and writing of the initial manuscript: C.V.; Reviewing/editing final manuscript: C.V., S.C., S.M., S.B., and O.R.

Data availability statement

The raw data for this project can be viewed at: https://osf.io/jekv8/.

Funding statement

The authors declare no funding was received for this research.

Conflict of interest

The authors have no conflicts of interest to declare.

References

Armstrong, N., Ward, A., Lomax, M., Tipton, M. J., & House, J. R. (2019). Wearing body armour and backpack loads increase the likelihood of expiratory flow limitation and respiratory muscle fatigue during marching. Ergonomics, 62, 11811192.CrossRefGoogle ScholarPubMed
Armstrong, N. C., Smith, S. J. R., Risius, D., Doyle, D., Wardle, S. L., Greeves, J. P., House, J. R., Tipton, M., & Lomax, M. (2022). Cognitive performance of military men and women during prolonged load carriage. BMJ Military Health, e002000.CrossRefGoogle ScholarPubMed
Bhattacharyya, D., Pal, M., Chatterjee, T., & Majumdar, D. (2017). Effect of load carriage and natural terrain conditions on cognitive performance in desert environments. Physiology & Behavior, 179, 253261.CrossRefGoogle ScholarPubMed
Close, G. L., Kasper, A. M., & Morton, J. P. (2019). From paper to podium: quantifying the translational potential of performance nutrition research. Sports Medicine, 49, 2537.CrossRefGoogle ScholarPubMed
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Earlbaum Associates.Google Scholar
Crawford, C., Teo, L., Lafferty, L., Drake, A., Bingham, J. J., Gallon, M. D., O’connell, M. L., Chittum, H. K., Arzola, S. M., & Berry, K. (2017). Caffeine to optimize cognitive function for military mission-readiness: a systematic review and recommendations for the field. Nutrition Reviews, 75, 1735.CrossRefGoogle ScholarPubMed
Crowell, H. P., Krausman, A. S., Harper, W. H., Faughn, J. A., & Sharp, M. A. (1999). Cognitive and physiological performance of soldiers while they carry loads over various terrains. Army Research Laboratory.CrossRefGoogle Scholar
Eddy, M. D., Hasselquist, L., Giles, G., Hayes, J. F., Howe, J., Rourke, J., Coyne, M., O’Donovan, M., Batty, J., & Brunyé, T. T. (2015). The effects of load carriage and physical fatigue on cognitive performance. PloS One, 10, e0130817.CrossRefGoogle ScholarPubMed
Giles, G. E., Hasselquist, L., Caruso, C., & Eddy, M. D. (2019). Load carriage and physical exertion influence cognitive control in military scenarios. Medicine and Science in Sports and Exercise, 51, 25402546.CrossRefGoogle ScholarPubMed
Hopkins, W. G. (2000). Measures of reliability in sports medicine and science. Sports Medicine, 30, 115.CrossRefGoogle ScholarPubMed
JASP. (2020). JASP (Version 0.14.1)[Computer software].Google Scholar
Kobus, D. A., Brown, C. M., Wu, L., Robusto, K., & Bartlett, J. (2010). Cognitive performance and physiological changes under heavy load carriage. Pacific Science and Engineering Group Inc.CrossRefGoogle Scholar
Lakens, D., Scheel, A. M., & Isager, P. M. (2018). Equivalence testing for psychological research: A tutorial. Advances in Methods and Practices in Psychological Science, 1, 259269.CrossRefGoogle Scholar
Levine, T. R., & Hullett, C. R. (2002). Eta squared, partial eta squared, and misreporting of effect size in communication research. Human Communication Research, 28, 612625.CrossRefGoogle Scholar
Ludbrook, J. (2010). Confidence in Altman-Bland plots: A critical review of the method of differences. Clinical and Experimental Pharmacology and Physiology, 37, 143149.CrossRefGoogle Scholar
Nibbeling, N., Oudejans, R. R. D., Ubink, E. M., & Daanen, H. A. M. (2014). The effects of anxiety and exercise-induced fatigue on shooting accuracy and cognitive performance in infantry soldiers. Ergonomics, 57, 13661379.CrossRefGoogle ScholarPubMed
Prajapati, B., Dunne, M., & Armstrong, R. (2010). Sample size estimation and power analysis. Optometry Today, 16, 123132.Google Scholar
Scribner, D. R. (2016). Predictors of shoot–don’t shoot decision-making performance: An examination of cognitive and emotional factors. Journal of Cognitive Engineering and Decision Making, 10, 313.CrossRefGoogle Scholar
Son, M., Hyun, S., Beck, D., Jung, J., & Park, W. (2019). Effects of backpack weight on the performance of basic short-term/working memory tasks during flat-surface standing. Ergonomics, 62, 548564.CrossRefGoogle ScholarPubMed
Son, M., Jung, J., Hwang, D., Beck, D., & Park, W. (2022). The effect of backpack weight on the performance of basic short-term/working memory tasks while walking along a pre-determined route. Ergonomics, 123.CrossRefGoogle ScholarPubMed
van Kampen, D. A., Willems, W. J., van Beers, L. W. A. H., Castelein, R. M., Scholtes, V. A. B., & Terwee, C. B. (2013). Determination and comparison of the smallest detectable change (SDC) and the minimal important change (MIC) of four-shoulder patient-reported outcome measures (PROMs). Journal of Orthopaedic Surgery and Research, 8, 40.CrossRefGoogle Scholar
Vine, C. A. J., Myers, S. D., Coakley, S. L., Blacker, S. D., & Runswick, O. R. (2021). Transferability of military-specific cognitive research to military training and operations. Frontiers in Psychology, 12, 386.CrossRefGoogle ScholarPubMed
Vrijkotte, S., Roelands, B., Meeusen, R., & Pattyn, N. (2016). Sustained military operations and cognitive performance. Aerospace Medicine and Human Performance, 87, 718727.CrossRefGoogle ScholarPubMed
Figure 0

Table 1. Descriptive statistics for cognitive assessments (mean ± SD [range]) during seated (S) and walking (W) conditions

Figure 1

Table 2. Descriptive statistics for cognitive assessments (mean ± SD) during seated (S) and walking (W) conditions

Supplementary material: File

Vine et al. supplementary material

Vine et al. supplementary material

Download Vine et al. supplementary material(File)
File 1.1 MB
Reviewing editor:  Gregory Postal Uniformed Services University of the Health Sciences, F Edward Hebert School of Medicine, Psychiatry, 130 South Churchill Drive, Fayetteville, North Carolina, United States, 28303-5065
Minor revisions requested.

Review 1: The Development, and Day-to-Day Variation, of a Military-Specific Auditory N-Back Task and Shoot-/Don’t-Shoot Task

Conflict of interest statement

Reviewer declares none.

Comments

Comments to the Author: Dear Authors,

I understand that the current study will be focused on effect of cognitive trials according to two conditions (Part 1 or Part 2) on three separate occasions. There are, however, serious issues regarding extreme small samples, different sample-size for each part’s experiment, poor information on definition of three separate occasions, how to select statistical analyses to clarify the objective of the present study and so on. Especially, I would like to suggest kindly that selection of statistical methodology would be reconsidered to clarify effect of two cognitive trials according to two experimental conditions (Part 1 and Part 2). For instance, if the interactive effect between conditions and occasions for each cognitive trial was examined statistically according to the objective of this study, I would suggest application of the repeated two-way ANOVA to analyze the data collected in this study. Considering the above concerns carefully, I would like to suggest kindly that the manuscript might need to be rewritten after reanalyzing the data.

Presentation

Overall score 2.2 out of 5
Is the article written in clear and proper English? (30%)
4 out of 5
Is the data presented in the most useful manner? (40%)
1 out of 5
Does the paper cite relevant and related articles appropriately? (30%)
2 out of 5

Context

Overall score 1.5 out of 5
Does the title suitably represent the article? (25%)
2 out of 5
Does the abstract correctly embody the content of the article? (25%)
1 out of 5
Does the introduction give appropriate context? (25%)
1 out of 5
Is the objective of the experiment clearly defined? (25%)
2 out of 5

Analysis

Overall score 1.2 out of 5
Does the discussion adequately interpret the results presented? (40%)
1 out of 5
Is the conclusion consistent with the results and discussion? (40%)
1 out of 5
Are the limitations of the experiment as well as the contributions of the experiment clearly outlined? (20%)
2 out of 5

Review 2: The Development, and Day-to-Day Variation, of a Military-Specific Auditory N-Back Task and Shoot-/Don’t-Shoot Task

Conflict of interest statement

Reviewer declares none.

Comments

Comments to the Author: This study examined the reliability of Military Specific Auditory N-Back Task and Shoot-/Don’t-Shoot Task. The present study yielded interesting findings. Please find below some specific comments that need to be addressed.

1.The Part 2 study assessed the day-to-day variability of the MSANT and SDST during a 10-minute walking activity. What was the aim of the Part 2 study? Did the authors want to examine the effect of fatigue on the day-to-day variability of tests or the effects of dual-tasking on the variability?

2.How did you recruit participants. Did the study have inclusion and exclusion criteria regarding study participants?

3.Although previous studies may have described, please explain the methodology of MSANT and SDST in more detail.

4.I was not sure of the meaning of a physiological steady-state. Does five minutes of walking result in a physiological steady-state among participants? A five-minute break would probably bring more physiological stability.

5.Why did the authors calculate the sample size only in the Part 1 study?

6.What were the intervals between trials (i.e., trial 1 to trial 3)?

7.Education level may confound the results of MSANT and SDST. Did you consider the effect?

Presentation

Overall score 3.9 out of 5
Is the article written in clear and proper English? (30%)
5 out of 5
Is the data presented in the most useful manner? (40%)
3 out of 5
Does the paper cite relevant and related articles appropriately? (30%)
4 out of 5

Context

Overall score 3.5 out of 5
Does the title suitably represent the article? (25%)
3 out of 5
Does the abstract correctly embody the content of the article? (25%)
4 out of 5
Does the introduction give appropriate context? (25%)
4 out of 5
Is the objective of the experiment clearly defined? (25%)
3 out of 5

Analysis

Overall score 3.6 out of 5
Does the discussion adequately interpret the results presented? (40%)
4 out of 5
Is the conclusion consistent with the results and discussion? (40%)
3 out of 5
Are the limitations of the experiment as well as the contributions of the experiment clearly outlined? (20%)
4 out of 5