Note: this document is a supplement to the Instructions for Contributors,
which should also be consulted as it contains information relevant for all submissions.

Guidelines for authors

Registered Reports are a form of empirical article in which the methods and proposed analyses are pre-registered and reviewed prior to research being conducted. This format is designed to minimize bias in deductive science, while also allowing complete flexibility to conduct exploratory (unregistered) analyses and report serendipitous findings.

The cornerstone of the Registered Reports format is that a significant part of the manuscript will be assessed prior to data collection, with the highest quality submissions accepted in advance. Initial submissions will include a description of the key research question and background literature, hypotheses, experimental procedures, analysis pipeline, a statistical power analysis (or Bayesian equivalent), and pilot data (where applicable).

In the context of Journal of Child Language, this format is most likely to be suitable for larger scale experimental studies, including formal replications, as opposed to observational or case studies.

There are three stages of the submission/review process. At Stage 0, an initial proposal is considered by the editorial team. There are then two stages of review (Stages 1 and 2). Note that this format therefore requires reviewers to commit to both stages of the process (although authors should note that that there may be exceptional circumstances in which reviewers have to withdraw and are replaced). At Stage 1, reviewers assess study proposals before data are collected. Upon successful completion of this stage an “in principle acceptance” (IPA) commitment by the journal is given to the authors (see below). At Stage 2, reviewers consider the full study, including results and interpretation. At this stage, acceptance leads to publication.

Stage 0: Initial proposal (submitted by email to the Editor)

To obtain permission to submit a stage 1 proposal, the prospective author submits a brief proposal via email to the journal’s editorial office (child_language@yahoo.co.uk), which will be passed to the Editor or an Associate Editor for initial assessment. This email should contain:

1. A brief description of the planned study – background, research question, basic proposed methodology (1 paragraph).

2. A justification for why a Registered Report (RR) is preferred over traditional submission types for the proposed study (1-3 sentences). Authors are encouraged to refer to the likely replication value of the research. High-value replication studies are welcomed in addition to novel studies.

3. Sufficient detail to evaluate whether the proposed study is likely to be of high quality and appropriate for the RR format.

4. A statement confirming that if the paper is accepted at Stage 1 and they choose to later withdraw their paper, or are unable to complete it within a reasonable time frame, they agree to JCL publishing the abstract on the journal website.

5. A statement confirming that the authors agree in principle to share their raw data, any digital study materials, laboratory log, and Stage 1 manuscript for all published results. It is understood that there may be ethical reasons why some data cannot be shared, and this can be explained at this stage.

After receipt of the Stage 0 proposal, the Editor or Associate Editor writes back to invite the author to submit a Stage 1 manuscript, or to recommend the traditional publication route. Decision-making is based on whether the authors make a good case for the study to be conducted as a RR, and very generally whether the potential merit of the work would justify putting resources toward reviewing the submission further. No commitments are made at this stage.

Stage 1: Manuscript (submitted via the JCL ScholarOne website)

Initial Stage 1 submissions should include the following sections:

• Abstract
  • This is a standard abstract but would also be made available on the JCL website in the event that the study receives Stage 1 approval but does not reach completion in a timely fashion.

• Introduction
  • A review of the relevant literature that motivates the research question and a full description of the experimental aims and hypotheses. Please note that following IPA, the Introduction section should not be altered (see below).
• Methods and Methods Addendum
  • Full description of proposed sample characteristics, including criteria for data inclusion and exclusion (e.g. outlier extraction). Procedures for objectively defining exclusion criteria due to technical errors or for any other reasons must be specified, including details of how and under what conditions data would be replaced.
  • A description of experimental procedures in sufficient detail to allow another researcher to repeat the methodology exactly, without requiring further information. These procedures must be adhered to exactly in the subsequent experiments or any Stage 2 manuscript can be rejected.
  • Studies involving standard methods of statistical inference must include a statistical power analysis. Estimated effect sizes should be justified with reference to the existing literature. In the case of highly uncertain effect sizes, a variable sample size and interim data analysis is permissible but with inspection points stated in advance, appropriate Type I error correction for ‘peeking’ employed, and a final stopping rule for data collection outlined. We appreciate in some areas (e.g. work with special populations) it may be difficult to obtain sufficient sample for very highly powered studies, or that sample size may depend on aspects of recruitment beyond experimenter control. In such cases, researchers should provide analyses of the level of power which they would achieve given their predicted sample size and should clearly lay out what factors will determine the sample size that they will finally obtain. Where lower power is likely in the final sample, the importance of conducting the study despite this likely lower power must be clearly articulated.
  • Methods involving Bayesian hypothesis testing are encouraged. For studies involving analyses with Bayes factors, the predictions of the theory must be specified so that a Bayes factor can be calculated. Authors should indicate what distribution will be used to represent the predictions of the theory and how its parameters will be specified. For further advice on Bayes factors or Bayesian sampling methods, prospective authors are encouraged to read this key article by Schönbrodt and Wagenmakers.
  • Explicit quality control checks are encouraged where possible. Such quality checks might include the absence of floor or ceiling effects in data distributions, positive controls, or other quality checks that are orthogonal to the experimental hypotheses.
  • Timeline for completion of the study and proposed resubmission date if Stage 1 review is successful. Extensions to this deadline can be negotiated with the action editor.
  • One deviation from the “traditional” manuscript format involves studies in which follow-up experiments rely on the outcome of prior ones. In this case the authors have two options.
    Option 1: Lay out the specific logic for follow-up experiments, carefully outlining which experiments will be performed under what research outcomes from prior experiments. A flow-chart format may be employed.
    Option 2: Authors can submit the first experiment as a stand-alone RR and pass subsequent follow-up experiments through the RR process (as successive Stage 1 submissions) as they go.
    
    Note that the above information can be placed in either the main Methods section or a Methods Addendum. We envisage that the Methods section will contain information standardly found in a methods section, with the addendum containing the additional material which would be necessary for an exact replication.
    
    
• Detailed Data Analysis Plan
  • Proposed analysis pipeline, including all preprocessing steps, and a precise description of all planned statistical analyses, including appropriate consideration of type 1 error inflations due to multiple comparisons. Any covariates or regressors must be stated. Where analysis decisions are contingent on the outcome of prior analyses, these contingencies must be specified and adhered to. Include details (where necessary/possible) about what will be done in the case that statistical models don’t converge. Any other cases of possible free parameters should be specified in advance as much as possible.
  • The plan should include information justifying the particular analysis plan.
• Potential results and implications section
  • Here the author outlines ALL possible results to the best of their ability and carefully interprets what each would say about the research question(s).
  • Dummy graphs for illustration purposes are encouraged.
• A timeline for completion of the research and Stage 2 manuscript
  
  
• Statement of Public Registration 
  
  • The submission must also include a statement confirming that, following Stage 1 in principle acceptance, the authors agree to register their approved protocol on the OSF or other recognised repository, either publicly or under private embargo until final acceptance of the Stage 2 manuscript. Accepted protocols can be quickly and easily registered using a tailored mechanism for Registered Reports on the Open Science Framework: https://osf.io/rr/ 
    

Authors are reminded that any deviation from the stated experimental procedures, regardless of how minor it may seem to the authors, could lead to rejection of the manuscript at Stage 2. In cases where the pre-registered protocol is altered after IPA due to unforeseen circumstances (e.g. change of equipment, methodological advances toward best practices or unanticipated technical error), the authors must consult the journal immediately for advice, and prior to the completion of data collection. Minor changes to the protocol may be permitted according to editorial discretion. In such cases, IPA would be preserved and the deviation reported in the Stage 2 submission. If the authors wish to alter the experimental procedures more substantially following IPA but still wish to publish their article as a Registered Report, then the manuscript must be withdrawn and resubmitted as a new Stage 1 submission. Note that in all cases registered analyses must be undertaken, but additional unregistered analyses can also be included in a final manuscript (see below).

Stage 2: Full manuscript submitted for review

Once the study is complete, authors prepare and resubmit their manuscript for full review, with the following additions:

• Submission of raw data and laboratory log
  • Raw data and any digital experimental materials (e.g. stimuli etc.) must be made freely available in a public repository such as OSF (unless not allowed by ethical approval, as will have been discussed with the editor). Other than pre-registered data, no data acquired prior to the date of IPA is admissible in the Stage 2 submission main analyses. Pilot data or exploratory analyses involving data collected prior to the IPA must be clearly marked. Raw data must be accompanied by guidance notes, where required, to assist other scientists in replicating the analysis pipeline. Authors are also expected to upload any relevant analysis scripts and other experimental materials that would assist in replication (e.g. stimuli & presentation code, analysis scripts). This material must be referenced in the text of the manuscript for publication (e.g. in a note or data statement at the end of the text), including a DOI link to the material.
  • Any supplementary figures, tables, or other text (such as supplementary methods) can either be included as standard supplementary information that accompanies the paper, or they can be archived together with the data. Please note that the raw data itself should be archived (see above) rather than submitted to the journal as supplementary material.
• The authors must collectively certify in the Stage 2 Cover Letter that all non-pilot data was collected after the date of IPA.
  
  
• Background, Rationale and Methods
  • Apart from minor stylistic revisions, the Introduction should not be altered from the approved Stage 1 submission, and the stated hypotheses cannot be amended or appended. At Stage 2, any description of the rationale or proposed methodology that was written in future tense within the Stage 1 manuscript should be changed to past tense. Any textual changes (barring typographical errors) to the Introduction or Methods must be clearly marked in the Stage 2 submission. Any relevant literature that appeared following the date of IPA maybe be included, but should be clearly marked and will require approval at Stage 2. If the new literature impacts the hypotheses in a substantive way, the new literature should be covered in the Discussion instead.
• Results and Discussion
  • The outcome of all registered analyses must be reported in the manuscript, except in rare instances where a registered and approved analysis is subsequently shown to be logically flawed or unfounded. In such cases, the authors, reviewers, and editor must agree that a collective error of judgment was made and that the analysis is inappropriate. In such cases the analysis would still be mentioned in the Methods but omitted with justification from the Results.
  • It is reasonable that authors may wish to include additional analyses that were not included in the registered submission. For instance, a new analytic approach might become available between IPA and Stage 2 review, or a particularly interesting and unexpected finding may emerge. Such analyses are admissible but must be clearly justified in the text, appropriately caveated, and reported in a separate section of the Results titled “Exploratory analyses”. Authors should be careful not to base their conclusions entirely on the outcome of statistically significant post hoc analyses.
  • Authors reporting null hypothesis significance tests are required to report exact p values and effect sizes for all inferential analyses.

The resubmission will most likely be considered by the same reviewers as in Stage 1, but could also be assessed by new reviewers.

Stage 1 public registration reference

The Stage 2 manuscript must contain the URL of the approved Stage 1 protocol on the Open Science Framework or other recognised repository. The cover letter should state the page number in the manuscript containing this information. If you choose to keep the protocol private until Stage 2 acceptance (and thus private during the Stage 2 review process) then please ensure that this URL is accessible to the reviewers. For protocols registered on the OSF, guidance is available on how to make a privately registered protocol accessible to reviewers without making it public: http://help.osf.io/m/links_forks/l/783581-create-a-view-only-link-for-a-registration

Manuscript withdrawal and Withdrawn Registrations

It is possible that authors with IPA may wish to withdraw their manuscript following or during data collection. Possible reasons could include major technical error, or an inability to complete the study due to other unforeseen circumstances. In all such cases, manuscripts can of course be withdrawn at the authors’ discretion. However, the journal will publicly record each case in that the original abstract will be published on a dedicated website. Partial withdrawals are not possible; i.e. authors cannot publish part of a registered study by selectively withdrawing one of the planned experiments. Such cases must lead to withdrawal of the entire paper. Studies that are not completed by the agreed Stage 2 submission deadline (which can be extended in negotiation with the editorial office) will be considered withdrawn and will be subject to a Withdrawn Registration.

Additional guidelines for review

Review criteria at Stage 1

In considering papers at Stage 1, you are asked to use similar criteria to regular manuscripts but with particular scrutiny on the details of the methodology and data analysis. In this scrutiny, you should keep in mind that accepting a study at Stage 1 is an in-principle commitment by the journal to publish regardless of the outcome of the analyses (including null findings) so it is important that the proposed methodology and data analysis add meaningfully to the literature regardless of outcome. Specifically, reviewers will be asked to assess:

1. Theoretical merit and relevance

• Is the question theoretically important and relevant? Does the study have the potential to tell us something interesting about child language?
• The logic, rationale, and plausibility of the proposed hypotheses.

2. Coverage of existing literature and writing

• Does the introduction do proper justice to the existing literature and tell a coherent story?
• Is the writing overall clear and appropriate to the JCL audience?

3. Soundness and feasibility of methodology and analysis pipeline

• Is the methodology sound and described in sufficient detail to exactly replicate the proposed experimental procedures and analysis pipeline?
• Are potential sources of bias described and sufficiently mitigated?
• Are appropriate quality control measures in place where possible?
• Are the analyses appropriate? Is it sufficiently powered? Is the timeline reasonable?

Following Stage 1 peer review, manuscripts will be accepted, offered the opportunity to revise, or rejected outright. As with a standard paper, multiple rounds of review are permitted at this stage assuming that progress continues to be made toward an acceptable submission and that the reviewers continue to believe that the theoretical merit and methodological rigor are sufficiently strong. Eventually manuscripts that pass peer review will be issued an in principle acceptance (IPA). This commits the journal to publish the outcome of that work pending successful completion of the study according to the pre-registered methods and analytic procedures according to the timeline provided (with reasonable allowances), and provided that any quality controls are met and provided that a defensible and evidence-based interpretation of the results is provided at Stage 2.

Review criteria at Stage 2

Following completion of the study, authors will complete the manuscript. The submitted manuscript will include the original Introduction and Methods sections and a new Results and Discussion sections (other sections approved at Stage 1 will appear in the final published manuscript as supplementary materials). These Stage 2 manuscripts will more closely resemble a regular article format. The manuscript will then be returned to the reviewers, who will be asked to appraise:

1. Consistency with Stage 1 approved submission and methodological rigor:

• Are the Introduction, rationale and stated hypotheses the same as the approved Stage 1 submission (required)?
• Introduction and Methods sections should be identical and therefore there is generally no need to re-evaluate, with the exception of minor changes which should be clearly marked. In some exceptional cases larger changes may be allowed, however in all such cases the authors must obtain permission from the editor and as a reviewer you may be asked to comment on the appropriateness of the changes.
• Is it clear that the authors adhered precisely to the registered experimental procedures?
• Do the outcomes of any quality control checks and/or positive controls outlined in Stage 1 support the interpretation of the findings?

2. Quality of results section

• Does this section precisely follow the procedures laid out in the detailed data analysis plan?
• Is the section clearly written with analyses clearly explained in a manner suitable for the JCL audience?
• Are any unregistered post hoc analyses added by the authors justified, methodologically sound, and informative?

3. Quality of Discussion section

• Is interpretation of the results evidence-based and are authors’ conclusions justified given the data?
• Is the writing clear and logical and suitable for the JCL audience?
• Note that criteria relevant for evaluating discussion sections of a standard article remain relevant, i.e. results should be critically interpreted in light of existing literature, discrepancies evaluated, and future plans discussed.

We emphasize that decisions at Stage 2 should not be based on the perceived importance, novelty or conclusiveness of the results. Thus while reviewers are free to enter such comments on the record, they will not influence editorial decisions. Please avoid proposing additional post hoc tests at Stage 2 unless they are necessary to satisfy one of the Stage 2 criteria described above. In general, suggestions for analyses should be made at Stage 1.

Last updated: 1 September 2018