Thank you and congratulations for accepting the request to review a JFP submission or revised submission. Your work will improve the functional programming community, and it will provide you with an opportunity to broaden your horizon.
For notes on the revision cycle, see further down.
Initial submission
If this is your first journal review (or if you have never received any advice on how to review journal submissions), here are some basic points.
1. The editor chose you for your expertise and your relationship to the "research neighbourhood" of the author(s). You know the area either as an insider (e.g., a member of the Haskell research community) or as a well-informed outsider (e.g., someone who conducts research on static analysis and is aware of Haskell's results in this direction). And the editor wishes to get your judgment from your perspective.
2. Your task is to ensure that the submission satisfies a basic level of quality. The submission ought to make a significant contribution that is interesting beyond the neighbourhood of the authors; the results "feel" correct; the presentation is readable and accessible to a well-trained PhD student outside the neighbourhood of the authors; and the authors have put in the care and polish that suggests they wish to give their best.
3. 3. Read the submission, reflect, and then write a report to the editor. Concisely summarize your understanding of the submission's content, and use it to articulate your "big picture" evaluation. For perfect submission, you need a single paragraph for the latter; ordinarily, the evaluation must bring across how you think the submission lives up to the above quality standards.
If you have time, add a list of page-by-page observations or suggestions.
Use polite language. Avoid "you" or "the authors" in the report unless you really wish to address the authors directly.
Do __not__ include a final recommendation (accept, reject, etc) in the report. Use MC's radio box instead or use the "confidential comments to the editor" text box when you upload your review.
4. While the editor needs your judgment (and that of your colleagues), your report is also for the authors. The ideal report helps the authors improve the submission so that it is worthy to appear in JFP or in another journal, or it explains why the result is not worth improving.
5. Some answers to questions you might have now:
TIME: Spend enough time reading and reflecting to make an informed judgment. A journal does not suffer from the time constraints of typical of conferences. But if you consider the submission a boring result or a bad match, spend just enough time to explain why in your review.
EFFORT: Write well to make sure you get your insights across. Set an example with your report, especially if your judgment is negative.
CONFLICTS: Inform the editor of any ethical concerns immediately. This includes conflicts of interests, personal conflicts with an author, or overlap in research (you have been working on the same/closely related result).
CONFIDENTIALITY: The paper has has been submitted to JFP in confidence. Please do not otherwise cite the paper, or pass it on to others, except with the author's explicit agreement. You are welcome to consult your colleagues about your review, or invite a suitably-qualified colleague to do the review, but please remind them of the restrictions above.
MORE ADVICE: We highly recommend Ian Parberry's short paper ["A guide for new referees in theoretical computer science"](https://doi.org/10.1145/74074.74090), SIGACT News, 20(4):92-109. Sadly, the official version is behind a paywall; if you can't access it, just ask.
Revisions
A managing editor may choose to (1) accept a paper, (2) request revisions, or (3) reject a paper. MC will BCC you on the decision letter so that you find out what happened and have anonymous access to the write-ups of the other reviewers.
In case (2), the authors have a limited amount of time to improve the paper, usually via editing prose, occasionally via new research, e.g., fixing a proof, running additional benchmarks. Once the authors submit their revised version, MC assigns it the same number as before PLUS a suffix: R1 for the first revision, R2 for the second, and so on. By default, MC assigns the same managing editor and those reviewers who have declared their willingness to inspect the revision.
When you review a revision, you ought to read the entire paper again with a special focus on those spots that your review marked up as questionable before. The authors are expected to provide a covering letter explaining how the paper has been revised, so you should not have to work hard to discover this. Your review may simply state "this revision takes care of all my complaints" or you may point to previous complaints like this "the revised paper still does not address reviewer 2's point about theorem 42 and the benchmark suites in section 17." The decision letter for the original manuscript shows you how your review was listed.
You may also wish to inspect how the authors addressed the complaints of other reviewers. If you wish to point out problems, use the same neutral language as above: "the revised paper still does not address reviewer 1's point about the possibilities of unicorns ruining the cooling system of the benchmark machines."
Publication Ethics
The Journal of Functional Programming is committed to respect high standards of ethics in the editorial and reviewing process and adheres to the code of conduct for editors enacted by the Committee of Publication Ethics (COPE). The journal wishes to make sure that reviewers respect these standards. The guidelines on good publication practice for authorship can be found here. During the peer review process, reviewers will be asked to follow these guidelines and declare any potential conflicts of interest. For further information on publication ethics at Cambridge University Press please see here.