Image: An instructor pilot debriefing the air combat mission of student pilots using the critical decision method protocol. Credit: Finnish Air Force/Fighter Squadron 41

Success in air combat is heavily based on the quality of pilots’ tactical decisions. The quality of the decisions, on the other hand, is based on the accuracy and correctness of pilots’ static and dynamic knowledge, or situation awareness (SA). Therefore, gaining expertise in air combat is not as much about obtaining motor skills as it is about acquiring necessary domain knowledge and SA, as well as learning to exploit them in tactical decision making.

It is very human to evaluate the quality of decisions against their outcome: if the outcome is desirable, the decision – along with the knowledge and SA underlying it – must be good as well and vice versa. However, with luck, one can arrive at a good outcome even with incorrect knowledge and poor SA.

Similarly, even with correct knowledge and good SA, a rational decision can result in bad outcomes simply by chance. Therefore, unless the knowledge and SA underlying the decisions are evaluated, one may arrive at false conclusions when assessing the quality of decisions: an unlucky pilot’s decision may be considered bad and a lucky pilot’s decision may be seen as good. If feedback to student pilots is false, the learning effect can be negative. An unlucky pilot determined to be poor may disband their correct knowledge or mental models, and the lucky pilots’ potentially false mental models are further strengthened.

While the instructor pilots are trained to assess the student pilots’ decisions, we hypothesised that with the help of short training, their abilities in evaluation of the student pilots’ decisions could be further enhanced.

We first trained four assessors, who monitored the instructor pilots’ debriefs. The assessors logged the number of decision-making points identified during debriefs and the number of times the instructor pilots explicitly evaluated the student pilots’ knowledge and SA underlying their decisions.

Then, the instructor pilots were exposed to a short training intervention about the decision-making process, the role of knowledge and SA in decision making, and the use of the critical decision method (CDM) in eliciting the student pilots’ knowledge and SA.

After the training intervention, the instructor pilots were encouraged to utilise the CDM protocol in their debriefs as they evaluated the quality of the student pilots’ decisions. Forty-one debriefs before the training intervention and 56 debriefs after the training intervention were observed. The impact of the training intervention was evaluated using Kirkpatrick’s four-level model.

The first three levels of evaluation (Reactions, Learning and Behaviour) focused on the instructor pilots and their behaviour during debriefs, whereas the fourth level (Results) focused on the student pilots’ performance, ie, their grades of overall performance and SA in air combat training missions. The training intervention had a positive impact on the Reactions, Learning and debrief Behaviour of the instructor pilots, as they made more frequent efforts to elicit the student pilots’ knowledge and SA at identified decision points. 

A total of 372 air combat training mission were flown before the training intervention and 402 missions flown after the training intervention.  The student pilots whose debriefs were based on the CDM protocol, had superior SA and overall performance compared to those pilots whose training mission were flown before the instructor pilots’ training intervention and who were thus debriefed without the CDM protocol.

In air combat, even small improvements in performance and SA can provide a significant tactical advantage. Therefore, we think our paper, Improving Pilots’ Tactical Decisions in Air Combat Training using The Critical Decision Method can help future fighter pilots to gain the tactical edge they need.

Our study demonstrates that the student pilots’ performance can be improved by providing training to the instructors and giving them the appropriate skills. In this respect, the approach is quite novel as normally the success of the training is conducted as a direct evaluation of the people being trained – in this case, the flight instructors. The CDM training approach demonstrated in our paper can be applied to almost any domain where there is a sufficient debrief tradition and appropriate tools to augment memory recall needed in the CDM protocol.

Improving Pilots’ Tactical Decisions in Air Combat Training using The Critical Decision Method by H. Mansikka, et al. This open access paper appears in The Aeronautical Journal.

The Royal Aeronautical Society is the world’s only professional body dedicated to the entire aerospace community. Established in 1866 to further the art, science and engineering of aeronautics, the Society has been at the forefront of developments ever since.

National Aerospace Library catalogue & e-books

Lastly, book reviews covering academic, scientific and technical books covering aeronautical engineering and topics relating to it can be found here: www.aerosociety.com/news-expertise/national-aerospace-library/book-reviews