Missing The Sixth Element

Assessing Crew Decision-Making takes time
and time is what Approved Check Pilots don't have

Jean LaRoche, FRAeS
Director of Research & Development
CQFA - Aviation College

Excerpt from TSB Accident Investigation Report A19W0015, 30 January 2019¹

(26 August 2020) Decision-making is defined as the thought process of selecting a logical choice from available options, projecting the consequences into the future and assessing possible collateral damage. It is a fundamental skill related to intelligence and autonomy. In aviation, decision-making often includes an interpersonal relations dimension. Not only does the pilot have to be able to weigh the pros and cons of several options before deciding on one, he/she needs the skills to effectively communicate the whole process to the other crewmember. This in turn must be paralleled by a sophisticated ability to integrate the other pilot’s processes into the mix. In complex-system operations such as aviation, crew decision-making is a demanding process. Putting the whole operation on hold long enough to reach a consensus opinion seems to be next to impossible.

Regulatory and procedural frameworks are provided to pilots to facilitate the recognition-primed decisions² (RPD) pilots make on a daily basis. RPDs are quick decisions such as going around when traffic blocks the landing runway. Certainly, the most famous example of RPD is Captain Chesley Sullenberger's call for ditching US Airways flight 1549 in the Hudson River in 2009. This article addresses the long decision-making process required when pilots face complex and nuanced situations that reduce the margins of safety. It focuses essentially on non-LOFT operations.

In 2017, the Canadian Minister of Transport leaped forward in the assessment of commercial pilots' competency by modernizing the six elements Approved Check Pilots (ACPs) must measure during the Pilot Proficiency Check (PPC)³. Along with the two technical elements 1) Aircraft Handling and 2) Tehcnical Skills and Knowledge, four new non-technical elements were introduced in a remarkable effort to measure the overall efficiency of the carrier's training programs to teach the crew 3) how to cooperate, 4) how to manage the flight deck, 5) how to build and maintain high levels of situational awareness and 6) how to make decisions. Twenty-five times⁴ during PPC rides pilots are evaluated in real time against a four-point marking matrix designed to record technical errors and non-technical deviances.

The actual process of running six elements on two pilots performing 25 in-flight exercises in real time requires a solid background in theory from ACPs. This knowledge base includes not only the company's Standard Operating Procedures (SOP) and the Flight Test Guide⁵, but what constitutes the foundation of good cooperation in the cockpit. Good leadership and managerial skills, how to maintain situational awareness at optimal levels and a solid knowledge of the five steps of safe and efficient decision-making round out the required body of theoretical knowledge. For the 30 years I have been training Canadian ACPs I have advised it will take up to two full years to feel comfortable in the role, due in part to the breadth of knowledge required. This learning curve is echoed by the ACPs themselves when they show up on recurrent courses every three years.

When ACPs observe a non-technical deviance during an exercise leading to a sub-standard mark for that phase of flight, a comment must accompany the score on the PPC Report Form. This comment documents the non-technical element(s) contributing to the weak performance. Statistics from the Flight Training and Aviation Education (FTAE)⁶ database involving the CAR 705 Canadian Air Carriers show non-RPD Decision-Making mentioned only twice for having genuinely contributed to sub-standard marks in 200,325 exercises evaluated in 2019-20⁷. The most commonly cited non-technical element is Poor Situational Awareness.

Over the years, more mandatory exercises found their way into the long list of PPC exercises, thus busying the traditional four-hour simulator booking. The speed with which ACPs must assess the crew's performance can certainly explain why the last element of the list is rarely evoked. But perhaps the most noteworthy reason why is the way PPC rides are scripted as concentrated successions of exercises with little time or even need, to actually make decisions. Decades of economic pressures, PPC Schedule I ⁸ growth and Operational Evaluation Board ⁹ compliance have driven the industry to produce efficient PPC scripts in order to fit ever more exercises into tight simulator bookings. Some operators even flirted with three and a half-hour simulator bookings in an effort to fit more sessions into each day.

Decision-Making Takes Time. So does measuring it.   

ACPs are directed to assess the element Decision-Making under five distinct sub-elements: 1) Problem Definition/Diagnosis, looking for personal-bias free information gathering and Crew Resource Management (CRM)-based review of the causal factors. 2) Option Generation, assessing how the crew worked together to develop alternatives based on all the available resources. 3) Risk Assessment, measuring how the crew projected the potential collateral damages on the remainder of the flight. 4) Option Selection, appraising how clearly the chosen option was stated and agreed on. 5) Outcome Review, validating that the crew compared the result with their expectations.

Research show four impediments to good decision-making in the cockpit: ambiguity of information; dynamically changing risks; goal conflicts (organisational or social pressures); and unanticipated consequences¹⁰. Time after time, accident reports show that when immersed in threat and/or error management, crew underestimated the risk associated with the aircraft state. Significantly, they persevered in a course of action in the face of evidence suggesting that the chosen course was inappropriate ¹¹.

When complex, nuanced threats are introduced to the crew, ACPs have great opportunities to effectively measure the sixth element. The more complex the scenario, the better the assessment and the C-A-L debriefing¹² that follows. That said, assessing crew decision-making takes time and time is what Approved Check Pilots don't have. They control when the process begins, but have to wait until the crew implements a consensual option, marking the end of the procedure. As the process runs into disruptions and interruptions, decision-making is not only variable in length, it might never reach a conclusion. ACPs know that engaging the crew into a time-variable, five-step decision-making process could impede PPC completion in the allocated time.

Introduced in September 2019 as an aftermath to Air France 447¹³, mandatory annual Surprise and Startle Effect Training (SSET) further emphasizes the need for pilots to slow down when facing complex threats to the operation. Handling procedures more deliberately – and slowing them down -- allows pilots to recognize the loss and to rebuild situational awareness and control. The key objective is to train pilots to do the strict minimum while they gather enough dynamic information before making any decisions that will impact the safety of the flight. This needed improvement in crew dynamics further slows down the actual process of making decisions in the cockpit.

Despite the shortcomings of the actual, perhaps outdated PPC program, its overarching objective remains to assess the quality of the company training program. Valid, representative and comprehensive PPC rides enable the Chief Pilot to steer the training pilots where they are most needed. But quality training is intrinsic to the program’s effectiveness and success, which also applies to LOFTs.

Empowering the training pilots

Unlike training schemas where elite Olympic athletes align peak performances with competition days, PPC rides are often seen as just another step in the pilots' learning curve. With time, this approach infuses a detached emotional response from trainers when substandard performances are observed during training sessions. "The pilots will perform better tomorrow." In some departments, training is a succession of tomorrows and the ride itself becomes just another tomorrow. "The pilots will learn on the line." The systemic deviance is progressively normalized.

Training decision-making and measuring its effectiveness is better achieved when performed several times on the same crew during different training sessions. It requires competent professional training pilots, well versed with the powerful cognitive biases affecting decision-making. Without exception, all training sessions lend themselves to this. Some operators have already begun offloading training lessons to give more discretionary time to training pilots in order to detect non-technical deviances, and, at last, to react before the deviances are normalized.

In 2018, Montreal-Based Air Transat streamlined Cross Crew Qualification (CCQ) courses and is planning a similar remodeling for new-hire courses with emphasis on the four non-technical elements. New resources are being added to better train and support training pilots, acknowledging a clear return on investment of every dollar invested in training.

Fast-growing Nolinor, based in Mirabel, also recalibrated its recurrent training program on a 3-year cycle to provide more discretionary time to training pilots. Using PPC, Loft and Flight Data Monitoring statistics, Nolinor reinforced the feedback loop with the training department in a systemic effort to address problematic issues before they become normalized.

Since 2015, Air Inuit has made a full ACP course mandatory for all training pilots, knowing that some could move to major airlines before becoming ACPs. "Not only does the training quality meet the best test standards but it tends to diffuse more efficiently during regular line operations. Air Inuit feels the ROI is well worth the effort."¹⁴

Based in Saint-Hubert, QC, 703 operator Max Aviation overhauled the company training program and made FTD use mandatory, from new recruit hiring to line training. The use of FTD was also made mandatory for SMS-related remedial training when warranted. "Even during pilot-shortage situations, we prefer terminating unsuccessful training attempts [rather] than allowing pilots to fly on the line with substandard proficiency".¹⁵

Empowered training pilots build commitment towards their role in a company. When properly supported by management, they don't disengage easily. They become risk-averse, willing to invest in problem-solving at their level. They readily deploy company and personal training strategies to actively transfer knowledge and validate the learning on a continuing basis. Empowerment tends to make training positions more attractive to pilots motivated by the desire to train, educate and elevate skillsets, rather than the desire to fly less, stay home more often or exert greater control over monthly schedules.

Affording the changes

One of the first questions that come to mind is "How much would Training 2.0 cost?"  Perhaps a more pertinent question should be "Can we really afford not to do it?"  "Many accidents occur as the result of repeated, persistent hazards and risks. These hazards and risks have been the subject of multiple reviews and recommendations calling for mitigation measures, and yet they continue to lead to accidents.^"16 The costs of haphazard pilot training are not limited to those resulting from bad judgment and mismanagement of critical operations. Recommending pilots for PPC ride who failed to reach criterion performance levels after incomplete, or worse, complacent training exposes the carrier to the ballistic costs associated with a fatal hull loss, notwithstanding intangible human and reputation-related costs.

More stakeholders need to facilitate extended training programs, beginning with the Regulator allowing operators to use affordable flight training devices, locally installed and readily available, with full training credits. Freeing the air carriers from the pseudo-scientific lobby for costly Full Flight Simulators for LOFT-like sessions would certainly pave the way for new processes. Certifying training pilots, especially CRM facilitators, as originally intended in AC 700-042¹⁷ would also serve the cause.

Never in the history of aviation has the cost of human error has been so high. The Canadian industry is doing fantastically well. But better training will mean even fewer reportable events, close calls and bad press.

Other languages Другие языки Autres langues 其他语言

1. Transport Safety Board of Canada. Accident Investigation Report A19W0015. 30 January 2019. Loss of control and collision with terrain. 1.1 History of the flight (abbreviated).
2. Klein, Gary, Calderwood, R., & Clinton-Cirocco, A. (2010). Rapid Decision Making on the Fire Ground: The Original Study Plus a Postscript. Journal of Cognitive Engineering and Decision Making, 4(3), 186 -209. doi.org/10.1518/155534310X12844000801203.
3. Approved Check Pilot Manual, TP6533, 10th Edition, 5.0 Principles of Evaluation.
4. On average, per pilot. The traditional airline-style PPC ride involves two pilots assessed during a 4-hour simulator session.
5. Pilot Proficiency Check and Aircraft Type Rating, TP14727, Rev 1.
6. Flight Training and Aviation Education, a Transport Canada computer system designed to utilize data compiled from flight test reports and written examination answer sheets
7. Transport Canada, Flight Training Standard. Non-AQP CAR 705 Canadian Air Carriers. From 2019-08-18 until 2020-08-18 period, 7793 successful and 220 failed PPCs were conducted. Of the total 200 325 exercises, 2495 received a score of 2 or 1.
8. Commercial Air Services. Standard 725. Airline Operations - Aeroplanes. 725.106 Pilot Qualifications.
9. Transport Canada. Operational Evaluation Board (OEB) Reports.
10. J. Orasanu and L. Martin, Errors in Aviation Decision Making: A Factor in Accidents and Incidents, presented at the 2nd Workshop on Human Error, Safety and Systems Development, April 1-2, 1998, Seattle, Washington, United States.
11. Transportation Safety Board of Canada. Aviation Investigation Report A05H0002, Runway Overrun. 16 October 2007
12. CRM-Analysis-Line Oriented debriefing technique centered on the pilots' own appreciation of their performance.
13. Air France 447 Final Report. Published July 2012 www.bea.aero
14. Private conversation with Capt. Sebastien Michel, Director of Flight Operations, Air Inuit Ltd.
15. Private conversation with Capt. Daniel Adams, VP Flight Operations, Max Aviation Ltd.
16. Transportation Safety Board of Canada. Air Transportation Safety Issue Investigation Report A15H0001. 703 Air-Taxi Operations in Canada: 1.4 The human and economic costs of accidents. 7 November 2019.
17. Transport Canada, Advisory Circular No. 700-042. Crew Resource Management (CRM). Issue No. 2, March 2020

Jean LaRoche has been training Canadian Approved Check Pilots, Training Captains, and Inspectors for 30 years. He is a Fellow of the Royal Aeronautical Society for his work in Human Factors and the AQTA 2019 Fecteau Award. He coauthored the WOMBAT psychological test. He can be reached by email at jlaroche@cqfa.ca From the same author: Pilot's Restricted Right To Medical Confidentiality  Cannabis and Pilots Sure Win, Risky Loss  Decision-Making: Missing The Sixth Element Unstable Approaches: Training the 97% The Pseudoscience of Flight Simulation Motion		Founded in 1968, CQFA is the national aeronautics institute of Quebec's Ministry for Education, Higher Education, Research and Science. Since more than 30 years, CQFA's Continuing Education Department, located at Montreal-Trudeau International Airport, serves the aviation industry in Canada and abroad. www.cqfa.ca