Thanks to the contents of the FAA’s new airman certification standards (ACS), which are replacing the practical test standards (PTS), most applicants for pilot certificates and ratings must now demonstrate that they can identify, assess and mitigate risk. Although the FAA and industry organizations have developed flight risk assessment tools (FRATs) to help pilots identify and manage risks, these tools often use a simplistic numerical scoring system that will produce a “go” decision when significant risk is still present. With a little more thought, analyzing risk can be more realistic and much more effective.
Setting A New Standard
The FAA began replacing the PTS with the ACS in 2016. Practical tests for private and commercial airplane certificates and instrument ratings must now be conducted in accordance with the ACS documents. Unlike the PTS, the ACS integrates specific aeronautical knowledge with proficiency in corresponding skills on the practical test. Applicants must now demonstrate identifying, assessing and mitigating risk on all practical test tasks and objectives. Applicants also must “show their work” and explain to the examiner how they performed this process. The examiner will expect applicants to demonstrate risk management proficiency throughout the practical test.
The problem with this process is that it supposes that applicants have been taught how to identify, assess and mitigate risk in the real world, as well as for the practical test. Lacking a significant change in the ways pilots are trained, very few applicants will have received such training and likely will be ill-prepared to demonstrate these skills to an examiner. I expect that there will be an extended transition period for the new ACS as instructors and examiners “wing it” while the FAA slowly develops new or revised guidance material on risk management.
Numerical FRAT Pitfalls
Recognizing the need for effective risk management among relatively inexperienced pilots, several organizations have developed numerical—or objective-based—FRATs. These organizations include the FAA, the AOPA Air Safety Institute and others. The FRATs available from the FAA and AOPA are described in greater detail in the sidebar on the opposite page. These FRATs, and others like them, are generally easy to use and definitely better than nothing. But they all have serious flaws in operational use.
The most critical flaw revolves around the go/no-go numerical or other threshold. You could have a near perfect low score which indicates it’s all right to launch or continue, but there might just be one item with a negative score, let’s say a line of embedded thunderstorms on your planned route. That one risk could have a high probability of occurrence with catastrophic consequences, and thus be a very high risk, if not effectively mitigated.
By their nature, FRATs consisting of a list of specific items and may not include all potential risks for a planned or ongoing flight. Yes, it’s almost impossible to list all potential flight hazards, since each flight is unique. But that’s the point: Unless you have risk categories that are all-inclusive, you may miss some that could critically affect your flight.
Further, any numerical-based FRAT could tempt the pilot to fudge the score by downgrading a risk to bring the final score to an acceptable level. Most pilots aren’t fools and will quickly see the lack of credibility and integrity in that approach.
But the key flaw in numerical FRATs is they encourage a lazy and hands-off approach to risk analysis. In other words, the pilot isn’t forced to consider all risk categories and fully analyze each of them to determine whether or not it requires mitigation.
Numerical Flight Risk Assessment Tools
The FAA Safety Team (FAAST) has developed a numerical FRAT in the form of an Excel spreadsheet. It can be downloaded at tinyurl.com/avsafe_frat and seems to have been adapted from one in the agency’sRisk Management Handbook(FAA-H-8083-2). This FRAT has several of the flaws of numerical FRATs identified in this article’s main text. For example, it makes no mention of convective activity, a serious potential hazard. It’s also possible to produce a low risk rating for an inexperienced pilot in mountainous terrain with winds blowing at 35 knots, a potentially very risky situation. On the other hand, it’s possible for this FRAT to give a no-go answer when it’s actually a low-risk flight.
Using the FAA FRAT, I imagined a VFR flight from my home airport in Bellingham, Wash., to Spokane in a Mooney, with clear weather at both ends and a line of embedded thunderstorms on the other side of the Cascade Mountains. The tool urged caution regarding both airports because they were in designated mountainous terrain, but unfortunately did not have a box to help flag the thunderstorms as a hazard.
This is the most important defect in these kinds of tools. Any approach that just requires you to check a box, without any analysis, is likely to miss critical hazards and risks. In any case, for the hypothetical flight conditions I chose, the tool only wanted me to fuss about the origin and destination airports and would not have highlighted the most critical risks on this imaginary flight that could result from attempted penetration of a line of thunderstorms.
Risk management: The silent killer
Many of my articles forAviation Safetycite poor risk management as a root cause of most general aviation fatal accidents. This rarely appears in NTSB accident reports, which most likely will cite the final event in the accident chain—such as loss of control in flight (LOC-I)—in the probable cause finding.
My own analysis of general aviation fatal accidents validates the prominence of poor risk management. I’ve classified poor risk management as a root cause of accidents ranging from between 67 percent (for a sample of Beech Bonanza accidents) to a high of 86 percent (for a sample of Cessna 172 fatals). Poor risk management was a root cause of nearly 50 percent of the fatal business aircraft accidents I analyzed.
As pilots, we encounter risks on all flights and significant risks on most of them. It is essential that we use an effective method to identify, assess and mitigate these risks. Generations of pilots, however, have not been taught how to perform risk management and consequently it hasbecome the “silent killer” in most general aviation fatal accidents.
A More Analytic Approach?
So now what? If a numerical or objective-based FRAT doesn’t adequately evaluate risk, what tool or procedure should a pilot use to evaluate risk on all flights? One answer might be to use a tool or procedure that compensates for all the flaws I already identified for numerically-based FRATS. Specifically, an effective FRAT should accomplish the following objectives.
The identified threats and go/no-go decision should not be based on a numerical score. Rather, the hazards identified should result from a comprehensive analysis of all potential threat areas and how they will specifically affect the planned or ongoing flight.
An effective FRAT should not have a finite list of standard hazards. Rather, it should be based on a broad analysis of all potential threat categories. You might consider using the commonly used acronym PAVE (for Pilot, Aircraft, enVironment and External Pressures) to identify threats and serve as the “buckets” for capturing risks.
An effective FRAT also should not allow a pilot to fudge a score without assessing risk likelihood and severity. It also should allow the pilot to specify mitigating actions that will reduce its likelihood and/or severity.
An effective FRAT should not permit false positives or negatives. That is, it should clearly and accurately show the actual hazards and risks, their likelihood and severity, and how the mitigation actions will lower them.
Finally, an effective FRAT should prevent pilots from becoming lazy about risk by merely checking off boxes and depending on a single number to make a go/no-go decision. Rather, it should lead a pilot through an intuitive process that accurately identifies, assesses, and mitigates risk.
A Better FRAT
As it turns out (drum roll), there is a FRAT that doesn’t use a “check the box” approach to risk management. It’s part of the document Risk Management Guide for Single-Pilot Light Business Aircraft and is located on the National Business Aviation Association (NBAA) web site: tinyurl.com/avsafe-nbaafrat.
This guide, and the FRAT it contains, was developed by the NBAA Safety Committee’s Single-Pilot Work Group (SPWG) to address the fatal accident rate of single-pilot business aircraft, which is higher than that of aircraft flown by professional two-pilot crews. By way of full disclosure, I’ll admit to having more than a casual role in producing this guide and FRAT. It was introduced at the Single-Pilot Safety Stand Down on October 31, 2016, just preceding that year’s NBAA Convention.
This guide is essentially a short course in risk management and provides a case study illustrating how to use the accompanying FRAT. I’ll repeat the caution in the guide that it should not be used in lieu of more complete risk management training. However, it is unique among FRATs in that it does not employ a numerical-score approach to risk analysis.
The FRAT in the guide is a single-page two-sided form that uses the PAVE method of risk identification, a risk management matrix to assess risk likelihood and severity, and a mitigation approach that requires the user to list concrete mitigations that reduce risk likelihood and/or severity. The case study that illustrates how to use the FRAT also has a “school solution,” although the guide emphasizes that there could be an infinite number of other solutions, depending on how the pilot tackles the specific risks in the case study.
Besides the four main risk categories represented by the PAVE acronym, there are 10 “sub-buckets” in the tool that help identify risk and that attempt to cover all the bases of typical risks of operating single-pilot high-performance aircraft. In terms of risk assessment, it uses the common four categories of overall risk: high (red), serious (yellow), medium (green) and low (white). The FRAT shows the expected pilot actions for each of these risk levels.
This FRAT is certainly not the full solution to all risk management challenges, but it provides a practical template for evaluating risk. Moreover, it forces the pilot to think his or her way through the problem and evaluate the risk with more finesse than just checking off a box.
Managing A Risky Flight
You’re probably wondering if pilots will really use such a tool in their everyday flight operations. Your skepticism may be warranted, but pilots are more likely to use a tool that manages risk effectively than one that may leave them hanging because it didn’t identify all the risks to begin with. Also, once a pilot has learned the risk management process, it begins to become intuitive and physical reference to the FRAT is not required.
I’ll close by repeating the guide’s suggestions about how to put it all together and become a better risk manager:
• Complete full risk management training. Take an online course in risk management or single-pilot resource management (SRM). A good SRM course is available atwww.trainingport.net(Note: Full disclosure again—I helped develop this course).
• Complete some exercises. Begin with the case study in the NBAA guide. Then do some “table top” hypothetical flights that you typically make to see how the common risks you normally face can be identified.
• Use the sample FRAT in the NBAA guide. You might complete the FRAT in full for your first five flights or so.
• Plan far ahead. Remember that risk management must often begin days before your planned flight and you may need to make alternate arrangements if you identify risks that can’t be resolved by the day of the planned flight.
• “Think” risk management. You’ll quickly find that the risk management process will become intuitive and you won’t need to complete the FRAT for many flights.
Robert Wright is a former FAA executive and President of Wright Aviation Solutions LLC. He is also a 9800-hour ATP with four jet type ratings, and he holds a Flight Instructor Certificate. His opinions in this article do not necessarily represent those of clients or other organizations that he represents.
