The FAA-prescribed practical test is the final hurdle every pilot must pass to earn a new certificate or rating, and the practical test standards (PTS) are where the requirements for success are spelled out. The PTS is what both aspiring pilots and examiners use to determine what’s to be done on a practical test and how. It’s supposed to be the final assessment of whether a pilot can conduct safe flight operations.
Yet, there are serious flaws with how the PTS is structured and administered that may compromise its effectiveness for this purpose. Thankfully, some major changes may be in the offing. Until they arrive, you can benefit from understanding their limitations in testing pilots and apply that knowledge to your everyday flying.
A Little History
The practical test for pilot certificates and ratings has been with us forever—or at least since the 1920s, when Congress passed the Air Commerce Act and the FAA’s predecessor agencies first established requirements for airmen certification and certificates. The PTS, however, are a more recent innovation.
The current PTS system was established in the 1980s to replace the previous flight test guides. These guides were very general and vague on acceptable standards, allowing the examiner considerable discretion in how the practical test was administered and deciding whether the applicant met requirements. The FAA correctly decided a more objective testing system was needed and the PTS—along with its more-specific description of the tasks to be demonstrated and objective measures of minimum performance—was one result.
288
The PTS system met a hostile reception from a few designated pilot examiners—those to whom the FAA delegates the examiner function—who considered it an affront to their dignity and ability to assess an applicant. At the time of transition, I was an aviation safety inspector at a Flight Standards District Office and was charged with explaining the PTS to the examiners I supervised. Most of them cooperated and saw the intent of the new system but a couple of the older ones grumbled about it and called for a return to the “good old days.” When I noted those would have been the days when the fatal accident rate was much worse, I was met with either a blank stare or worse.
Since their inception, the PTS have been progressively revised to incorporate requirements for testing a pilot’s single pilot resource management (SRM) skills, such as risk management. The instrument rating PTS was revised in 2010 to incorporate these requirements. The private and commercial pilot PTS were similarly revised in November 2011, to become effective June 1, 2012. Other PTS will be similarly changed during their revision cycles.
You would expect me to give a rousing cheer and declare victory at this point. Indeed, the revised PTS are a big improvement over their predecessors. I must, however, report the job is not complete. The practical test, as currently constructed, provides a poor “scenario” platform. In addition, the new requirements are treated differently than other PTS tasks, which may limit the effectiveness of how applicants are tested for these crucial skills.
Poor Scenario
Some leaders in the aviation safety community have expressed the opinion that the practical test standards were already adequate before these changes, without any additional “tinkering.” For example, a high official in one safety organization stated in another publication’s 2011 article, “If all pilots merely flew to the current private pilot PTS just as they are written today, the number of accidents would plummet—no need for further tinkering.” He acknowledges, “The lack-of-skill accidents usually don’t result in fatalities or even serious injuries. It’s poor judgment or decision making that causes the bad ones,” but goes on to state that, “Testing someone’s everyday judgment on a check ride just isn’t realistic.”
288
With all due respect, I disagree with this assessment. We don’t need to evaluate an applicant’s judgment on the practical test, but rather their ability to identify, assess and mitigate risks. The difference between the two functions is not subtle. Judgment is a subjective skill and arguably is enhanced by a person’s attitudes and experience. Risk management, on the other hand, is a skill with specific elements that can be performed and measured on an appropriately designed practical level.
If you’re a long-time reader of Aviation Safety, you’re familiar with my often-cited assertion that poor risk management is at the root cause of most general aviation fatal accidents. In numerous samples and analyses, the data I have examined in accident reports shows that about 75 percent of fatal accidents result from the pilot’s failure to identify, assess, and mitigate known risks.
The problem with the current PTS, even as recently revised, is the “scenario” under which each practical test is supposed to be conducted does not realistically test an applicant’s ability to perform effective risk management. Picture the situation as it occurs on a typical practical test for the private pilot certificate.
The applicant endeavors to schedule the test on a perfect day so weather will not be an issue. The examiner will look at the applicant’s preflight planning, including his/her analysis of the weather. On a typical practical test when the prevailing weather is good, the applicant’s ability to identify, assess and mitigate weather risks—leading factors in fatal accidents, thanks to poor risk management—really cannot be tested effectively. The applicant will not be able to demonstrate effective risk mitigation, such as planning an alternate route, when there are no weather risks to mitigate.
288
Furthermore, the PTS speaks to creating a “scenario” for the practical test, but its purpose is not to present a challenging typical situation that the applicant must evaluate for risk, but rather to improve the efficiency of the test in terms of evaluating all other required tasks. For example, the PTS requires an examiner to create a “plan of action” for each practical test. The private pilot PTS states “the plan of action will include a scenario that allows the evaluation of as many required Areas of Operation and Tasks as possible without disruption.” There is nothing inherently wrong with this goal, but it tends to compromise the ability of the practical test to assess an applicant’s risk management proficiency under realistic conditions.
Special Emphasis Areas
You are probably wondering about the recent changes to the PTS that introduce scenarios and risk management tasks and objectives. Don’t they assure the examiner will adequately assess an applicant’s risk management ability and other high-order skills? Maybe, but I think the jury is still out on this one. I have three concerns.
First, the new scenario-based and risk management tasks and objectives are in the front material of the PTS and are listed as “special emphasis” items. My experience within the FAA is that “special emphasis” items rarely receive special emphasis. Or, if they do, it’s to check off some box to satisfy headquarters that the issues were addressed somehow and then it’s back to business as usual. The FAA could more easily ensure their effectiveness if examiner standardization training was changed to emphasize their importance. Perhaps this will take place, or has already.
My second concern is more fundamental than examiner standardization. The scenario and risk-management tasks would be more effective if they were integrated at the area of operation, task and objective level throughout the PTS. For example, in the Cross Country Flight Planning task in the private pilot PTS, there are nine objectives—none of them emphasize risk management. Wouldn’t it be important for the applicant to demonstrate he/she can complete a risk analysis for the proposed flight as a part of the preflight planning task, given that 75 percent of fatal accidents have a root cause related to poor risk management?
Yet, in this day of iPads, integrated avionics and smart cockpits, we still insist on seeing the applicant complete a flight log the old-fashioned way, just as Lindbergh did it. I’m all for determining that applicants demonstrate backup navigation skills and an understanding of fundamental concepts such as time-speed-distance relationships.
However, I believe it would be a more effective evaluation if the examiner asked to see the applicant’s risk analysis rather than a paper or even electronic flight log with a bunch of computations and artificial checkpoints. There are better ways and methods to ensure the applicant has planned the route, calculated fuel requirements and performed other tasks related to flight planning. Now, if you’re ferrying a no-radio airplane across an ocean, I might be more interested in your dead-reckoning calculations.
My concern in this area extends even to sport pilot applicants. Even in the simplest of airplanes for local flights, risk management is as important as stick and rudder skills. On that practical test, for example, we could emphasize the importance of the crosswind component chart as a tool to manage risks on windy days. Even the simplest of missions requires a risk analysis of some sort. It’s the only way we will ever change the general aviation safety culture.
Speaking of which, I mentioned that I had three concerns regarding the special emphasis areas and the general aviation safety culture is at the center of this concern. As a community, we are having trouble with this discussion because some of the safety leaders frame it by saying we would have to impose airline-like controls to affect the accident rate and assert that we would have to give up too much freedom to do it differently. The individual I mentioned earlier even stated, in the previously cited article, that it’s more desirable to “accept gradual improvements and tolerate individual lapses as a cost of doing business.” I can only point out that such “individual lapses” all too often result in fatal accidents, and that one of the “costs of doing business” is a safety record that discourages a latent market of new entrants not subscribing to this safety philosophy.
Needless to say, I disagree we would have to give up too much freedom to change our safety culture. Positive change can occur through a variety of ways, starting with how we train and test new pilots, rather than by imposing airline-like controls.
Make Each Flight A Checkride
Even though you may already have a pilot certificate and are not contemplating adding ratings requiring you to take another practical test, you may still apply lessons regarding the practical test process to your current flying activities. These are just some sample suggestions of how you might approach each flight as a realistic practical test of your ability to accomplish the “mission” while minimizing risk:
• Perform at least an informal risk analysis for every flight, to identify, assess and mitigate the identified hazards. There are flight-risk assessment tools available to accomplish this. I will review these in a future article.
• To mitigate risks, use personal minimums to provide margins for certain risks. For example, you might want to add to minimum ceiling and visibility levels for instrument approaches to compensate for your lack of recent instrument currency—the FAA requirements aren’t enough. Add even more margins if you’re also operating without a working autopilot.
• If you’re a savvy pilot, you probably ditched the Lindbergh-era flight log after you passed your private pilot checkride. However, to help manage risk, you still need to do supporting calculations to determine time en route, fuel requirements, runway requirements and weight and balance. A real pro can do these quickly in his/her head without spinning a prayer wheel or using an electronic equivalent. Your advanced cockpit or portable navigator (if you have either) can help with this.
• Remember that risk management—and especially risk mitigation—is all about layers of defenses no matter what kind of aircraft you are flying. Always keep a Plan B handy (or even a Plan C on really marginal days and missions). If you’re flying a no-radio J-3 Cub cross-country and use a portable GPS, for example, remember that your backup navigation “system” requires you to keep your thumb on the sectional at all times.
