Dissecting The NTSBs Glass-Panel Study

In March, the National Transportation Safety Board (NTSB) released a brief statement stating the results of a study it conducted into the relative safety of general aviation airplanes equipped with glass-panel technologies. As we reported in our April issue, the NTSB concluded the last decades glass-cockpit revolution has not resulted in enhanced safety for the period and aircraft it studied. It came to such a conclusion, in part, because it found single-engine airplanes manufactured

between 2002 and 2006 and equipped with glass cockpits had a higher fatal accident rate than similar aircraft with conventional instruments.

At that time, very little else was known about the study, including the specific aircraft models involved, its methodology and other metrics, raising many questions. Since our April issue was finalized, the NTSB has released the full 91-page study results, including a separate seven-page letter detailing a series of recommendations to the FAA. These documents may be retrieved from the NTSB Web site using links published in the box in the upper right corner of the opposite page.

Meanwhile, what else did the NTSB say about personal airplanes with glass panels? What about the industry reaction? And what specific recommendations did the Board make?

The NTSB said it used three different methodologies to conduct its study. First, it performed a retrospective statistical analysis of manufacturer records, aircraft investigation information and activity survey data. This was done to compare the accident experience of recently manufactured piston-powered single-engine airplanes, sorted according to whether or not they were equipped with glass cockpit displays.

Second, it evaluated glass-cockpit training requirements and resources. This was accomplished to characterize the training and identify areas for potential safety improvement. Finally, specific accident cases were reviewed to identify emerging safety issues associated with the introduction of glass cockpit displays into this class of aircraft.

In its most eye-opening findings, the NTSB said its statistical analysis found that for 2002-2008, light single-engine aircraft equipped with glass cockpit displays experienced higher fatal accident rates-although accompanied by lower total accident rates-than the same type of aircraft equipped with conventional analog instrumentation.

Unsurprisingly, the Board said accidents involving glass-cockpit aircraft were more likely to be associated with personal/business flights, longer flights, instrument flight plans and single-pilot operations when compared to their conventional, “steam-gauge” counterparts. Those aircraft were more likely to be associated with instructional flights, shorter flights and two-pilot operations. Again, theres really nothing surprising there to us; the additional investment a customer makes in a glass panel usually is made because he or she plans to use the aircraft as a traveling machine.

Meanwhile, and putting aside the NTSBs finding that glass-panel aircraft experienced higher fatal accident rates than their steam-gauge counterparts, the study also concluded “accident pilots flying glass-cockpit equipped aircraft were found to have higher levels of pilot certification and more total flight experience than those flying conventional aircraft.”

In other words, the pilots more likely to be involved in an accident with one of the glass-panel aircraft studied generally enjoyed greater experience than those bending aircraft with conventional panels. To us, thats a surprising result, and one well explore more fully in a future article.

Finally, the NTSB found that, despite updated training handbooks and test standards incorporating generic information about electronic displays, the FAAs current written exams “do not assess pilots knowledge of the functionality of glass cockpit displays.” But this is kind of a “chicken-or-egg” situation since, as the NTSB states, the FAA has no specific training requirements for pilots operating glass cockpit-equipped piston-powered aircraft. Before testing applicants on their glass panel knowledge, the FAA needs to develop its own standards for what that knowledge should entail. So far-at least when considering aircraft for which a type rating is not required-no such standards exist.

However, the FAA-along with several industry organizations-has developed a series of recommended training materials addressing glass panels under a program designed to specifically consider the educational and operational challenges posed by what it calls “technically advanced aircraft” (TAA). As such, and while the FAA has not put into place specific requirements, a large quantity of guidelines and recommendations from the FAA do exist. As the NTSB noted, the lack of training requirements results in a “wide range of initial and recurrent training experiences among pilots of glass cockpit aircraft. With the exception of training provided by airframe manufacturers with the purchase of a new aircraft, pilots must currently seek out and obtain equipment-specific glass cockpit training on their own.”

But seeking out and obtaining specific training is common in the industry, especially when considering the wide variety of aircraft types being operated (i.e., Beech Bonanza vs. Cessna Centurion) and their different systems, powerplants and flight characteristics. Type-specific training isnt required by the FAA unless the aircraft is a turbojet or large airplane, yet conscientious operators seek out initial and recurrent training designed to maximize their familiarity with discrete aircraft models. The same thing occurs when considering glass panels. The issue is whether this training should be required.

Among industry groups publicly responding to the NTSB glass-panel study were the General Aviation Manufacturers Association (GAMA) and AOPA. According to GAMA, “glass cockpit avionics greatly enhance situational awareness and when operated by a properly trained pilot also greatly enhance the safety of flight for a specific flight mission.”

The association continued: “The NTSB study methodology compares accident rates for 2006 and 2007 between glass cockpit avionics equipped aircraft and conventional cockpits and identifies differences in how these aircraft are used by pilots. The study, however, does not determine whether cockpit avionics onboard the aircraft was a factor in either accident cohort and does not necessarily compare accidents in an apples-to-apples fashion; i.e., airplanes equipped or not equipped in similar operations. The study recommendations also do not consider the change in operations that has been stimulated by the introduction of glass cockpit avionics; i.e., more longer flights resulting in more challenging terrain and weather.”

Among its other conclusions, GAMA expressed concern “that because of the study relies on an older data set; the data may not fully consider the effects of enhanced training and other factors on the safety impact of glass cockpit technology. It should be noted that the study does not identify any reliability issues with glass cockpit avionics.”

The AOPA echoed many of GAMAs concerns, noting the NTSBs effort “leaves unanswered questions” and “falls short.” Among other issues expressed by AOPA, the study “did not take into account the type of mission the aircraft were being used for, making it difficult for such a generalization to stand.” The association went on to note many industry organizations “already are following practices that fulfill the NTSBs recommendations.” In other words, and as GAMA stated, “The six recommendations provided by the NTSB are in line with work that has already been accomplished independently by industry or in cooperation with the FAA.”

On one level, the NTSB appears to have opened up a can of worms for glass-panel pilots, manufacturers of aircraft equipped with them, and for the FAA. On another level, there are several things in the NTSB study and its recommendations deserving our attention and reflection.

If nothing else, the NTSB study highlights how different glass-panel aircraft are from their conventional counterparts and how intimate knowledge of their capabilities and operation is critical to aviation safety. Theres nothing wrong with that.