by Ed Solski
Flight test safety might sound like an oxymoron to those unfamiliar with the business of testing aircraft by flying them. The typical perception of those who dont understand flight testing is that of a group of daredevil pilots trying to rip the wings off a new airplane with little or no regard to the danger involved-loving the danger, in fact. Movies like The Right Stuff did a lot to create this image and misperception.
The truth is, people in the business of testing airplanes are probably the most conservative group of aviation professionals who fly. There are a number of reasons this is the case, not least of which is self-preservation. Additionally, the cost of a prototype aircraft today is astronomical, not to mention the political damage to a major aircraft program, civilian or military, if a major accident occurs during the test program.
Flight test safety, just like safety in any other aspect of everyday life such as driving a car, is not safety at all costs. We cannot guarantee 100% safety or a zero accident rate in anything that involves large, fast-moving objects with people in them. We drive our cars on busy freeways even though there are accidents, some fatal, every day. We fly on commercial passenger flights even though there are crashes every year that kill hundreds of people. We pay for and fly our own general aviation aircraft even though hundreds are killed each year doing the same thing. Why? Because we feel the benefits are worth the risks.
Of course, a slightly irrational element of human nature whispers in the back of our minds that it cant happen to me. This is true in the flight-testing business, not just in general aviation or flying on commercial flights, even though we see ourselves as being more mathematically inclined and analytical (not necessarily more logical!) than general aviation or commercial pilots.
Evolving Concepts
The concepts of systems safety and of risk management are not new. Back in the 1950s, the philosophy of flight test was fly, crash, fix, fly. Flight testing teams of that era would often wait until an accident occurred during testing, identify and correct the problem, and then fly again to see if the fix worked. The fighter and bomber flight test community averaged about 25 accidents a year in those days, one every two weeks! And this was the Golden Age of flight testing!
The Cold War and arms race with the Soviet Union combined with rapidly increasing defense budgets to create acceptance of this accident rate as simply a cost of doing business. But, as the cost of prototype aircraft increased rapidly in the late 1950s and 1960s, this philosophy had to change. Meanwhile, the emerging flight testing industry, airframe manufacturers and government just could not afford to lose expensive prototypes at such an alarming rate.
The system-safety concept was born during this period. Along with achievements in computer-based simulation, largely due to intercontinental ballistic missile development, satellites and space exploration, the concept became much more viable. Designers and system-safety experts could look at a proposed design while it was still on the drawing board then simulate and evaluate failure scenarios and consequences and effects on other systems. This new capability allowed the engineers to determine how much redundancy was required to obtain the required level of safety.
The two big advantages of this approach to system safety became obvious: not having to wait for an accident to occur during test and evaluation to identify design deficiencies, and being able to make changes in that design before production or prototype construction. As a result, system safety is sometimes described as cradle to grave safety, where highly refined safety principles are applied throughout the entire life cycle of a system, not just during its test and evaluation.
Trickle Down
At this point, the Cessna 182 owner would be justified in thinking that systems safety is great for a mega-million dollar F-22 fighter or 747-400, but of limited utility to him and his everyday flight operations. The answer is that the system-based approach to safety is all around him, whether he sees it or not. In one common example-todays GPS-based avionics-even a small system costing thousands of dollars instead of millions uses these principles. The system-based approach means that VOR/ILS equipment can back up GPS automatically and vice versa, with modern, integrated systems making the changeover transparent to the pilot. Looking down the road, NASA is evaluating technologies that will allow small, relatively inexpensive general aviation aircraft to take off, fly to a destination and land automatically the way large commercial jets do today. The concepts and principles of system safety must be applied to those designs, just as Boeing or Airbus uses them today for the big boy jets.
Risk Management
Besides systems safety, another tool used in flight-test planning is risk management. As discussed earlier, we cannot expect a zero accident rate or 100% safety in anything we do, let alone test and evaluation of a new aircraft design. What we must do, though, is try to achieve the maximum level of safety consistent with cost, schedule and operational effectiveness. We do this by evaluating the risks associated with our test plan.
This is done in several steps. First, we identify the potential hazards in terms of the injury or damage they may cause: fatal, serious, or minor injury to personnel; catastrophic, major or minor damage to aircraft or equipment. Then we attempt to quantify the probability of the hazard occurring: one-in-a-thousand, one-in-a-million, one-in-a-billion, etc. Once these two steps are complete, we evaluate their combination. A high probability of occurrence combined with severe consequences in terms of injury or damage means the test being conducted should be classified as a high risk. Medium probability combined with the medium damage or injury category would be classified as medium-risk testing, and so on. However, something in the severe injury or damage category combined with a remote or extremely remote probability might still be classified as low-risk testing. In other words, combination the severity and the probability of occurrence is what determines the risk category.
For tests classified as high-risk, we then take whatever steps are necessary to reduce the probability of the hazard occurring and/or minimmizing its consequences.
One example is an anti-spin parachute installed during stall or spin tests. Another is parachutes for flight crew. Both are examples of steps taken to reduce the consequences should the hazard occur and aircraft control irretrieveably lost. Outside of flight testing, similar steps you can take with, say, a new engine installation, might include remaining over an airport for the first few flights, avoiding night operations and conducting extensive pre-flight inspections, especially under the cowling, until a comfortable number of flight hours has been reached safely.
Steps such as these usually reduce the risk category from high to medium. Once the risks have been identified, categorized and all the steps taken to reduce the risk as much as possible, someone-usually the person in charge of the flight test organization-must give approval to start the testing. This is usually done after a safety review board has evaluated the test plan, calculated the likelihood of something bad happening and considered all the consequences involved in the risk assessment and all of the safety precautions.
Evaluating Risk
By now, the Cessna 182 pilot asking how system safety applies to him and his airplane is probably wondering the same about risk management. Perhaps without realizing it, the 182 driver applies risk management to everyday life. If you stand on the second step of a stepladder to change a light bulb, the probability of falling off is low (but not zero). Similarly, the consequences are not severe: a bump on the head, perhaps, and some embarrassment are usualy involved in the worst-case scenario. If you take your extension ladder out of the garage and lean it up against your two-story house to paint the eaves, the probability of falling probably isnt much higher, but the consequences are much more severe.
If you are planning a VFR cross-country flight through the mountains and the weather is clear-and-a-million with light winds and minimal turbulence, enjoy the flight-the risks are low, even if the consequences of smacking into a ridge or suffering through bone-jarring bumps that twist aluminum out of shape are greater. If the weather is marginal VFR and you do not have an Instrument rating, you must weigh the probability of below-VFR weather blocking your route, combined with the severity of the consequences. You may end up trapped in a narrow pass with rising terrain, unable to climb or turn to avoid the cumulo-granite. If you were planning a cross-country across the flat terrain of the Midwest, marginal VFR weather may still present the same probability of below-VFR conditions, but the consequences are not as severe. Similarly, decisions and evaluations must be made when contemplating either operation in daylight or nighttime conditions. The probability of something bad happening must be combined with its probable severity and the likelihood of damage or injury to determine the overall risk level.
If the probability of something bad happening is high and the consequences disastrous, the risk level is high-dont go. If the risk level is moderate, always have an alternate-a backup plan of action-not just a weather alternate.
The thought process and the planning conducted to enhance flight-test safety isnt really that much different from the steps taken in commercial, military or general aviation operations. We may try to formalize the process for detailed evaluation by quantifying variables and assigning numerical probabilities in terms of failures rates, etc., but it really comes down to things like common sense, judgment, experience, discipline, thinking logically and not letting pride or ego get in the way.
You apply these concepts in everyday activities-like changing a light bulb. Applying them to your flight operations in a logical, consistent manner might help make you an older, wiser pilot. And thats not such a radical, concept, after all.
Also With This Article
“Charting Risk Management”
-Ed Solski is Director of Marketing and a Test Pilot Instructor (Fixed Wing) for the National Test Pilot School in Mojave, Calif. (www.ntps.com)
