Synthetic Vision: A Must-Have for the Next Generation of Pilots?

A Boeing 787-9 is climbing out of Ecuador’s Mariscal Sucre International Airport, one of the busiest airports in South America. Volcanic ash suddenly sweeps across the airplane, rupturing the engines dead, disabling the auxiliary power unit (APU) and leaving one hydraulic system operational.

This is one of the scenarios in which 24 junior “ab-initio” pilots from Avianca Airlines used synthetic vision to perform a near safe landing — with no prior experience with the equipment. Ab-initio pilots are pilots who are in the earliest stages of their career and are preparing to obtain a type rating and begin flying commercial passenger-carrying aircraft.

In a 2017 Space Act Agreement between Avianca Airlines, Boeing and the National Aeronautics and Space Administration (NASA), young, non-native, English-speaking, non-military pilots demonstrated with a full-motion 787-9 with modified software providing weather-independent synthetic-vision cockpit displays that they could learn to use synthetic vision to land an airplane under some stressful situations pilots might experience on a daily basis.

As part of this research project, Rockwell Collins provided the synthetic vision hardware, which L3 Technologies integrated into the simulator facility. Over the past decade, NASA has published several studies on using synthetic vision for improved situational awareness for pilots.

Because of security constraints, NASA typically relies on research study participants who are highly experienced pilots (often military) and native English speakers, which doesn’t reflect today’s global pilot community.

Avianca pilots, like most commercial airline pilots today, train in simulators and cockpits that use classic cockpit displays using the blue-over-brown depiction of the aircraft’s heading, airspeed, attitude and other critical information about how the aircraft is flying on a primary flight display. Synthetic vision is a cockpit technology that merges high-resolution display with databases of terrain and obstacle data, aeronautical information, data feeds from other aircraft and global positioning system (GPS) to show pilots where they are and what exists outside of the aircraft.

The technology on the displays was the subject of research performed at NASA Langley Research Center and led by Kyle Ellis, a research technical lead at the agency.

NASA selected Avianca for the research because it wanted to go outside of its normal assortment of pilots that it uses for such research, typically career pilots with thousands of hours of experience flying commercially. Aside from having no prior training using synthetic vision to land airplanes, the pilots had never seen it prior to being placed into simulated approaches into some of Latin America’s most difficult airports, represented by Boeing’s full-motion simulators in Miami. Their performance using the technology to land in life-threatening situations is being analyzed by NASA right now to determine how little training the next generation of pilots would need with such an intuitive technology that represents the type of interactive graphical user interface that they have become accustomed to on modern mobile smartphone and tablet apps.

“We have destinations at high terrain altitudes in mountainous areas, where the airway structure is complex, where the route altitudes are pretty high,” said Leonardo Roldan, Avianca’s training manager. “We have many route altitudes up to 25,000 feet. Having a perspective of what the terrain looks like, it allows the pilot to have more knowledge of the surrounding, to make better decisions.”

Roldan had to go through the approval of his manager, Nelson Quiroga, training director of Avianca Holdings, as well as the Colombian carrier’s VP of Flight Operations Eduardo Mendoza to get approval for the Avianca pilots to perform the training.

Eventually Avianca CEO Hernan Rincon approved the use of the pilots for the research project free of charge to NASA, which also obtained the use of the Boeing 787-9 simulator with no cost, under its Space Act Agreement.

Currently, NASA researchers are analyzing the data collected from the Avianca experiment, concentrating on the performance of the young pilots. But Roldan, who witnessed the scenarios himself was convinced of the safety improvement benefits of synthetic vision, especially for the millennial generation of pilots.

“Their performance using the technology was better under those scenarios than it would have been while using the current technology that they fly with,” said Roldan. “I think it has to do with the new generation and what they’re used to handling earlier in life. They have the iPads, the touchscreens at a very early age, so it made it easier for them to use this type of virtual world representation that they could see on the display, even if they could not see outside of the cockpit.”

Avianca’s route structure features some of the most difficult environments in South America. They encounter unpredictable severe weather patterns often, where synthetic vision could allow for a safer method of dealing with degraded visual environments.

“Everything is LaGuardia on steroids,” said Daniel Kiggins, a 34-year captain with American Airlines and NASA research pilot, describing the airport approach environments with which pilots are commonly faced in Colombia and throughout South America. Kiggins helped write the scenarios where volcanic ash would disable the engines of a 787 while on approach to Mariscal Sucre, an international airport in Quito, Ecuador, among other hazardous in-flight situations. The airport historically is one of the most difficult high-altitude airports to land at in the world.

According to Kiggins, all 24 of Avianca’s ab-initio pilots were able to get within 500 feet of the runway despite losing engines, APUs and all but one hydraulic system on approach.

“Losing both engines out of the volcanic ash, you don’t have a quick fix or a real set of options that you can apply to that type of situation,” said Roldan. “Seeing the performance of the co-pilot while having these issues in a simulator — and how close [the pilots] were able to get to the runway — and the final result of the exercise is a safe landing, it’s clear that synthetic vision is a game-changing navigation capability.”

According to the U.S. Bureau of Labor Statistics, overall employment of commercial airline pilots is projected to grow 4% from 2016 to 2026, with the majority of job opportunities resulting from the need to replace pilots who are approaching retirement age. Recent examples of industry moves by major carriers show that a potential pilot shortage is real.

Ryanair, for example, made international headlines in September 2017 when it canceled 20,000 flights because it did not have enough standby pilots.

In February, Australia’s largest carrier, Qantas, announced plans to establish an academy to train 500 pilots per year in order to address its need for skilled aviators. The academy will open in 2019.

At the 2018 Singapore Air Show, Singapore Airlines announced a new partnership with Embry-Riddle Aeronautical University to give future pilots a fast track to a career flying airplanes in Asia Pacific, which Boeing estimates needs 253,000 new pilots between now and 2036 to support projected air traffic growth in the region.

Combine this with evidence presented to Embry-Riddle’s 2017 National Training Symposium. Geoffrey Murray, an aerospace sector leader for aviation consulting firm Oliver Wyman, said that 10 years ago, the typical ATP-rated pilot had 15 years of experience and was a college graduate in his or her late 30s. Today, the military accounts for only 30% of new airline hires. Graduates with professional pilot degrees decreased by more than 23% between 2002 and 2012, according to Murray.

As younger pilots are needed to fill the growing shortage faster, studies such as the Avianca experiment can provide proof that placing the next generation of pilots into cockpits with synthetic vision could only increase their chances of surviving the types of stressful scenarios that pilots can only become accustomed to with real-world, on-the-job experience.

“The pilot population is getting younger and younger, and they’re coming to commercial aviation with less experience,” said Capt. Jim Wilkerson, flight crew training instructor for Boeing.

Wilkerson provided an overview of the scenarios used by the Avianca pilots inside the simulators at Boeing’s Miami training facility. “When the engines fail, that can be jarring, and synthetic vision has the ability to improve the reaction time of pilots to allow them to safely recover the airplane,” he said.

Synthetic vision is not a standard capability in today’s commercial airliner cockpits because airlines have to think in terms of the training expense that is associated with any and every cockpit modification they make. As an example, in its annual report published in February, Delta Air Lines claimed to have 13,324 pilots on staff.

When considering investing in synthetic vision, an airline such as Delta has to consider the cost of time in flight-training simulators, which can cost up to $500 per hour in some facilities. Multiply that cost per pilot, and add the additional cost of paying workers to install it on all 900 of Delta’s airplanes, and there is no business case. Furthermore, it also isn’t going to necessarily save costs by reducing fuel burn or carbon emissions; it’s main purpose is to improve safety.

Through trainings and research around studies though, such as the one with Avianca, Boeing and NASA, the evidence is becoming overwhelming for why the next generation of pilots entering cockpits should have synthetic vision as a standard feature in future airliner cockpits.

“Every pilot wants it, and we want it for our kids. We want safer planes for our kids,” said Kiggins.

The research performed using the Avianca pilots could lead to a future where the next generation of airliners come with synthetic vision as standard. NASA researchers also see it as a pre-cursor to the future use of combined vision systems, which merge synthetic vision and enhanced vision systems — a display with forward-looking infrared systems and millimeter wave radar — to give pilots a view of the outside world that the human eye is incapable of producing. AVS