In an aircraft hangar, a maintenance technician accesses the history of a faulty part, calls in advice from a superior on how to best fix it and logs the maintenance work, all with a few verbal commands and with his hands still in the aircraft. In an airport lounge, a customer service representative for an airline is able to pull information on a VIP customer in seconds and help speed him to his destination all while engaging him in conversation. And in the sky, a fighter pilot is able to access a 360-degree view of his surroundings without losing sight of his instruments or displays. All areas of the aviation industry are on the cusp of a revolution brought on by wearable technology, but is the hardware advancing fast enough to warrant widespread adoption?
Wearables, such as Google Glass and smart watches, made a big splash in 2014 as commercial carriers such as Japan Airlines and Virgin Atlantic struck out to trial how the tech could be used to streamline maintenance and customer service operations. Two years later, however, the aviation industry hasn’t seen these trials amount to full-blown adoption and, instead, operators are struggling in making the business case for the trendy technology.
“Just like all technologies, wearables experienced a lot of interest when it was first introduced just by virtue of the “cool factor,” which was evidenced by the interest in 2014,” Kevin Deal, vice president for aerospace and defense, at the North American division of software company IFS, tells Avionics Magazine. “Now, players in the industry are wondering when [wearable technology] will really take off.”
The take off will happen, according to Deal, when the users “pull” and not when the providers “push” the devices in the workplace. “Where you begin to see the adoption of the technology is when you see the utility of the device being able to provide you with the ability to do your job better, faster, more safely,” Deal says.
He believes that as the ‘cool factor’ wears off, where the industry stands on wearable technology today is in figuring out exactly what its application is, particularly when it comes to aircraft maintenance. As existing technologies come up, such as smartphones, smart glasses, and smart watches and new devices enter the market, such as a bracelet that can transmit to the touchscreen interface of a tablet onto the skin of your arm, the challenge no longer lies in the technology itself, but in how to configure it to the benefit of each use case.
Pushback from its consumer base stemming from privacy and safety concerns caused Google to forego production of its wearable headset prototype, Google Glass, on Jan. 15, 2015. While the device failed to take off in the consumer market, smart glasses likely have more of an application in enterprise use cases, according to Jim Peters, chief technology officer at SITA, which helped conduct a 2014 Google Glass and Sony smart watch trial in conjunction with Virgin Atlantic to test the ability of the devices to deliver information on VIP passengers to customer service representatives in Virgin Atlantic’s airport lounge.
“It seemed that wearables may have peaked-out because Google removed Glass from the market when there were issues with the consumer base. We spoke with Google after the Glass trial with Virgin Atlantic, and Google seemed to believe that Glass was a consumer device and should not be used for enterprise or industry. They seem to have since shifted direction and have restarted the program to look at enterprise use cases, such as aviation or health care,” said Peters.
While Peters notes that the customer-facing wearables trial went over remarkably well with both staff and passengers, the most promising use case for the technology in the aviation industry likely lies in maintenance.
“Wearable technology, whether it is a headset or a device the technician wears on his arm, etc., enables anticipatory maintenance so workers can work more effectively during scheduled maintenance and avoid what may have been an Aircraft on the Ground [AOG] situation later,” explains IFS’ Deal. This can impact the bottom line by shortening the maintenance tail and opening up an extra “seat or two” in profit out of the aircraft.
“Now that you have the ability to access information on the aircraft and parts more quickly and easily, and provide anticipatory maintenance, you are really exponentiating [sic] the impact that you have on the maintenance and logistics tail of your operations. It becomes hugely impactful,” he adds.
This hasn’t been lost on the airlines. Japan Airlines trialed Google Glass for their maintenance operations at its Honolulu station in 2014 with the aim to increase work efficiency. During the trial, maintenance staff could receive advice and instruction by audio during operations as well as photograph or video real-time information that could be shared with colleagues off-site.
“The trial confirmed the technology’s contribution toward efficiency of operations and its contribution towards increased operational safety, since by providing maintenance staff with the wearable device, it allowed for a hands-free environment,” Junya Okuda, manager of network infrastructure strategies at Japan Airlines tells Avionics Magazine. “In the future, the hands-free device with AR [Augmented Reality] technology is expected to be used for providing instructions to frontline operators and directly offer related information if needed.”
Okuda says the company wasn’t perturbed when Google took its Glass off of the market, and is still looking at other Glass-like tech to improve the efficiency of maintenance operations, up operational safety and reduce aircraft turnaround times.
Microsoft’s Hololens, which is currently only available in a developer version, could be one such alternative that could easily support maintenance operations.
“The Hololens provides a holographic overlay of what you are looking at, so if you are looking at an engine, for instance, you could pin a Skype conversation to the side of the engine with the expert and have a video conference with him about how to best conduct repair work in a holographic view,” said Peters.
Another smart glass alternative is Sony’s SmartWear, a more ruggedized version of smart glasses. Sony partnered with Virgin Atlantic in March 2015 to launch a trial of the technology in the U.K. airline’s maintenance operations. Engineers working on Virgin Atlantic aircraft at the airport and in the hangar tested the use of Sony’s SmartEyeglass Developer Edition SED-E1, tablet, mobile phone and SmartWatch 3 in an eight-week trial at London Heathrow to test how the technology could be used for real-time communication between the engineering team on the aircraft and in the engineering support areas.
Engineers used the glasses to take pictures or video of the tasks they were working on, which was linked to an app running on a smartphone that would allow the engineers to efficiently complete and submit a form requesting further technical assistance. The glasses were also used for real-time video streaming to allow office-based engineering staff to see a problem from the engineers’ point of view in order to provide more rapid technical assistance.
With Virgin Atlantic’s Sony SmartWear trial, while the engineers found it useful, the technology is still too immature to incorporate into maintenance operations on a large scale.
“What we did with our maintenance engineers using our Sony SmartEyeglass had kind of a limited application, it is still in development at the end of the day. Sony were as interested to find out what the uses would be and get our feedback for future elements of the device as much as we were interested in seeing what the practical applications would be,” Tim Graham, senior manager of integration, development and innovation at Virgin Atlantic Airways tells Avionics Magazine. “Some of the glass technology is still in its infancy. Whilst it was useful, it wasn’t groundbreaking. It still needs to develop a bit more before you can roll anything out in any significant numbers with the airline professional areas.”
While airlines struggle to break wearable tech into their operations, the tempo and urgency of military operations have brought wearable technology into the cockpit already. Elbit Systems has developed two Helmet-Mounted Display (HMD) systems that aim to incorporate Head-Up Displays (HUDs) and even avionics into a wearable.
Elbit’s Targo product is a complete mission life-cycle Helmet Mounted Avionics (HMA) technology, which relocates aircraft avionics to the pilot’s helmet for intense military flying environments. The helmet can incorporate GPS / Inertial Navigation System (INS) capabilities, Digital Video Recording (DVR), the company’s pilot health monitoring system known as Canary, and a range of displays and databases for obstacles, terrain, threats, and more. Incorporating these technologies into the helmet as opposed to keeping them on the aircraft allows pilots to access HUD symbology at 360 degrees, and can help to save on weight, cabling and install work for operators that are looking to save on costs.
“The big advantage of the helmet is that every avionics system that is integrated on the aircraft can be integrated with the helmet, same as with the legacy system, but the customer can add on avionics systems in case the aircraft isn’t already equipped with them or if the operator only wants to enable low-cost, compact integration,” explains Yaron Kranz, senior director of business development and research and development for helmet-mounted systems at Elbit Aerospace Systems. Adding avionics on helmet can reduce the number of aircraft installed units, reduce cabling, and simplify aircraft installation and integration. All these reduce system, installation and maintenance costs.
The company has also launched a commercial equivalent, Skylens, which acts as a wearable HUD implementation for platforms that cannot accommodate or afford to integrate HUDs.
“By enabling HUDs on a wearable device, we open up the head-up display market for many more customers who couldn’t have this capability before, and ATR is typical of that demographic,” says Elbit’s Vice President of Commercial Aviation Dror Yahav. The technology is set to make its debut on the regional turboprop ATR-600 series aircraft when it is fully certified in the second half of 2017.
“For a traditional head-up display, the ATR’s aircraft’s cockpit is too small,” says Yahav, but the Skylens wearable HUD, which fits like a pair of ski goggles onto the pilot’s head to display Elbit’s Clear Vision Enhanced Flight Vision System (EFVS) information on a pilot’s visor, is compact enough to fit in the cockpit. Also, it not only enables traditional HUD functionality but also, as the system is not fixed to a structure in the cockpit, the system can present information to the pilot from his or her own point of view. This means HUD and EFVS functionality, which has traditionally been limited to a view through the front window, can also service side views.
While the military is not yet using wearables for maintenance operations, the technology could help to close the knowledge gap.
“More and more in the military arena there is someone who is qualified to do the job, but they need to be able to pull from the knowledge base of more senior technicians to enable them to be more powerful, more efficient and more capable. Providing them with wearable technology allows them to have that window to the back office,” says Deal.
The market will likely begin seeing an impact from wearables in the next one to three years. Virgin Atlantic’s Graham says that while the technology still needs to mature, at the rate wearables are developing, the company will likely begin integrating wearables into its business to manage passenger services and resources — feeding employees and customers information on flight times, aircraft turnaround times and operational messaging — within this time frame. Maintenance operations will take a bit longer to incorporate, however, and Graham estimates it will be five years before the airline can roll it out in a larger capacity.
“In five years or so, the technology is moving at such a pace that there will be a point at which there will be a serious contender in the marketplace that we can roll out and have it be supportable while offering us the benefits we need,” Graham said, although he is unsure exactly what device that may be.
“What we’re slowly discovering is that amongst the wearable technology that’s out there, there isn’t a killer device that we can that we can look at and roll out to everyone for every purpose. It’s going to be more of a case of which is the appropriate device for the interaction we need.”
The technology still comes up against a few snags before airlines can begin thinking about a full-scale roll out, including a lack of cybersecurity.
“The market needs to look into cybersecurity surrounding wearables as to how you wipe the data if they get lost or stolen. There isn’t much available to the wearable market now unlike the smartphone field, and that could potentially be holding some companies back from adopting,” says Graham.
Similarly, he sees a snag around the necessary blanket coverage of data connectivity that many devices require to operate effectively.
“Our staff works in mobile around the world and if we roll out a particular device, we need to make sure it is going to work everywhere that we operate, and that is a challenge today even with smartphones,” Graham adds. “We can make the business case in specific use cases quite easily; it’s when you want to roll out a global use case that it begins to get more challenging.” AVS