Avionics upgrades in development for the F-35 Lightning II include a new automatic ground collision avoidance system and a distributed aperture system (DAS) with five times the resolution of its current 360-degree sensor package. Software and applications for the fifth-generation fighter jet are being updated on a weekly basis, and Lockheed Martin has some 60 software and hardware upgrades planned for the multi-role fighter over the next 10 years.
Lockheed Martin is also researching increased integration of artificial intelligence (AI) and machine learning on the F-35. One of the technological challenges for the aircraft is parsing, sharing and making use of the data it gathers within its powerful sensor suite. To that end, the OEM has a future vision of establishing a reality where everything the pilot sees out of the cockpit is completely augmented with constant AI-recommended decision-making.
In 2011, the F-35 Lightning II Joint Strike Fighter (JSF) program was close to being canceled. Then U.S. Defense Secretary Bob Gates put the bloated, over-budget and behind-schedule program on probation with two years to shape up before the U.S. government would cut its losses and it could have gone either way, according to the program manager for the F-35’s drag chute system, Arthur Sheridan.
The Lockheed Martin fighter jet is built around information. It’s supposed to give its team the upper hand by always having more information than the enemy thanks to its advanced sensor suite, ability to share information, low profile and ability to adapt to almost any situation with its three variants: the standard F-35A with conventional takeoff and landing capabilities, the F-35B with short-takeoff-and-vertical-landing capability and the F-35C, which has larger wings with foldable tips and improved low-speed control for landing on aircraft carriers.
None of that matters, though, if it’s too heavy to fly the way it needs to. Weight problems previously restricted fuel capacity and flight distance, speed and agility beyond what was acceptable, especially in the case of the F-35B. It also put too much stress on certain joints and rivets, which contributed to well-documented reliability issues. Fixing hardware and software problems and redesigning sent costs sky-rocketing.
Lockheed Martin spent two probationary years making more than 600 changes to optimize size, weight and power, resulting in the shedding of 2,600 lb and a 700-lb thrust increase. That got the JSF back on track and, ultimately, off probation — but not off the hot seat. Since then, Lockheed Martin and its myriad partners have been focused on keeping weight and costs down and quality up, and working toward that promise of a flying do-it-all sensor laboratory.
Integral to that promise are the purported 9.1 million lines of code running under the surface of the F-35. Many of those direct the data fusion engine that connects all the powerful sensors the F-35 carries and combines everything into information pilots can use to get that crucial leg up on enemy combatants.
Data Fusion and Sharing
The F-35’s calling card is being a mobile sensor package that can improve the effectiveness of an entire fleet by sharing data and coordinating information such that the situational awareness of every warfighter present is increased exponentially.
Making good on that promise has been a rocky endeavor, but the potential has kept the program a priority. Lockheed and the military are now happy with what it enables: Costs are coming down, reliability and production are up, and pilots can use the cutting-edge sensor suite to maintain tactical advantage and respond to threats in a wide variety of situations.
Lockheed Martin fellow and Information Fusion Chief Scientist Thomas Frey provided an example of how the fighter fuses multiple pieces of data into useful information at June’s AIAA Aviation Forum in Atlanta.
He explained, if there were 35 aircraft near an F-35, the infrared sensor could detect all of the them and the direction they are in by heat signals, but has no way to tell who is friend or foe. The electronic warfare systems may pick up 22 of the 35, giving additional information about those. Six of the 35 are within the Doppler radar’s field of view, giving a clear picture of those six.
The F-35 also incorporates data sent from air- and ground-based allies. That helps sort friend from foe and provides angle and range data originating from different points. With that, the system will automatically triangulate a target’s geolocation from multiple sources who picked it up.
Once all of that is incorporated, the fusion engine can combine it into one picture of the battlefield that it displays to the pilot, identifying the location and distance of all 35 targets and which are foes. The autonomous sensor manager assigns a percentage to how confident it is in each of its determinations.
At the same time, it’s constantly rechecking each of those determinations and sending data into a network for other aircraft to use while doing the same thing. One important distinction is that the final map of information it comes up with after all that combination is referred to as Tier 3 data: incorporating everything the F-35 observed as well as everything it received from allies, what Frey calls “rumors.”
The F-35 sends Tier 1 data it has collected with its own sensors to allied battlefield networks. That way, each jet is only feeding the network with firsthand, reliable information. Then the others, and the network as a whole, can be the source of new Tier 3 data without being muddied by compounding rumor data that may or may not have been reliable.
Testing shows that this is important to create the clearest final information and strongest conclusions: Frey said blind-firing missiles based on shared targeting data has been successful in tests.
This method of data collation also lets each machine use the data as it sees fit since depending on where a jet is on the battlefield, different targets and actions need to be prioritized.
If a target is nearby, the system automatically makes it a high priority and focuses sensors and processing power on accurately determining the nature and location. The sensor knows the pilot will imminently need to make decisions about the target. If the same target is farther away, it might be lower priority, so it doesn’t receive the same prominence.
Further, while this all happens under the surface, pilots can interact and request certain information, targets or focuses if they so choose. In those cases, it’s helpful if the system has the raw data every F-35’s sensors collected rather than the impressions the F-35 already synthesized that data and the rumors it has received into.
To handle the necessary bandwidth, the F-35 has two primary data-sharing channels: the multi-function advanced datalink (MADL) network and Link 16.
“For MADL, you can best think of that as a way to extend one avionics system into multiple aircraft,” said Greg Lemons, Lockheed Martin’s missions systems expert for the F-35. “It’s designed around a four-ship flight group to be able to exchange the data that each aircraft sees, and for each airplane to take that data and fuse that into the information for the pilot.”
MADL allows greater, faster data sharing than other systems and is difficult for enemy forces to jam. The downside is it’s designed solely for communication between F-35s. The B-2 Spirit bomber is MADL-compatible. There were plans to upgrade the F-22 Raptor, but they were scrapped. The U.S. Defense Department has talked of upgrading other aircraft to MADL, but it’s expensive and cumbersome.
The legacy Link 16 system has wide compatibility. Lemons said that unlike most platforms that simply have a Link 16 communication box on board, the F-35 flies with a receiving box to see what other aircraft would receive from the fifth-gen fighter.
“On Link 16, you have to understand how the other platform uses your data and processes it so you can send it to them in a way that is meaningful to them,” he said. “The challenges were more associated with how the other platforms would interpret our data and making sure what we sent them provided them good information rather than information they couldn’t use because of how it was formatted.”
Automatic Collision Avoidance
One upcoming advancement for the F-35 is the automatic ground collision avoidance system (AGCAS). On the F-16, the system automatically avoids the ground when necessary. To date it has saved the lives of eight pilots (but only seven jets).
More specifically, the AGCAS uses GPS, terrain data and spatial awareness to recognize when the jet is heading toward the ground or a mountain, and if it’s too late for a pilot to react to that fact, the system intervenes and pulls up on its own before returning control to the pilot.
Lockheed Martin experimental test pilot Billie Flynn worked on the AGCAS system along with Air Force Research Lab engineers at what is now the NASA Armstrong Flight Research Center. Decades of flight data showed it’s extremely rare that pilots crash simply because they’re flying too close to the ground, he said.
“Pilots get disoriented in bad weather,” Flynn said. “They suffer spatial disorientation because they are moving aggressively at night. They get mixed up sometimes doing tasks in the cockpit and lose perspective of exactly where they’re at.”
Alerts won’t help if a pilot is disoriented or momentarily unconscious. AGCAS prevents a ground collision in that situation and gets the pilot out of harm’s way.
“We determined that a human can only stand 1.5 seconds before he thinks he better do something,” Flynn said. “This system is so robust that it’ll take control long after you and I, the human, would want to be pulling away from the ground if we were paying attention. It’s so precise that it will miss the ground essentially every single time. And that means that it won’t take over early from the pilot — whatever task we’re doing we will be allowed to do — but if we were way too close to the ground, it would take control and save us.”
Flynn is in charge of integrating AGCAS onto the F-35 five years ahead of schedule. It should be operational on an F-35 in 2019 instead of 2024. And he says the applicability doesn’t stop there.
“This technology is so interesting that it’s going to end up in corporate jets,” Flynn said. “It’s going to end up in private airplanes that people fly, and it’s going to end up in commercial airplanes that you and I load onto every single week.”
Flynn said he expects it to require commercial pressure for airlines and OEMs to decide to implement AGCAS. Once the decision is made, though, it won’t take long, thanks to the sophistication of technologies such as GPS, which commercial airliners already have.
Normally, the AGCAS activates at 2,300 feet. For testing, Lockheed adds 10,000 more to that floor, so the F-35 thinks it is close to the ground when it gets to 12,300 feet and pulls up. That way, they can make sure the system recognizes the jet’s distance from the ground and reacts appropriately without putting a test pilot in a situation where the AGCAS is the only thing preventing him from crashing.
Electronic warfare is also a concern in a hacking event in which someone could snatch control away from the pilot.
“That has never been an issue with this system,” Flynn said. “The way we’ve always worked is it defaults off. So, it’s never going to fly you up.”
A so-called “nuisance fly-up” is when the system pulls the aircraft up unexpectedly without pilot input.
“We have spent years stripping out the potential of any nuisance issues with the flight controls and how the aircraft behaves,” Flynn said.
Lockheed still needs to do live testing with the AGCAS in the F-35 before pilots can use it in combat, Flynn said. “It’s a software drop. It’s like updating your iOS,” he added.
Among the F-35’s radar capabilities, its Northrop Grumman-built AN-APG-81 AESA is notable.
“It’s the biggest antenna on any airplane, so it gathers the most data,” said Dan Dixon, Northrop Grumman’s director of F-35 development planning. Compared to a mechanically scanned array, AESA radars can quickly scan any direction.
Pilots appreciate the information the sensor package ultimately gets them, Dixon said, and the military is pleased with the ripple effect of increased situational awareness for the entire fleet. He said, however, that there’s more work to do determining who gets what information through MADL and Link 16.
“It’s basically the internet of the sky for all U.S. and coalition partners,” Dixon said. “You would have had to do that with your voice in the past. … It’s a mix of who gets what, but I have the opportunity to choose the richness of the data that gets exchanged. I think that’s probably the game-changer at that point.”
The Northrop team “is spiraling capability every week” in a way that “commercial folk would find compelling,” Dixon said. Software and apps are being updated on a weekly basis, he added, and that the platform is agile in a way belied by stories of decade-long processes for follow-on modernization.
The testing and verification process and a desire by operators to have a stable baseline for a couple of years to unify training prevents slow major upgrades, he said. Northrop is always working on new capabilities, but the military wants upgrades about every two years.
Northrop’s biggest challenge in the past and a major focus going forward is on affordability.
The “Joint Program Office is under a lot of pressure to work affordability initiatives in, and we get it. That’s all recompete-driven, so the enterprise gets it, too,” Dixon said. “Nobody’s above reproach. We treat it that way. We’re always looking for affordability features. We’ve come down 30 to 40 percent since we started making the radar.”
The F-35’s signature DAS sensor will also be changing in the future.
The AN/AAQ-37 DAS produced by Northrop Grumman uses six electro-optical sensors that operate in the mid-wave infrared spectrum to provide a 360-degree view around the plane in an attempt to maximize situational awareness. It aids in missile and launch point detection, and its cameras are integral to the helmet-mounted display’s (HMD) ability to let the pilot see through the plane.
Northrop’s contract to supply the DAS only extends through 2023, but the company declined to compete for a follow-on contract. Raytheon won the contract beginning with Lot-15 F-35 DAS, which Lockheed Martin’s Skunk Works head and former F-35 lead Jeff Babione said will have five times the resolution of the current DAS at a lower cost.
The DAS may also be the seed of a future technology Babione envisions for the F-35: the ability for the pilot to look anywhere and see a highly augmented version of what surrounds the fighter regardless of the canopy. He described that possibility at AIAA as a potential technology being explored that could come to the F-35 much later than AGCAS and the DAS switch.
The root of that technology may already exist on the F-35, though. The DAS surrounds the F-35 with cameras feeding real-time video and, in the words of Lockheed Martin VP of F-35 Engineering and Technology Santi Bulnes, “The current HMD augments what the pilot sees with tactical symbology today no matter where he looks.”
It even works with two forward-facing cameras so that “with a press of a button, an IR image can also be displayed on the HMD, which enables looking through aircraft structure or augmented night vision,” Bulnes said.
The technology can virtually erase a portion of the front canopy bow that would otherwise block the pilot’s line of sight. Lemons said that could potentially be a building block for 100% augmented vision.
The F-35 currently costs between $94 million (F-35A) and $122 million (F-35B) for low-rate initial production run 10, though sustainment cost projections are as big a concern as production costs for a program that is expected to cost $1.5 trillion over its 55-year lifespan.
The unit prices are expected to continue to drop as the fighter enters full-rate production this year. Soon the unit price for all three versions of the jet is expected to drop below $100 million.
At the same time, Lockheed is shifting its focus from upfront development toward its continuous capability development and delivery (C2D2) strategy, which focuses on agile upgrades. That’s where things such as constant small software updates, additional capabilities like AGCAS and long-term planning for things like full augmented vision fit in.
The F-35 program has no shortage of critics. But it seems like the promise it has always held might be on the cusp of realization. AVS