So significant are DOT&E’s concerns about the integrity of ALIS that the report reiterates an earlier warning that program officials should find a way to operate the F-35 entirely without it, in case the network is compromised. F-35 program office officials claim that an F-35 can fly for at least 30 days without connecting to ALIS to exchange data and log maintenance actions. DOT&E wants the program to do better than that. “In light of current cybersecurity threats and vulnerabilities, along with peer and near-peer threats to bases and communications, the F-35 program and Services should conduct testing of aircraft operations without access to ALIS for extended periods of time.” However, DOT&E has not planned or mandated a testing event to confirm whether or not the F-35 can operate without ALIS for 30 days or more.

The fully integrated nature of all F-35 systems makes cybersecurity more essential than for any other aircraft. Legacy aircraft already in service are equipped with software-enabled subsystems, and while a hacker could penetrate the GPS system in a legacy system, because the subsystems are not fully integrated, a hacker could not also access the communications system, for example. The F-35 is inherently far more vulnerable. Lockheed Martin brags on its website about the aircraft’s “sensor fusion” that connects all of the onboard subsystems, such as the Active Electronically Scanned Array radar; the Distributed Aperture System; and the Communications, Navigation, and Identification Avionics system. That means enemy cyber-warriors need only compromise the software of one of these to corrupt the entire system. According to the 2018 Government Accountability Office report: “A successful attack on one of the systems the weapon depends on can potentially limit the weapon’s effectiveness, prevent it from achieving its mission, or even cause physical damage and loss of life.”

A 2007 incident shows what this could look like. A flight of F-22 fighters crossing the Pacific lost all of their systems when they passed over the International Date Line. In that case, a software glitch in the main processor wreaked havoc on all of the systems connected to it, including navigation, communications, and fuel indicators, forcing the flight to divert back to Hawaii. That was just the result of a coding error. It is not difficult to imagine what a hacker with malign intent could accomplish.

F-35 Can’t Be Tested Against the Most Serious Threats

The transition from developmental to operational testing is a milestone in any weapons program. For the F-35, it came nearly a decade late. Now, even with the extra time, the program has started the process with no apparent plan to resolve major, potentially life-threatening design flaws, and without several of the tools necessary to properly evaluate the aircraft’s combat effectiveness or suitability in the hands of troops serving on the front lines. The program office appears to be planning to complete this phase of testing without making a proper evaluation possible.

Prior to beginning the operational testing phase, officials had also failed to properly address 941 design flaws during the program’s development phase, with 102 listed as “Category I” flaws that “may cause death [or] severe injury,” or lead to major damage to the aircraft or seriously inhibit combat effectiveness. As POGO reported, rather than taking the proper corrective actions, program officials made paperwork adjustments in a series of meetings during the summer of 2018 to make some design flaws, like one involving the emergency transponder and another with the F-35A’s emergency tailhook, appear to be less serious “Category II” deficiencies.

Each of these 102 flaws could ground aircraft or force them to abort missions. These design flaws likely also contribute to the program’s poor availability rates. According to DOT&E, this will have an impact on the operational testing process, which requires an 80 percent availability rate for the 23 aircraft instrumented for operational testing. The fleet is averaging a monthly rate “well below” 80 percent (the rate is not specified in the report), which “will remain a challenge for the efficient conduct and timely completion of [operational testing].”

The operational testing plan also hinges on use of a complex simulation facility capable of reproducing the multi-plane enemy and friendly formations and the dense threat environment inevitable in any war against a near-peer adversary. This is necessary because the available test aircraft and the open-air test ranges in the western United States cannot replicate all the modern threats flights of six or more F-35s might face.

But, troublingly, DOT&E reports that the simulation facility is not expected to be fully functional until late 2019, right at the end of the current operational testing schedule. “Without [the simulator], the IOT&E will be unable to adequately assess the F-35 against dense and modern threats that are not available for open-air testing, resulting in operational risk,” the report states.

Known as the Joint Simulation Environment and located at Maryland’s Naval Air Station Patuxent River, the facility took over from the mismanaged and failed Lockheed VSim simulator development program. Programmers are now attempting to develop accurate, verified and validated F-35 cockpit simulators and ground and airborne threat simulators for pilots to fly virtual multi-ship missions against multiple enemy missile and aircraft defensive arrays, but, according to the report, this has run into serious problems.

The simulation facility can only produce credible and useful test results if its computer programs are based on accurate data from the F-35’s demonstrated flight, sensor, and weapons performance—and, according to the report, it is not clear that they are. The necessary data is gathered during flight tests and integrated into the simulation program, which is then supposed to complete a verification and validation process. However, DOT&E found that simulator development began without the crucial verification and validation step having been completed. On top of that, the basic and essential terrain modeling—familiar to anyone who has used a flight simulator on their home computer—has yet to be finished. Most distressingly, the physical facilities where all of this operates, which are to include cockpits and visuals, and even the buildings themselves, were not even completed by the start of operational testing.

Yet it appears that the F-35 program office intends to keep to its current operational testing schedule. Because the simulators will likely not be complete until the tail end of the schedule, the resulting report assessing the F-35’s combat suitability will likely have no valid basis for judging whether the F-35 can survive against a dense, multi-threat environment or whether it can function effectively in real-world four- or eight-ship formations.

Conflicts of Interest Skewing Operational Testing Results

Ensuring the absence of bias in test planning, execution, and reporting is just as important as having adequate resources and managing testing competently. Congress created DOT&E in the 1980s as an independent testing office in the Department of Defense to end the practice of contractors and the services’ acquisition advocates writing and grading their own exams. Nevertheless, the F-35 program office approved Lockheed Martin, the F-35’s prime contractor and the firm with the most to gain from a favorable operational assessment, to conduct one key F-35 system test and to analyze and report on the results of another.

First, instead of using a government cyber red team, as it should have done to avoid even the appearance of a conflict of interest in the process, the program office paid a Lockheed Martin red team to do the cybersecurity testing of the company’s own ALIS network—the heart and brains of the worldwide ALIS network—as part of the overall cyber-vulnerability evaluation of ALIS.

Second, the program office paid Lockheed Martin to analyze F-35 live-fire vulnerability tests and produce the F-35 Vulnerability Assessment Report on whether the aircraft met their contractual specifications and military requirements for pilot survival against four air-defense threat weapons. The Lockheed study determined, according to this year’s DOT&E report, that the three “F-35 variants meet [Joint Strike Fighter] JSF contract specification requirements to enable safe ejection of the pilot in the event of an engagement” for three out of the four threats. That’s the opposite result from the 2017 DOT&E report, which stated that the F-35B failed to meet pilot-survival specifications for three out of the four threats, and that the F-35’s vulnerability was higher than expected. The Lockheed analysis also concluded, unsurprisingly, that the F-35 met its military requirements to be at least as survivable as the legacy F-16 by managing to return to safe territory after being hit by each of the four threats.

Curiously, Lockheed’s Vulnerability Assessment Report did not analyze the other major legacy plane slated to be replaced by the F-35, the famously survivable A-10. Apparently to temper suspicions of bias, DOT&E states it will do an independent review of Lockheed’s conclusions—but it will not report the results until after operational testing is complete, so the results will be all but useless for this key phase.

Lockheed was also entrusted with doing an overall analysis and summary of all of the Navy’s F-35 live-fire-test data, which will likely influence all future computer modeling of F-35 survivability rates in combat. The DOT&E report does not indicate when this study is expected to be released.