F-35 vs. A-10, Explained: Emotionally, it was never going to be a fair fight.

The grizzled A-10 Thunderbolt II “Warthog” close air support aircraft, with its unmistakable profile, ear-splitting 30mm Gatling-style autocannon, and reputation for delivering ground troops from certain death, is so beloved that lawmakers and fans have successfully protested its retirement even as it nears a half-century of service.

By comparison, the F-35 Joint Strike Fighter, a stealthy fifth-generation jet whose best capabilities are high-end and invisible, is relatively new to the fight, and still best-known by many for its arduous path to production, fraught with cost overruns, delays, and setbacks.

Nowadays, the F-35 has begun to prove itself. The unit cost of the jets dropped to a low of $78 million last year, down from a shocking $133 million more than a decade ago. The aircraft was first used in combat in 2019 and has begun to get its share of good press for its high-tech capabilities – such as its ability to create a network linking other aircraft in the sky and to obliterate enemy assets before they even know it’s in their vicinity. But in 2016, when rumors of an A-10 vs. F-35 “flyoff” first began to surface, many, including then-congresswoman and former A-10 pilot Martha McSally, thought the challenge would prove, once and for all, that the older and more visibly muscular Warthog could put the fancy new F-35 to shame.

The “flyoff” test happened in 2018, but the results remained largely cloaked in secrecy, even as organizations like the Project on Government Oversight (POGO) called for details. But at the end of October – nearly five years after the flyoff took place – POGO released the much-anticipated report. How did the two planes fare? It’s complicated.

POGO’s own analysis highlights the F-35’s failure to dominate in all categories, as some believed a fifth-generation aircraft would.

“The results were apparently not what the Air Force’s leaders expected, because they fought to hide them completely for years,” the independent oversight group said in its buzzy announcement of the newly public report. “The released report is heavily redacted. Still, the information they did release does not paint a very positive picture of the F-35’s ability to fill the highly critical role the A-10 has performed capably in the United States’ last three major ground wars.”

The organization added that the newly released information raises fresh concerns about Congress’s pending plan to approve the retirement of 42 A-10s in fiscal 2024, “a decision they are making in an information vacuum since the report has not been widely circulated,” POGO said. The organization received the previously classified report, which was completed in 2022, after taking the Pentagon to court over its initial refusal to release the information.

While large swaths of the report remain redacted, the Pentagon’s director of operational test and evaluation summarizes the outcome with a number of qualifiers.

“The comparison test was adequate to compare the mission effectiveness of two very different aircraft in a limited set of operationally representative ‘permissive’ and medium-threat ‘contested’ environments,” the executive summary states. High-threat missions, it adds, were not included as all variants of the F-35 were then in tests to confirm these capabilities.

The three capabilities that were involved in the flyoff were combat search-and-rescue; forward air controller (airborne), which involves in-flight airspace management of planes supporting ground troops; and direct close air support. The trials were conducted over Edwards Air Force Base, California, and Nellis Air Force Base, Nevada, and involved a mix of day and nighttime scenarios in both permissive and contested exercise environments. In all, according to the report, the flyoff involved 69 sorties and 117.5 hours of flight time. Nineteen A-10 pilots and eight F-35 pilots participated. The report acknowledged an imbalance from the start, as the A-10 pilots generally had more experience in a specific set of missions, while the F-35 pilots, though they had similar numbers of flight hours, had gotten most of their experience on other aircraft and in a more diverse set of missions.

From the outset, there were a few limitations in the parameters of the test due to training conditions and some communication system issues. Neither F-35 nor A-10 pilots could coordinate digitally with the joint terminal attack controller on the ground due to radio problems; digital communication with wingmen, and strikes during forward air controller (airborne) missions was also spotty. Only inert ordnance was used in the test, and though close-air support testing plans called for helicopters to support, none were available to participate. Ground troops were also unavailable for the test.

While none of these issues invalidated the tests, according to the report, “a more dynamic and representative environment for operational testing of these missions may be needed to judge improvement in [various systems’] performance.”

“The typical loadout of the A-10 enabled more attacks than the typical loadout of the F-35A, particularly in the test environment,” the report states of the close air support mission findings, amid thick blocks of redacted text.

Another partial statement describes “location errors,” for GPS-aided weapons, saying that A-10 pilots were able to fly closer to their targets than F-35 pilots were due to their tactics, and that this may account in the observed difference between the platforms regarding target location errors. Further down, a partly redacted paragraph appears to indicate a possible need for additional aircraft to conduct threat suppression and counter-air roles while the F-35 completes close air support missions.

Ultimately, though, the report reiterates that the F-35 maintains its “clear capability advantage over the A-10C in higher-threat environments – an environment where the A-10 was not designed to operate.”

For the sensitive forward air controller (airborne) test, nearly all findings are redacted except for a sentence stating that A-10 pilots reported “a significantly lower workload” than their F-35 counterparts while conducting the same mission.

In combat search and rescue, the report states that pilots of both aircraft found they would perform best if operating together to “combine the strengths of both platforms.”

Recommendations for the F-35 following the flyoffs are completely redacted from the report, although the introduction hints at some of the takeaways. Pentagon head weapons tester Nickolas Guertin wrote in that intro that the F-35 Joint Program Office should work with the Air Force and F-35 maker Lockheed Martin to fix the Joint Strike Fighter’s gun and shore up digital communications, video data link capability, and the ability of the plane to work in concert with fourth-generation aircraft. (The F-35’s main gun, a 25mm GAU-22/A cannon, has previously been cited for accuracy issues.) A partial sentence from the same intro appears to share a telling takeaway: that F-35s had to conduct more sorties than A-10s did to accomplish the same mission objectives.

Even with the limited data available, POGO Senior Defense Policy Fellow Dan Grazier wrote in his analysis of the report that the test revealed a lack of serious interest in comparing the two aircraft in combat-like conditions. The lack of ground troops in the close air support exercise, he said, suggests that “Air Force leaders weren’t interested in a true close air support demonstration.” The F-35, he added, wasn’t really put to the test in an environment that would require it to track and engage elusive ground targets. Likewise, he wrote, the open ranges of China Lake did little to replicate wooded and camouflaged environments where real combat search and rescue is most likely to happen.

For Grazier, the test did little to allay concerns over retiring the A-10 with no dedicated close air support platform to take its place. Doing so, he added, would be an “irrevocable mistake.”

“The ability of the F-35 to replace the A-10 has never been demonstrated,” he wrote. “Further, Air Force leaders have yet to take even preliminary steps to prepare their F-35 pilots to perform the most important combat role with which they have been entrusted.”

That’s not how the Air Force sees it, though. Last year, Air Force Secretary Frank Kendall called the A-10 “an old aircraft” that was “not survivable against modern threats,” joining the service’s yearslong call for the plane’s retirement. Kendall has since added that the aircraft “doesn’t scare China.”

These statements get to the heart of the greatest tension in defense planning and investment: should services continue to spend on what they know has worked in past conflicts, or pour resources into the tools and systems they believe will serve them well in the future. The F-35 is built for standoff missions against sophisticated near-peer foes; it’s not made to specialize in dogfights or close air support. The war in Ukraine illustrates the challenges of close air support on a modern battlefield where both sides have effective anti-aircraft weapons. Yet critics of the move to retire the A-10s rightly point out that having no direct replacement leaves the U.S. military vulnerable if an operational need for close air support does arise.

The Air Force’s current plan has all A-10s headed to the boneyard by 2029. The newly public flyoff report makes clear that the move leaves many questions unanswered about what will happen when the era of the Warthog ends.