In a simulated dog fight between two F-16 Fighting Falcons, an AI programmed by defense contractor Heron Systems beat a human pilot, callsign “Banger” five rounds to zero.

This outcome is not-at-all surprising. I would expect an AlphaZero style reinforcement learning agent would easily be able to achieve superhuman skill in aircraft dogfights. I haven’t been able to figure out how the AIs were trained, but it sounds like most of them were “expert systems”.

The simulation environment surprised me a bit. I am having trouble finding details on the rules, but it looks like the aircraft had a fixed number of “hit points”, and you damaged the other aircraft by getting it directly in front of you at relatively close range, maybe 1,500-2,000 ft. I’m not sure if this was meant to model an engagement with guns, or if air-to-air missiles have an extremely short range and narrow target acquisition cone.

I believe that in modern history the overwhelming number of aircraft engagements have been between aircraft and surface-to-air missiles, so this experiment already seems largely irrelevant to the practicalities of modern warfare.

I am frustrated that I keep seeing AI competitions where the human only gets a handful of chances against an AI opponent it has never seen before. An AI will often have a different set of strength and weakness than a human, and 5 matches does not give the human very much time to discover how the AI works. This problem was particularly egregious with DeepMind’s Starcraft bot, AlphaStar, and OpenAI’s Dota bot, OpenAI Five. Both AIs handily beat human professional players during their debut. OpenAI Five was open to humans to play against for a weekend, and OpenAI boasts that it won 99.4% of its games. But at least one team of non-elite human players was able to win a four game streak against the OpenAI Five, which makes me pretty confident that most professional teams could regularly beat it with a little practice.