Artificial intelligence and human factor

The unique human capacity for moral judgment and ethical decision-making is more than a complex set of algorithms, and this capacity cannot be reduced to the programming of a machine which, however “intelligent” it may be, remains a machine.

June 16, 2024

Pope Francis at the G7 summit in Puglia, Italy (Vatican Media)

By Andrea Tornielli
“Autonomous weapon systems, including the weaponization of artificial intelligence, are a source of serious ethical concerns. Autonomous weapon systems can never be morally responsible subjects. The unique human capacity for moral judgment and ethical decision-making is more than a complex set of algorithms, and this capacity cannot be reduced to the programming of a machine, which, however “intelligent” it may be, remains a machine. , it is imperative to ensure adequate, meaningful and consistent human oversight of weapons systems. This is what Pope Francis writes in his. Message for the World Day of Peace 2024.

An episode that occurred forty years ago should become a paradigm whenever we talk about artificial intelligence applied to war, weapons and instruments of death.

It is the story of a Soviet officer who, thanks to his protocol-defying decision, saved the world from a nuclear conflict with catastrophic consequences. This man's name was Stanislav Yevgrafovich Petrov, a lieutenant colonel in the Russian army.

On the night of September 26, 1983, he was on night duty in the “Serpukhov 15” bunker, monitoring American missile activities. The Cold War is at a crucial turning point, American President Ronald Reagan is investing massively in weapons and has just described the USSR as an “evil empire”, while NATO is engaging in military exercises simulating scenarios of nuclear war.

In the Kremlin, Yuri Andropov had recently spoken of an “unprecedented escalation” of the crisis, and on September 1, the Soviets shot down a Korean Air Lines commercial airliner over the Kamchatka Peninsula, killing 269 people.

On the night of September 26, Petrov noticed that the Oko computer system, the “brains” considered infallible for monitoring enemy activity, had detected the launch of a missile from a base in Montana aimed at the Soviet Union.

Protocol required the officer to immediately notify his superiors, who would then give the green light for a retaliatory missile launch toward the United States. But Petrov hesitated, reminding himself that any potential attack would likely be massive. He thus considered the solitary missile as a false alarm.

He did the same thing for the next four missiles that appeared on his monitors shortly after, wondering why no confirmation had come from the ground radar. He knew that the intercontinental missiles took less than half an hour to reach their destination, but he decided not to sound the alarm, astonishing the other soldiers present.

In reality, the “electronic brain” was wrong; there was no missile attack. Oko had been deceived by a phenomenon of refraction of sunlight upon contact with high altitude clouds.

In short, human intelligence had seen beyond that of the machine. The providential decision not to act was made by a man whose judgment was capable of looking beyond the data and protocols.

The nuclear catastrophe was averted, although no one became aware of the incident until the early 1990s. Petrov, who died in September 2017, commented that night in the “Serpukhov 15” bunker: “What did- I am doing ? Nothing special, just my work. I was the right person, in the right place, at the right time.

He was a man capable of evaluating the potential error of the supposedly infallible machine, the man capable – to use the words of the Pope – “of moral judgment and ethical decision-making”, because a machine, also “intelligent » whatever it may be, it remains a machine.

War, Pope Francis repeats, is madness, a defeat for humanity. War is a serious violation of human dignity.

Waging war by hiding behind algorithms, relying on artificial intelligence to determine targets and how to reach them, thus relieving one's conscience because it was the machine that made the decision, is even more serious. Let's not forget Stanislav Evgrafovich Petrov.–Vatican News