Autonomous weapons face UN talks as AI race grows

A United Nations conference in Geneva will discuss curbs on lethal autonomous weapons from Monday, November 16, as analysts warn that artificial intelligence is changing how targets are selected and attacked.

Lethal Autonomous Weapons Systems can identify, select and engage a target without a human operator carrying out those tasks, said Jean-Marc Rickli of the Geneva Centre for Security Policy. He said the spread of AI-enhanced drones means such weapons can adapt to their environment and engage targets that were not predetermined.

The talks are being held under the Convention on Certain Conventional Weapons. Analysts said the central regulatory issue is how much human control is required before a weapon is used.

UN Secretary General Antonio Guterres and International Committee of the Red Cross president Mirjana Spoljaric warned in August that the world was nearing what they called a moral red line: the autonomous targeting of humans by machines.

Rickli said the Geneva agenda is narrow and does not cover the full use of AI on the battlefield, including target generation and decision-support systems. Laure de Roucy-Rochegonde of the French Institute of International Relations said systems already used on battlefields raise questions about whether human operators are making genuine judgments or simply approving machine recommendations.

Rear Admiral Vincent Sebastien, deputy director of the French defence ministry's Agency for Defence Artificial Intelligence, said autonomy will quickly become a battlefield reality. Dario Amodei, head of Anthropic, warned this year that frontier AI systems are not reliable enough to power fully autonomous weapons.

Analysts said major powers still have strong incentives to develop the technology, including autonomous drone swarms that could overwhelm an adversary.