'Killer robots': autonomous weapons pose moral dilemma

Will robots soon replace human decision-making in warfare? Even though no single definition for fully automated weapons systems exists, one thing is certain — they raise a host of legal, moral and arms-control questions.

The United Nations began talks on Monday on lethal autonomous weapons systems amid calls for an international ban on these "killer robots" that could change the nature of warfare.

Business | 03.11.2017

The weeklong meeting of a disarmament grouping known as the Convention on Certain Conventional Weapons (CCW) in Geneva comes after more than 100 leaders in the artificial intelligence industry warned in August that these weapons systems could lead to a "third revolution in warfare."

"Once developed, they will permit armed conflict to be fought at a scale greater than ever, and at timescales faster than humans can comprehend," the signatories said in a letter. "The deadly consequence of this is that machines — not people — will determine who lives and dies." 

While the rapid development of artificial intelligence and robotics in the past decade have led to improvements for consumers, the transport sector and human health, the military application of greater autonomy in weapons systems has evoked images of Terminator-type sci-fi war machines entering the battlefield to hunt down adversaries without any human behind the controls.

Conflicts | 30.06.2017
Filmstill Terminator Genisys 2015 EINSCHRÄNKUNG

Hollywood sci-fi movies have helped conjure up images of robots dominating humans on the battlefield

What is a lethal autonomous weapons system?

There is no international consensus on what constitutes a lethal autonomous weapon system, also known as a fully autonomous weapons system. It is often defined as a system that can target and fire alone without meaningful human control. In essence, they are machines with built-in hardware or software that allow them to function independently of humans once they are turned on. They function on the basis of artificial intelligence — algorithms assess a situational context and determine the corresponding response. 

Multiple weapons systems from drones, precision-guided munitions and defense batteries already have levels of autonomy, albeit with various degrees of human control. Several countries' militaries also use human controlled robots to search for mines, traps and unexploded ordnance. 

"Many people underestimate the extent of automation and computerization of warfare today. The use of modern sensors and munitions has already generated greater distance between the human and the battlefield in some cases," said Michael Horowitz, a professor of political science at the University of Pennsylvania who researches autonomous warfare. "Any discussion of autonomous weapon systems should start by understanding how they are similar to, and different from, existing military technologies."

Boston Dynamics Roboter US Marines Soldaten

A handout photo from 2015 shows US Marines using a dog-like prototype robot in a clearing operation

Are there lethal autonomous weapons systems in the world today?

Strictly speaking, lethal autonomous weapons systems do not exist today. Israel's Harpy anti-radar "fire and forget" drone is closest. After launch by ground troops, it autonomously flies over an area to find radar that fits pre-determined criteria and then unleashes a kamikaze strike.

South Korea has developed a sentry gun system to guard the heavily militarized border with North Korea. The weapons system, which includes surveillance sensors and tracking as well as automatic firing, can be made completely autonomous, but in its current use, it requires a human to approve before engaging. 

israelische unbemannte Drohne (UCAV) Harop

The Israeli made IAI Harop, or Harpy, is launched by ground troops to seek out and destroy enemy radar across the front lines

Slippery slope?

The Campaign to Stop Killer Robots, a group of NGOs seeking a ban on lethal autonomous weapons, says that sensors and advances in artificial intelligence make it "increasingly possible" weapons systems of the future would target and attack without human intervention.

"If this trend towards autonomy continues, the fear is that humans will start to fade out of the decision-making loop, first retaining only a limited oversight role, then no role at all," the group said in a statement.

In some cases, such as with cruise missiles, its sensors and terrestrial guidance systems can lead to more targeted strikes and fewer unintended casualties compared to traditional bombing. 

But the experts who signed the August letter expressed moral concern over the development of fully autonomous weapons systems out of today's semi-autonomous and human-supervised autonomous systems.

"Lethal autonomous weapons systems that remove meaningful human control from determining the legitimacy of targets and deploying lethal force sit on the wrong side of a clear moral line," they wrote.

London Aktion Campaign to Stop Killer Robots 2013

The Campaign to Stop Killer Robots has called for a ban on lethal autonomous weapons systems before they have the potential to proliferate

There is no guarantee any technological system will work perfectly, "but the moment you give it lethal weapons, the danger increases manifold," said Ulrike Franke, a policy fellow at the European Council of Foreign Relations who researches drones at Oxford University. "A smart armed system can become a dumb armed system quickly." 

A fully autonomous system gone awry could take unwanted action in complex battlefield situations, target civilians or engage in friendly fire. The possibility of mistakes with little or no human role also raises questions around the laws of war and military policy, such as who bears responsibility. 

"To what extent can we hold a military commander that deploys such a system responsible, if there is no meaningful way for him or her to predict how it will behave?" asked Franke. 

In the hands of unsavory regimes with little worry about such questions, such systems could be used against its own people. And in the hands of terrorists or non-state actors, a "killer robot" could result in devastating destruction. 

Afghanistan US Militär Drohnen

A US pilot controls a drone in Afghanistan. Drones have created much debate, but are ultimately controlled by humans

Arms control

But not everyone thinks that advances in fully autonomous weapons systems will diminish the human role in warfare.

Retired US Colonel Brian Hall, an Autonomy Program Analyst at the Joint Chiefs of Staff, wrote in July that the advantage of autonomous weapons systems will "come from augmenting human decision making, not replacing it."

Still, he cautioned that given the pace of advances in science and technology the weapons capability of autonomous weapons in the future are difficult to predict, which may require legal and policy changes. 

History

AI: 'Third revolution in warfare'

Over 100 AI experts have written to the UN asking them to ban lethal autonomous weapons — those that use AI to act independently without any human input. No "killer robots" currently exist, but advances in artificial intelligence have made them a real possibility. The experts said these weapons could be "the third revolution in warfare," after gunpowder and nuclear arms.

History

Gunpowder

The "first revolution in warfare" was invented by the Chinese, who started using the black substance between the 10th and 12th centuries to propel projectiles in simple guns. It gradually spread to the Middle East and Europe in the following two centuries. Once perfected, firearms using gunpowder proved to be far more lethal than the traditional bow and arrow.

History

Artillery

The invention of gunpowder also introduced artillery pieces to the battlefield. Armies started using basic cannons in the 16th century to fire heavy metal balls at opposing infantrymen and breach defensive walls around cities and fortresses. Far more destructive field guns were invented in the 19th century and went on to wreak havoc in the battlefields of World War I.

History

Machine guns

Guns that fire multiple rounds in rapid succession were invented in the late 19th century and immediately transformed the battlefield. Machine guns, as they came to be known, allowed soldiers to mow down the enemy from a protected position. The weapon's grisly effectiveness became all too clear in WWI as both sides used machine guns to wipe out soldiers charging across no man's land.

History

Warplanes

Military thinkers did not ignore the invention of the first airplane in 1903. Six years later, the US military bought the first unarmed military aircraft, the 1909 Wright Military Flyer. Inventors experimented with more advanced fighter and bomber aircraft in the following years. Both became standard features in many of the national air forces established by the end of WWI.

History

Mechanization

Armies had traditionally used soldiers and horses to fight and transport military equipment. But around WWI, they started using more machines such as tanks and armored vehicles. Faster and more destructive armies were the result. Nazi Germany put this new form of "mechanized warfare" to destructive effect in WWII using an attack strategy known as "Blitzkrieg" ("lightning war").

History

Missiles

Although artillery was effective, it had a relatively limited range. The missile's invention in WWII suddenly allowed an army to strike a target hundreds of kilometers away. The first missile — the German V-2 — was relatively primitive, but it laid the foundation for the development of guided cruise missiles and intercontinental ballistic missiles (ICBM) capable of carrying nuclear warheads.

History

Jet engine

Jet aircraft first saw action alongside traditional propeller airplanes at the end of WWII. Jet engines dramatically increased an aircraft's speed, allowing it to reach a target quicker and making it far harder for an adversary to shoot it down. After WWII, military reconnaissance planes were developed that could fly higher than 25 kilometers (15.5 miles) and faster than the speed of sound.

History

Nuclear weapons

The "second revolution in warfare" announced its horrific arrival on August 6, 1945 when the US dropped the first nuclear bomb — "Little Boy" — on the city of Hiroshima in Japan, killing between 60,000 and 80,000 people instantly. In the Cold War that followed, the US and Soviet Union developed thousands of even more destructive warheads and raised the specter of a devastating nuclear war.

History

Digitization

Recent decades have witnessed the ever more prevalent use of computers to conduct war. The devices made military communication quicker and easier and radically improved the precision and efficiency of many weapons. Armed forces have recently focused on developing cyber warfare capabilities to defend national infrastructure and attack foreign adversaries in cyberspace.

Several countries, including the United States, Russia, China and Israel are researching or developing lethal autonomous weapons systems out of concern adversaries may not be bound by humanitarian, moral and legal constraints, resulting in a potential "killer robot" arms race in the years to come as the technology improves. 

As Russian President Vladimir Putin said in September, whoever is the leader in artificial intelligence "will become the ruler of the world."

While the powerful potential of autonomous weapons on the battlefield causes concerns, it also makes them more difficult to ban or regulate, experts said.

"This is the arms control dilemma. The more useful potential weapons are for militaries, the harder it is to regulate or ban them," Horowitz said. "Uncertainty about what an autonomous weapon is further complicates the discussion – states are unlikely to agree to regulations or bans if they do not know what will be covered," he addded.

For Franke, an outright ban or arms-control regime is unlikely. Lethal autonomous weapons systems are not like nuclear weapons since they cannot be counted, which is a key requirement for arms control agreements. They also are unlike chemical weapons, which have been banned. And with no strict definition of what a lethal autonomous weapons system is, "there is no way to identify it by just looking at it," she said.