1 The terminology and framework are derived from Article 36 of Protocol Additional to the Geneva Conventions of 12 August 1949, and Relating to the Protection of Victims of International Armed Conflicts, 8 June 1977, 1125 UNTS 3 (entered into force 7 December 1978), Art. 35(1) (hereinafter Additional Protocol I).

2 This article distinguishes only between ‘autonomous’ and ‘remote’ weapons systems. This is to differentiate between direct human control over a weapons system, which is retained in remote weapons systems, from the unique departure from direct human control over the use of weapons signalled by introducing autonomy into weapons systems. The ICRC distinguished between different levels of autonomy within weapons systems by formulating three separate categories: remote controlled weapons systems; automated weapons systems; and autonomous weapons systems. ICRC, ‘International Humanitarian Law and the Challenges of Contemporary Armed Conflicts’, 31st International Conference of the Red Cross and Red Crescent, Geneva: Switzerland, 28 November–1 December 2011, 31IC/11/5.1.2, pp. 38–40.

3 Hart, H. L. A. and Gardner, John, Punishment and Responsibility: Essays in the Philosophy of Law, Oxford University Press, New York, 2008, pp. 211–237CrossRefGoogle Scholar.

4 Ibid., p. 214.

5 Ibid., pp. 211–237. Role responsibility considerations may be relevant to autonomous and remote weapons systems because of the likelihood that their efficacy will be assessed upon fulfilment of their objectives. This, however, will fall foul of the disjuncture created between role and outcome responsibility, ultimately exacerbating the diffusion of responsibility for the consequences of utilizing such weapons systems.

6 See Liu, Hin-Yan, ‘Leashing the corporate dogs of war: the legal implications of the modern private military company’, in Journal of Conflict and Security Law, Vol. 15, 2010, pp. 141–168CrossRefGoogle Scholar; and Liu, Hin-Yan, Law's Impunity: Responsibility and the Modern Private Military Company, Hart, Oxford, 2014Google Scholar (forthcoming).

7 Singer, Peter W., Wired for War, Penguin, New York, 2009, pp. 94–108Google Scholar.

8 Arkin, Ronald, Governing Lethal Behaviour in Autonomous Robots, Chapman & Hall/CRC, Boca Raton, 2009, pp. 7–27CrossRefGoogle Scholar.

9 Ibid., pp. 10–14. See also, Krishnan, Armin, Killer Robots, Ashgate, Farnham, 2009, pp. 28–30Google Scholar.

10 A. Krishnan, above note 9, pp. 27–28.

11 BBC News, ‘Mapping US drone and Islamic militant attacks in Pakistan’, in BBC News, 22 July 2010, available at: http://www.bbc.co.uk/news/world-south-asia-10648909 (last visited 9 December 2012). See also The Bureau of Investigative Journalism, ‘Covert war on terror – the data’, in The Bureau of Investigative Journalism, 8 May 2012, available at: http://www.thebureauinvestigates.com/category/projects/drone-data/ (last visited 9 December 2012), and the section entitled ‘Targeted killings and remote weapons systems’ below.

12 P. W. Singer, above note 7, pp. 120–122. In this context, it should be noted that while nuclear and other weapons of mass destruction are prohibited in orbit or on celestial bodies, conventional military activities are only forbidden on celestial bodies per Article 4 of the Treaty on Principles Governing the Activities of States in the Exploration and Use of Outer Space, including the Moon and Other Celestial Bodies, UNGA Res. 2222 (1966). This leaves open the potential for lawful conventional military activity to take place in orbit under the Treaty.

13 For a brief history, see P. W. Singer, above note 7, pp. 46–65, and A. Krishnan, above note 9, pp. 13–32.

14 Full and direct, albeit remote, human control should ground concomitant responsibility for the use of these weapons systems. The most uncontroversial category of unmanned vehicle would be those that ‘are used for any purpose other than the delivery of kinetic force against enemy personnel and objects’, see Boothby, William, Weapons and the Law of Armed Conflict, Oxford University Press, Oxford, 2009, pp. 229–230CrossRefGoogle Scholar.

15 A. Krishnan, above note 9, pp. 23–24.

16 Ibid., pp. 33–59. See also, Human Rights Watch and International Human Rights Clinic (Harvard Law School), Losing Humanity: The Case Against Killer Robots, November 2012, pp. 6–20.

17 The independent capability of weapons to inflict violence is apparent in mines for instance, and is usually objected to on the grounds of indiscriminateness or existence of threat after the cessation of hostilities.

18 It is clear, for example, that the Rome Statute for the International Criminal Court only contemplates the inclusion of natural persons as perpetrators of the international crimes it establishes. See Article 25(1): ‘The Court shall have jurisdiction over natural persons pursuant to this Statute’ (emphasis added). While this has been interpreted to exclude legal persons such as corporations, the emergence of autonomous weapons systems challenges both the presumption that only natural persons can be perpetrators, and also the continued tenability of the provision of Article 25(1).

19 Section 220 of the National Defense Authorization Act for Fiscal Year 2001 (Public Law 106–398; 114 Stat. 1654A–38): ‘(a) GOAL. – It shall be a goal of the Armed Forces to achieve the fielding of unmanned, remotely controlled technology such that – (1) by 2010, one-third of the aircraft in the operational deep strike force aircraft fleet are unmanned; and (2) by 2015, one-third of the operational ground combat vehicles are unmanned’.

20 Section 941(b)(2) of National Defense Authorization Act for Fiscal Year 2007 (Public Law 109-364; 120 Stat. 2083).

21 US Department of Defense, ‘Unmanned systems roadmap 2007–2032’, Washington DC, 2007, available at: http://www.dtic.mil/cgi-bin/GetTRDoc?Location=U2&doc=GetTRDoc.pdf&AD=ADA475002 (last visited 18 January 2012).

22 Schmitt, Michael, ‘War, technology and the law of armed conflict’, in Helm, Antony (ed.), War in the 21st Century: Weaponry and the Use of Force, Naval War College Studies: International Law Studies, Vol. 82, 2006, p. 149Google Scholar.

23 Gulam, Hyder and Lee, Simon, ‘Uninhabited combat aerial vehicles and the law of armed conflict’, in Australian Army Journal, Vol. 3, No. 2, 2006, p. 126Google Scholar.

24 A. Krishnan, above note 9, pp. 35–37.

25 R. Arkin, above note 8.

26 See for instance, Nick Hopkins, ‘MoD announces further 4,200 armed forces personnel cuts’, in The Guardian, 18 January 2012, available at: http://www.guardian.co.uk/politics/2012/jan/17/mod-4200-armed-forces-cuts?INTCMP=SRCH (last visited 18 January 2012).

27 At least twenty countries are known to possess significant military robotics research programmes. See A. Krishnan, above note 9, p. 13.

28 BBC News, ‘Iran unveils first bomber drone’, in BBC News, 22 August 2010, available at: http://www.bbc.co.uk/news/world-middle-east-11052023 (last visited 9 December 2012); and Robert Beckhusen, ‘China unveils its new drone fleet to the world’, in Wired, 28 November 2012, available at: http://www.wired.co.uk/news/archive/2012-11/28/china-unveils-new-drones (last visited 9 December 2012). For a glimpse into how the US could lose the robotic revolution, and for the growing non-state use of military robotics, see P. W. Singer, above note 7, pp. 237–278.

29 Additional Protocol I of 1977, Art. 36.

30 Oxford Dictionary of English, Oxford University Press, Oxford, 2005.

31 For example, it is not the inherent characteristics of a weapon that are of concern under international law, but rather the manner in which it is used: ‘The aim of Article 36 [of Additional Protocol I] is to prevent the use of weapons that would violate international law in all circumstances and to impose restrictions on the use of weapons that would violate international law in some circumstances’ (emphasis added). International Committee of the Red Cross Geneva, ‘A guide to the legal review of new weapons, means and methods of warfare: measures to implement Article 36 of Additional Protocol I of 1977’, in International Review of the Red Cross, Vol. 88, No. 864, 2006, p. 933Google Scholar.

32 Indeed, even in the Commentaries for Article 36, Jean de Preux makes numerous references to the term ‘weapon’ without elaborating upon its characteristics or attempting to provide a definition. Pilloud, Claude et al. , Commentary on the Additional Protocols of 8 June 1977 to the Geneva Conventions of 12 August 1949, ICRC and Kluwer, 1987Google Scholar, paras. 1463–1482. The same holds true for The Convention on Prohibitions or Restrictions on the Use of Certain Conventional Weapons Which May Be Deemed to Be Excessively Injurious or to Have Indiscriminate Effects as amended on 21 December 2001.

33 W. Boothby, above note 14, p. 1.

34 McClelland, Justin, ‘The review of weapons in accordance with Article 36 of Additional Protocol I’, in International Review of the Red Cross, Vol. 85, No. 850, 2003, p. 404Google Scholar. This connotation is especially noteworthy because a weapon could not, by this definition, be used to target civilians. Clearly, this approach would need to be broadened to account for other uses of weapons.

35 ICRC, above note 31, p. 932, fn 1.

36 Ibid., p. 937, fn 17.

37 Ibid., p. 937, fn 17 (emphasis added).

38 Lawand, Kathleen, ‘Reviewing the legality of new weapons, means and methods of warfare’, in International Review of the Red Cross, Vol. 88, No. 864, 2006, p. 927CrossRefGoogle Scholar.

39 Daoust, Isabelle, Robin Coupland and Rikke Ishoey, ‘New wars, new weapons? The obligation of states to assess the legality of means and methods of warfare’, in International Review of the Red Cross, Vol. 84, No. 846, 2002, p. 352Google Scholar.

40 J. McClelland, above note 34, p. 405.

41 Coeckelbergh, Mark, ‘From killer machines to doctrines and swarms, or why ethics of military robotics is not (necessarily) about robots’, in Philosophy and Technology, Vol. 24, 2011, p. 273CrossRefGoogle Scholar.

42 Pagallo, Ugo, ‘Robots of just war: a legal perspective’, in Philosophy and Technology, Vol. 24, 2011, pp. 307–323CrossRefGoogle Scholar. See also, Anderson, Kenneth and Waxman, Matthew, ‘Law and ethics for robot soldiers’, in Policy Review, No. 126, 2012Google Scholar.

43 Johannson, Linda, ‘Is it morally right to use unmanned aerial vehicles (UAVs) in war?’, in Philosophy and Technology, Vol. 24, 2011, pp. 279–291CrossRefGoogle Scholar; and Schulzke, Marcus, ‘Robots as weapons in just wars’, in Philosophy and Technology, Vol. 24, 2011, pp. 293–306CrossRefGoogle Scholar.

44 Military Manual of Germany, as quoted in Henckaerts, Jean-Marie and Doswald-Beck, Louise, Customary International Humanitarian Law Volume I: Rules, ICRC and Cambridge University Press, Cambridge, 2005, p. 13CrossRefGoogle Scholar (hereinafter ‘ICRC Study’).

45 The question of categorizing autonomous weapons systems as combatants is not considered further in this article. It should also be noted that the right of the belligerents to choose their means and methods of warfare is not unlimited, see ICRC, above note 31, p. 931. See also, Article 22 of the 1907 Hague Regulations Respecting the Laws and Customs of War on Land, and Additional Protocol I, Article 35(1), above note 1.

46 USs Department of State, ‘Treaty between the United States of America and the Union of the Soviet Socialist Republics on the Elimination of Their Intermediate-Range and Shorter-Range Missiles’, 1988, available at: http://www.state.gov/www/global/arms/treaties/inf2.html (last visited 9 December 2012).

47 US Department of State, ‘Definitions Annex: Treaty Between the United States of America and the Union of Soviet Socialist Republics on the Reduction and Limitation of Strategic Offensive Arms’, 1991, available at: http://www.state.gov/www/global/arms/starthtm/start/defini.html#36 (last visited 9 December 2012).

48 The US did, however, abandon the deployment of the Harpy, an Israeli UCAV for fear of violating the 1987 Treaty. See H. Gulam and S. Lee, above note 23, p. 130; and Antony Lazarski, ‘Legal implications of the uninhabited combat aerial vehicle, in Air & Space Power Journal, 2001, available at: http://www.airpower.maxwell.af.mil/airchronicles/cc/lazarski.html (last visited 9 December 2012).

49 Roberts, Adam, and Guelff, Richard, Documents on the Laws of War, Oxford University Press, Oxford, 2000, p. 53Google Scholar.

50 J.-M. Henckaerts and L. Doswald-Beck, above note 44, Rules 70 to 86, pp. 237–296. For an opposing perspective, see Turns, David, ‘Weapons in the ICRC Study on Customary International Humanitarian Law’, in Journal of Conflict and Security Law, Vol. 11, 2006, pp. 201–237CrossRefGoogle Scholar.

51 International Court of Justice (ICJ), Legality of the Threat or Use of Nuclear Weapons, Advisory Opinion, ICJ Reports, 1996Google Scholar, para. 78.

52 Jean de Preux, in C. Pilloud et al., above note 32, para. 1402.

53 ICRC, above note 31, p. 938.

54 A cursory analysis is provided in W. Boothby, above note 14, pp. 229–232.

55 Ibid., pp. 229–230.

56 Ibid., p. 230.

57 Ibid., p. 230.

58 Ibid., p. 233.

59 Sharkey, Noel, ‘Cassandra or false prophet of doom: AI robots and war’, in IEEE Intelligent Systems, Vol. 23, 2008, p. 17CrossRefGoogle Scholar.

60 On these facts, because the Predator is strictly a remote weapons system, combining the review may not be problematic since it may not significantly alter the means and methods of warfare that previously passed the legal review test. By contrast, autonomous weapons systems may significantly alter the legality review.

61 W. Boothby, above note 14, pp. 55–68.

62 Ibid., pp. 69–85. See also ICJ, above note 51, p. 257.

63 J.-M. Henckaerts and L. Doswald-Beck, above note 44, pp. 237–244.

64 W. Boothby, above note 14, p. 230.

65 See Spencer Ackerman, ‘US Troops will soon get tiny kamikaze drone’, in Wired Magazine, 18 October 2011, available at: http://www.wired.com/dangerroom/2011/10/tiny-kamikaze-drone/ (last visited 18 January 2012).

66 J.-M. Henckaerts and L. Doswald-Beck, above note 44, pp. 244–250.

67 Sharkey, Noel, ‘Grounds for discrimination: autonomous robot weapons’, in RUSI Defence Systems, Vol. 11, 2008, p. 87Google Scholar.

68 P. W. Singer, above note 7, pp. 124–125.

69 Ibid., p. 125.

70 Ibid., p. 125. While this concerns ‘blue-on-blue’ fire and thus does not engage discrimination in the sense of being able to differentiate between combatants and civilians, it does illustrate the crudity of current systems in this area.

71 Seen in this light, it may be aberrant that a body of jurisprudence has, however, emerged in the United States attesting to the superiority of robotic judgement and requiring deference to this judgement by human beings. In Klein v. US. (13 Av.Cas. 18137 [D. Md. 1975]), the court found that in cases of negligence, and whilst the pilot is not required to use the autopilot on a landing, his failure to use it may be inconsistent with good operating procedure and may be evidence of a failure of due care. In Wells v. U.S. (16 Av.Cas. 17914 [W.D. Wash. 1981]), another court inferred negligence on the part of the human pilot from evidence that he switched from automatic pilot to manual control in a crisis situation.

72 J.-M. Henckaerts and L. Doswald-Beck, above note 44, pp. 56–58. This Rule is further supplemented by Rules 18–21, ibid., pp. 58–67.

73 W. Boothby, above note 14, p. 233.

74 ‘Each party to the conflict must do everything feasible to cancel or suspend an attack if it becomes apparent that the target is not a military objective or that the attack may be expected to cause incidental loss of civilian life, injury to civilians, damage to civilian objects, or a combination thereof, which would be excessive in relation to the concrete and direct military advantage anticipated’. See J.-M. Henckaerts and L. Doswald-Beck, above note 44, Rule 19, pp. 60–62.

75 W. Boothby, above note 14, p. 233.

76 Dinstein, Yoram, The Conduct of Hostilities under the Law of International Armed Conflict, Cambridge University Press, Cambridge, 2004, p. 59CrossRefGoogle Scholar (emphasis added).

77 N. Sharkey, above note 67, p. 88.

78 J.-M. Henckaerts and L. Doswald-Beck, above note 44, Rule 46, p. 162; Rule 47, p. 164, and Rule 65, p. 225.

79 C. Pilloud et al., above note 32, para. 1402.

80 There is evidence that the US is creating a global apparatus to carry out targeted killings. See Greg Miller, ‘Under Obama, an emerging global apparatus for drone killing’, in The Washington Post, 28 December 2011, available at: http://www.washingtonpost.com/national/national-security/under-obama-an-emerging-global-apparatus-for-drone-killing/2011/12/13/gIQANPdILP_story.html (last visited 18 January 2012).

81 Melzer, Nils, Targeted Killing in International Law, Oxford University Press, Oxford, 2008, p. 5CrossRefGoogle Scholar.

82 Philip Alston, ‘Statement by Professor Philip Alston, Special Rapporteur on Extrajudicial, Summary or Arbitrary Executions’, United Nations Human Rights Council Geneva, 3 June 2009, available at: http://www.un.org/webcast/unhrc/11th/statements/Alston_STMT.pdf (last visited 18 January 2012).

83 Harold Koh, ‘The Obama Administration and international law’, Annual Meeting of the American Society of International Law, Washington, DC, 25 March 2010, available at: http://www.state.gov/s/l/releases/remarks/139119.htm (last visited 20 December 2010).

84 Christof Heyns, however, has raised doubts about the legality of targeted killing practices used by the United States, stating that ‘mere reference to a statement made by a senior State official is insufficient’. Report of the Special Rapporteur on extrajudicial, summary or arbitrary executions, Christof Heyns, Addendum Follow-up to country recommendations – United States of America, UN Doc. A/HRC/20/22/Add.3, 30 March 2012, paras. 76-84, especially para. 79.

85 Targeted killings outside of the context of armed conflict or within the territory of the state itself are unlikely to be lawful. Philip Alston, ‘Report of the Special Rapporteur on Extrajudicial, Summary or Arbitrary Executions, Philip Alston: Addendum Study on Targeted Killings’, UN Doc A/HRC/14/24/Add.6. 25, 2010, available at: http://www2.ohchr.org/english/bodies/hrcouncil/docs/14session/A.HRC.14.24.Add6.pdf (last visited 18 January 2012). This article does not consider other situations that are not governed by IHL.

86 Additional Protocol I of 1977, Art. 36 (emphasis added). C. Pilloud et al., above note 32, para. 1469.

87 ICRC, Interpretive Guidance on the Notion of Direct Participation in Hostilities under International Humanitarian Law, Nils Melzer (ed.), ICRC, May 2009, pp. 70–71.

88 International Human Rights and Conflict Resolution Clinic (Stanford Law School) and Global Justice Clinic (NYU School of Law), Living Under Drones: Death, Injury, and Trauma to Civilians from US Drone Practices in Pakistan, September 2012; see also, BBC News, above note 11.

89 Scott Shane, ‘C.I.A. is disputed on civilian toll in drone strikes’, in The New York Times, 11 August 2011, available at: http://www.nytimes.com/2011/08/12/world/asia/12drones.html?hp (last visited 12 January 2012).

90 N. Melzer, above note 81, pp. 368–370.

91 P. W. Singer, above note 7, pp. 56–57.

92 Gillespie, Alexander, A History of the Law of War: The Customs and Laws of War with Regards to Arms Control, Hart, Oxford, 2011, pp. 7–78Google Scholar.

93 David S. Cloud, ‘Civilian contractors playing key roles in US drone operations’, in The Los Angeles Times, 29 December 2011, available at: http://www.latimes.com/news/nationworld/world/la-fg-drones-civilians-20111230,0,6127185.story (last visited 18 January 2012).

94 Shue, Henry and Wippman, David, ‘Limiting attacks on dual-use facilities performing indispensible civilian functions’, in Cornell International Law Journal, Vol. 35, 2001–2002, p. 562Google Scholar.

95 See generally, N. Melzer, above note 87, pp. 41–68.

96 Brian Bennett, ‘Police employ Predator drone spy planes on home front’, in The Los Angeles Times, 10 December 2011, available at: http://articles.latimes.com/2011/dec/10/nation/la-na-drone-arrest-20111211 (last visited 18 January 2012).

97 Jonathan Franklin, ‘Whaling: campaigners use drones in the fight against Japanese whalers’, in The Guardian, 1 January 2012, available at: http://www.guardian.co.uk/environment/2012/jan/01/drones-fight-japanese-whalers (last visited 18 January 2012).

98 W. J. Hennigan, ‘Idea of civilians using drone aircraft may soon fly with FAA’, in The Los Angeles Times, 27 November 2011, available at: http://articles.latimes.com/2011/nov/27/business/la-fi-drones-for-profit-20111127 (last visited 18 January 2012).

99 Bremner, Susan, Cyberthreats: The Emerging Fault Lines of the Nation State, Oxford University Press, Oxford, 2009, p. 65CrossRefGoogle Scholar.

100 Office of the Secretary of Defense, 110th Congress, Annual Report to Congress: Military Power of the People's Republic of China, 2007, p. 22, available at: http://www.defenselink.mil/pubs/china.html (last visited 18 January 2012).

101 Quoting a Liberation Army Commentator, ibid., p. 21.

102 Associated Press, ‘Computer virus infects drone plane command centre in US’, in The Guardian, 9 October 2011, available at: http://www.guardian.co.uk/technology/2011/oct/09/virus-infects-drone-plane-command (last visited 18 January 2012).

103 Agence France-Presse, ‘Iran to “reverse-engineer” seized stealth drone after hacking operating system’, in The National Post, 12 December 2011, available at: http://news.nationalpost.com/2011/12/12/iran-to-reverse-engineer-u-s-stealth-drone/ (last visited 18 January 2012).

104 In other words, will the controller of such a system have the capacity to be responsible for the actions of the system? Clearly, this issue is magnified if the system possesses any level of autonomy.

105 S. Bremner, above note 99, p. 7.

106 R. Arkin, above note 8, 2009, pp. 29–36 and 211–212.

107 J. McClelland, above note 34, pp. 408–409 (emphasis added).

108 N. Sharkey, above note 67, pp. 87–88. Not only is there a need for ‘a clear computational definition of a civilian, [but] we would still need all of the relevant information to be made available from the sensing apparatus … These may be able to tell us that something is a human, but they would not be able to tell us much else’.

109 R. Arkin, above note 8, pp. 29–30.

110 Cane, Peter, Responsibility in Law and Morality, Oxford University Press, Oxford, 2002, pp. 31–34Google Scholar.

111 M. Coeckelbergh, above note 41, p. 273.

112 Ibid.

113 Kennedy, David, The Dark Sides of Virtue: Reassessing International Humanitarianism, Princeton University Press, Princeton, 2004, pp. 296–297Google Scholar.

114 Ibid., p. 297.

115 Meron, Theodore, The Humanization of International Law, Martinus Nijhoff, Leiden, 2006, p. 85Google Scholar.