"Fire! Do Not Fire!": A new paradigm testing how autonomous systems affect agency and moral decision-making.

Abstract

Autonomous systems have pervaded many aspects of human activities. However, research suggest that the interaction with those machines may influence human decision-making processes. These effects raise ethical concerns in moral situations. We created an ad hoc setup to investigate the effects of system autonomy on moral decision-making and human agency in a trolley-like dilemma. In a battlefield simulation, participants had to decide whether to initiate an attack depending on conflicting moral values. Our results suggest that our paradigm is suitable for future research aimed at understanding the effects of system autonomy on moral decision -making and human agency.