Imaginary trolleys, real people, and true safety design for autonomous vehicles, SCV SSIT Chapter Meeting, September 16, 2020

Autonomous vehicles have the potential to transform not just the technology but the sociology of our roads. The public has been captivated by the question of how these vehicles will solve ethical dilemmas. One broad school of effort uses "trolley problems" -- thought experiments where an AV must choose who to hurt -- as a method for thinking about how AVs should behave. But in the real world, are trolley problems the right way to think about safety? Or are they simultaneously too pessimistic and too optimistic, vastly underestimating the risk of certain failure modes while focusing attention on overwhelmingly unlikely and avoidable failures? I argue the latter, and present a framework for thinking in a practical, deployment oriented way about how AVs should behave around humans.