When Autonomy Breaks: The Hidden Existential Risk of AI

Speaker: Dr Joshua Krook (University of Southampton,

Responsible A.I. UK and the UKRI Trustworthy Autonomous Systems (TAS) Hub) https://joshuakrook.com/

Abstract:

AI risks are typically framed around physical threats to humanity, a loss of control or an accidental error causing humanity’s extinction, such as through cybersecurity breaches or biosecurity. However, I argue in line with the gradual disempowerment thesis, that there is an underappreciated risk, in the slow and irrevocable decline of human autonomy. As machines start to outcompete humans in every area of life, a tipping point will be reached where it no longer makes sense to rely upon human decision-making, creativity, empathy or even leadership.

What may follow is a process of gradual disempowerment and de-skilling, until humans lose control over their lives, leading to a complete abdication of human agency. Traditionally, it is argued that AI will gain human skills over time, and that these skills are (implicitly) innate and immutable in humans. By contrast, I argue that it is possible for humans to lose our capacity for these skills in an AGI world. The biggest threat to humanity is therefore not that machines will become more like humans, but that humans will become more like machines. This threat is hard to quantify in traditional domains like cybersecurity and law. This means that we may need new legal protections, digital human rights, and other remedies to prevent undue harm to our intangible freedoms.

Bio:

Joshua Krook is a Visiting Academic at Responsible A.I. UK and the UKRI Trustworthy Autonomous Systems (TAS) Hub, based out of the University of Southampton. The project oversees the development of socially beneficial AI systems that are both trustworthy in principle and trusted in practice by individuals, society and government.