Abstract

Artificial intelligence (AI) is increasingly embedded in healthcare in a variety of ways, ranging from semi‐autonomous decision support systems to the various visions of completely autonomous clinical systems. This article explores the ethical and legal issues that accompany this shift and argues that traditional ways of understanding medical responsibility and the physician‐patient relationships have been strained in light of the increasing autonomy of AI. Ethically, AI raises very acute concerns around transparency and informed consent in the case of “black‐box” systems, the persistence and amplification of bias, privacy risks arising from data‐intensive uses in system development, as well as the harmful effects of automation bias and professional de‐skilling. The article also brings to foreground an underexplored concern—the weakening of relational trust in contexts where clinical judgment and interactions are mediated by an AI, particularly where human oversight of the situation is reduced. Legally, this article outlines challenges of liability across the autonomy spectrum and criticizes proposals that would see AI as an independent liable agent. It promotes a human‐accountability model that includes clinical responsibility for assistive AI, coupled with enterprise measures of liability and post‐market oversight for ever‐more autonomy, backed by mutually appropriate insurance and governance measures. The article ends with some regulatory and institutional recommendations in order to facilitate safe, ethical, and equitable integration of AI in healthcare.