Abstract

This paper explores the importance of ensuring AGI (Artificial General Intelligence) controllability and safety as AI systems advance from narrow AI (NAI) to more autonomous systems. AGI's ability to learn and make decisions independently introduces significant challenges, particularly in critical sectors like healthcare, finance, infrastructure, and also when it is embedded in a physical object, such as a robot. Traditional AI governance principles of transparency, explainability, and accountability become insufficient when dealing with more sophisticated AGI, which, in this paper, refers to AI systems that exhibits significantly higher autonomy than current systems, as these models are too complex to be fully understood or controlled by humans. Instead, the paper argues that “controllability” element should be the primary focus of AGI governance to prevent unintended consequences.The paper examines technological approaches such as control by design, fail-safes and redundancy mechanisms, formal verification, adversarial testing, adaptive ethical constraints and sandboxing, alongside institutional strategies including business continuity planning, continuous monitoring, AI ethics boards, and multi-layered audits. It stresses that a combination of technological, institutional, and regulatory measures is essential to ensure AGI remains safe and aligned with human intent. The paper concludes by emphasizing the need for interdisciplinary collaboration among engineers, ethicists, legal experts and policymakers and calls for AGI development to be guided by human values and governance frameworks to avoid catastrophic risks and ensure that AI serves societal benefits.*Please note that this research is preliminary and intended to serve as a basis for open discussion during the Special Session.