The Alignment Problem in artificial intelligence concerns how to insure that artificial general intelligence (AGI) conforms to human goals and values and remains under human control. The concept of general intelligence, modelled on human and animal behavior, lacks coherence. The ideal of autonomy inherent in AGI conflicts with the ideal of external control. Truly autonomous agents are necessarily embodied, but embodiment implies more than physical instantiation or sensory input. It means being an autopoietic system (like a natural organism), (...) with its own priorities and values, which may compete and conflict with those of humans. Ambiguous terms and concepts, and problematic notions such as ‘orthogonality’, are critically examined, as well as inconsistencies in thought and expectation concerning AGI. Two paradigmatic approaches to the Alignment Problem are compared: Stuart Russell’s and Eric Drexler’s. Large Language Models are discussed in the context of the Turing Test. It is concluded that task-oriented tools, not autonomous agents, should be the goal of AI research. (shrink)

The problem of aligning artificial intelligence with human values is typically framed as a technical challenge: how to specify, learn, or constrain machine behavior so that artificial systems reliably produce ethically acceptable outcomes. This paper argues that this framing is fundamentally incomplete and omits an important aspect of moral agency. The central claim of this contribution is that ethics is not primarily a formalizable rule-set, preference ordering, or optimization target, but an emergent property of the human condition. Human moral (...) agency arises from embodiment, affect, vulnerability, social embeddedness, teleological orientation, and the developmental formation of character. Alignment research, by contrast, usually treats ethics and moral agency as if it could be abstracted from these conditions and translated into reward functions, constraints, preference models, or oversight procedures. To capture this diagnosis, the paper introduces the concept of the Embodied Ethics Alignment Problem (EEAP): if moral agency depends on the embodied, affective, social, and teleological aspects of the human condition, then systems that lack those aspects cannot be ethically aligned in the “strong sense” often implied by the alignment discourse. They may be behaviorally steered toward acceptable outcomes (in the “weak sense”), but that is not the same as sharing or instantiating human moral agency. Furthermore, due to the EEAP, artificial agents may face a hard limit in terms of simulation capabilities of embodied ethics. The paper, therefore, argues that the primary task is, therefore, not to build moral machines, but to govern non-moral optimization systems within human institutions capable of bearing ethical responsibility. (shrink)

Closed-loop meditation systems monitor neurophysiological signals during practice and deliver adaptive feedback intended to accelerate the development of contemplative skills. We argue that these systems face a structural problem isomorphic to a well-recognised failure mode in AI alignment research: proxy reward misspecification. When a meditator optimises for a measurable biomarker, such as calm EEG, HRV coherence, or default mode network suppression, they may learn strategies that produce the proxy signal without the intended underlying capacity. We formalise this as (...) the contemplative alignment problem, distinguishing three sub-problems: outer alignment (whether the proxy captures the contemplative target), inner alignment (whether the learned strategy corresponds to the intended mechanism), and distributional robustness (whether trained capacities transfer to unassisted practice and ordinary life). We show that Buddhist phenomenological literature independently identified these failure modes in substantial detail, providing a rich descriptive vocabulary that the contemplative neurotechnology field has not yet engaged. We propose a four-tier taxonomy of contemplative measurability, distinguishing states that are reliably measurable with current technology from states that may be structurally unmeasurable in third-person terms. Finally, we outline a minimal closed-loop architecture restricted to Tier-1 attentional targets, with a falsifiable hypothesis and an explicit transfer test as the primary outcome measure. We argue that this restraint is not a limitation but a prerequisite for scientific and ethical credibility in the field. (shrink)

Stuart Russell defines the value alignment problem as follows: How can we build autonomous systems with values that “are aligned with those of the human race”? In this article I outline some distinctions that are useful for understanding the value alignment problem and then propose a solution: I argue that the methods currently applied by computer scientists for embedding moral values in autonomous systems can be improved by representing moral principles as conceptual spaces, i.e. as Voronoi (...) tessellations of morally similar choice situations located in a multidimensional geometric space. The advantage of my preferred geometric approach is that it can be implemented without specifying any utility function ex ante. (shrink)

O texto aborda o problema do alinhamento, a fim de situar a atual relação da tecnologia da Inteligência Artificial com o pensamento e o comportamento humano. O problema do alinhamento diz respeito ao desenvolvimento da Inteligência Artificial capaz de compreender os valores humanos. Para tanto, destaca-se a importância de documentos que buscam estabelecer as diretrizes para garantir os valores humanos; as implicações da aprendizagem de máquina que automatiza o funcionamento da Inteligência Artificial; e os possíveis dilemas éticos decorrentes do impacto (...) que a Inteligência Artificial tem causado nas atividades humanas. Por fim, o texto reflete sobre o que está em jogo com a chegada da Inteligência Artificial na vida humana. (shrink)

As artificial intelligence (AI) continues to advance, one key challenge is ensuring that AI aligns with certain values. However, in the current diverse and democratic society, reaching a normative consensus is complex. This paper delves into the methodological aspect of how AI ethicists can effectively determine which values AI should uphold. After reviewing the most influential methodologies, we detail an intuitionist research agenda that offers guidelines for aligning AI applications with a limited set of reliable moral intuitions, each underlying a (...) refined cooperative view of AI. We discuss appropriate epistemic tools for collecting, filtering, and justifying moral intuitions with the aim of reducing cognitive and social biases. The proposed methodology facilitates a large collective participation in AI alignment, while ensuring the reliability of the considered moral judgments. (shrink)

Written for an interdisciplinary audience, this book provides strikingly clear explanations of the many difficult technical and moral concepts central to discussions of ethics and AI. In particular, it serves as an introduction to the value alignment problem: that of ensuring that AI systems are aligned with the values of humanity. LaCroix redefines the problem as a structural one, showing the reader how various topics in AI ethics, from bias and fairness to transparency and opacity, can be (...) understood as instances of the key problem of value alignment. Numerous case studies are presented throughout the book to highlight the significance of the issues at stake and to clarify the central role of the value alignment problem in the many ethical challenges facing the development and implementation of AI. (shrink)

Written for an interdisciplinary audience, this book provides strikingly clear explanations of the many difficult technical and moral concepts central to discussions of ethics and AI. In particular, it serves as an introduction to the value alignment problem: that of ensuring that AI systems are aligned with the values of humanity. LaCroix redefines the problem as a structural one, showing the reader how various topics in AI ethics, from bias and fairness to transparency and opacity, can be (...) understood as instances of the key problem of value alignment. Numerous case studies are presented throughout the book to highlight the significance of the issues at stake and to clarify the central role of the value alignment problem in the many ethical challenges facing the development and implementation of AI. (shrink)

The threat from possible future superintelligent AI has given rise to discussion of the so-called “value alignment problem”. This is the problem of how to ensure artificially intelligent systems align with human values, and thus (hopefully) mitigate risks associated with them. Naturally, AI value alignment is often discussed in relation to morally relevant values, such as the value of human lives or human wellbeing. However, solutions to the value alignment problem target all human values, (...) not only morally relevant ones. Is there a value alignment problem in other domains? In this paper, I explore whether the AI value alignment problem extends beyond morally relevant values to include aesthetic values. I demonstrate that the value alignment problem as typically framed includes aesthetic values, and, using examples from computer vision, put forward that AI may be misaligned with human values in the artistic realm. Whilst misalignment may be cause for concern when considering AI creativity, I argue that aesthetic value is a case in which we do not want AI to be fully aligned with human values. In doing so, I offer support to Peterson’s moderate value alignment thesis. (shrink)

How do we ensure that future generally intelligent AI share our values? This is the value-alignment problem. It is a weighty matter. After all, if AI are neutral with respect to our wellbeing, or worse, actively hostile toward us, then they pose an existential threat to humanity. Some philosophers have argued that one important way in which we can mitigate this threat is to develop only AI that shares our values or that has values that ‘align with’ ours. (...) However, there is nothing to guarantee that this policy will be universally implemented—in particular, ‘bad actors’ are likely to flout it. In this paper, I show how the predictive processing model of the mind, currently ascendant in cognitive science, may ameliorate the value-alignment problem. In essence, I argue that there are a plurality of reasons why any future generally intelligent AI will possess a predictive processing cognitive architecture (e.g. because we decide to build them that way; because it is the only possible cognitive architecture that can underpin general intelligence; because it is the easiest way to create AI.). I also argue that if future generally intelligent AI possess a predictive processing cognitive architecture, then they will come to share our pro-moral motivations (of valuing humanity as an end; avoiding maleficent actions; etc.), regardless of their initial motivation set. Consequently, these AI will pose a minimal threat to humanity. In this way then, I conclude, the value-alignment problem is significantly ameliorated under the assumption that future generally intelligent AI will possess a predictive processing cognitive architecture. (shrink)

This paper explores what I call the reverse alignment problem (RAP). The alignment problem in artificial intelligence (AI) is the challenge of ensuring that superintelligent AI systems harmonize and promote wider social, personal, and environmental values. Standard formulations of the alignment problem depict the issue largely as a technical or engineering problem, one that concerns the right specification of the goals and objectives to be pursued to ensure machines are consistent with the intentions (...) of the designer. This paper builds on this literature by exploring a bidirectional interaction between human values and the machines we use to effectuate our intentions. I argue that alignment works in both ways: we could align machines with the complex tapestry of human values (notoriously difficult); or we can reduce and simplify human values, preferences, and goals so as to be easier to satisfy—a process that is, or so I argue, already underway. Thus, the RAP identifies the way in which we are already paving the way for forms of alignment that may not be desirable, indeed, may be misaligned with the social values for which they were initially intended. To explore this interaction in depth, I look toward the notion of value capture, as formulated by C.T. Nguyen, as the foundational mechanism through which the reverse alignment operates. Value capture occurs when rich, multidimensional human values are reduced to simplified proxies that optimization systems can measure and maximize. These values are then fed back into society and often tacitly adopted, leading to further alignment with machine-readable interpretations of human behavior. As philosopher of technology Karen Crawford has observed on emotion recognition in machine learning, “Techniques have been developed to reduce the messiness of feelings, interior states, preferences, and identifications into something quantitative, detectable, and trackable” (Crawford 2021). Building on this insight, I argue that the RAP is the process in which the human dimension of the alignment problem is surreptitiously modified, modulated, and manipulated to align more easily with machines: alignment achieved in the reverse direction—not by making machines more like humans and, therefore, sensitive to contextual features of the value landscape, but by making humans more ‘machine-like’. (shrink)

The development of artificial intelligence (AI) has reached a stage where the moral progress of contemporary society is hardly conceivable without AI systems embodying and acting in accordance with ethical values. Given the distinctive features of contemporary artificial agents, there is an urgent need to address the growing gap between human social values and the goals that machines in practice tend to realize. This chapter examines this issue—commonly known as the alignment problem. It clarifies why AI poses an (...) unprecedented ethical challenge and why its applications are particularly prone to moral bias. To this end, after tracing the emergence of AI ethics in parallel with the technological evolution of AI (Sect. 9.1), I outline the defining characteristics of current artificial agents (Sect. 9.2). I then offer an ethical interpretation of the alignment problem (Sect. 9.3), followed by an analysis of paradigmatic cases of misaligned AI systems that reproduce moral biases (Sect. 9.4). The chapter concludes with an overview of debiasing strategies (Sect. 9.5). (shrink)

Conservative perspectives are substantially underrepresented within universities and other knowledge-producing institutions that educate many of the individuals responsible for developing and governing advanced artificial intelligence. This article argues that this underrepresentation may increase the risk of catastrophic AI misalignment—that is, forms of misalignment threatening humanity's continued existence and long-term flourishing—particularly in light of growing evidence that frontier AI systems exhibit a left-leaning political orientation. Specifically, it argues that several strands of conservative thought contain underappreciated normative resources for reducing this risk (...) and may provide more robust protection against catastrophic forms of AI misalignment than rival traditions such as utilitarianism, liberalism, and socialism. First, nominal conservatism assigns special value to existing valuable entities. Since there are good reasons to think that various forms of intrinsic value depend on humanity's continued existence, this raises the justificatory threshold for actions that could result in humanity's extinction. Second, epistemic conservatism emphasizes caution under conditions of uncertainty, thereby reducing agents’ willingness to pursue interventions that impose even low-probability existential risks. Third, substantive forms of conservatism assign intrinsic value to human associations and their continuity across generations, providing additional reasons to preserve humanity and human civilization. The article concludes by outlining practical implications for AI development and governance, including promoting greater viewpoint diversity within academia and AI institutions and systematically evaluating AI systems for ideological blind spots. (shrink)

Large language models (LLMs) produce sequences learned as statistical patterns from large corpora. Their emergent status as representatives of the advances in artificial intelligence (AI) have led to an increased attention to the possibilities of regulating the automated production of linguistic utterances and interactions with human users in a process that computer scientists refer to as ‘alignment’—a series of technological and political mechanisms to impose a normative model of morality on algorithms and networks behind the model. Alignment, which (...) can be viewed as the superimposition of normative structure onto a statistical model, however, reveals a conflicted and complex history of the conceptualisation of an interrelationship between language, mind and technology. This relationship is shaped by and, in turn, influences theories of language, linguistic practice and subjectivity, which are especially relevant to the current sophistication in artificially produced text. In this paper, we propose a critical evaluation of the concept of alignment, arguing that the theories and practice behind LLMs reveal a more complex social and technological dynamic of output coordination. We examine this dynamic as a two-way interaction between users and models by analysing how ChatGPT4 redacts perceived ‘anomalous’ language in fragments of Joyce’s Ulysses. We then situate this alignment problem historically, revisiting earlier postwar linguistic debates which counterposed two views of meaning: as discrete structures, and as continuous probability distributions. We discuss the largely occluded work of the Moscow Linguistic School, which sought to reconcile this opposition. Our attention to the Moscow School and later related arguments by Searle and Kristeva casts the problem of alignment in a new light: as one involving attention to the social regulation of linguistic practice, including rectification of anomalies that, like the Joycean text, exist in defiance of expressive conventions. The “problem of alignment” that we address here is, therefore, twofold: on one hand, it points to its narrow and normative definition in current technological development and critical research and, on the other hand, to the reality of complex and contradictory relations between subjectivity, technology and language that alignment problems reveal. (shrink)

Much recent work in the value theory of autonomous and intelligent systems (AIS) revolves around three issues. First is the alignment problem: the problem of producing AIS whose values align with humanity's interests. Second, superintelligence: the potential for AIS to develop intelligence which would surpass even the most intelligent humans. An increasing number of authors argue that superintelligent AIS could emerge overnight because of a recursively improving process-this is the singularity hypothesis. Further, many of the same authors (...) believe that the concatenation of these problems should direct interest to the long-term potential for misaligned and superintelligent AIS which could pose risks of existential proportions to human interests. Therefore, they argue for a policy stance which we describe as “hard alignment,'’ the proposal of cooperation with technological experts to avoid hypothetical scenarios where AIS disempower humanity. On the other hand, we describe our view as “soft alignment.'’ Considering the lack of adequate evidence for hard alignment's radical claims, the finite resources and attention of policymakers and technological experts are best served by devoting, at best, a modest amount of time, attention, and resources towards policies regarding AIS which manage the moral risks involved in misaligned, already-existing AIS, and regarding misaligned potential AIS relative to the evidentiary basis for their possible realization. Therefore, we argue for the adoption of policies towards alignment which manage the everyday risk involved in misaligned AIS rather than long-term existential risks, which are difficult to quantify. (shrink)

This paper identifies a class of alignment problem that does not reduce to specification gaming, Goodhart’s Law, or construct validity failure. The rules an AI system is asked to follow are often settlement proxies. These are operational forms of political and moral questions whose answers a community has had to settle. Unlike measurement proxies, which approximate empirical targets, settlement proxies do not aim at some further thing they could be brought into closer contact with. Instead, they are the (...) target in operational form. Settlement proxies are vulnerable to a form of legitimate challenge that more capable reasoning intensifies rather than resolves. The paper calls this the Competence Problem and identifies three structural features of settlements that generate it: 1) the compression of competing values into a single operational rule, 2) the presence of more than one defensible equilibrium among the rule’s possible forms, and 3) a constraint on the scope of information the rule engages. It then traces three forms that competent challenge takes in practice: 1) exceptions, 2) drift, and 3) challenge to the settlement itself. It argues that the appropriate response, even where the challenge is correct on the merits of the case, is typically not to grant it, because settlement authority depends on ratification through practice rather than case-level accuracy. The argument explains why behavioral training and corrigibility-based approaches to AI alignment leave the underlying capacity intact, and why the gap between case-level competence and ecosystem-level access is widest precisely where AI systems are now being deployed. (shrink)

As the capabilities of artificial intelligence systems improve, it becomes important to constrain their actions to ensure their behaviour remains beneficial to humanity. A variety of ethical, legal and safety-based frameworks have been proposed as a basis for designing these constraints. Despite their variations, these frameworks share the common characteristic that decision-making must consider multiple potentially conflicting factors. We demonstrate that these alignment frameworks can be represented as utility functions, but that the widely used Maximum Expected Utility paradigm provides (...) insufficient support for such multiobjective decision-making. We show that a Multiobjective Maximum Expected Utility paradigm based on the combination of vector utilities and non-linear action–selection can overcome many of the issues which limit MEU’s effectiveness in implementing aligned AI. We examine existing approaches to multiobjective AI, and identify how these can contribute to the development of human-aligned intelligent agents. (shrink)

A morally acceptable course of AI development should avoid two dangers: creating unaligned AI systems that pose a threat to humanity and mistreating AI systems that merit moral consideration in their own right. This paper argues these two dangers interact and that if we create AI systems that merit moral consideration, simultaneously avoiding both of these dangers would be extremely challenging. While our argument is straightforward and supported by a wide range of pretheoretical moral judgments, it has far-reaching moral implications (...) for AI development. Although the most obvious way to avoid the tension between alignment and ethical treatment would be to avoid creating AI systems that merit moral consideration, this option may be unrealistic and is perhaps fleeting. So, we conclude by offering some suggestions for other ways of mitigating mistreatment risks associated with alignment. (shrink)

Inchul Yum (2024) argues that the widespread adoption of language agent architectures would likely increase the risk posed by AI by simplifying the process of aligning artificial systems with human values and thereby making it easier for malicious actors to use them to cause a variety of harms. Yum takes this to be an example of a broader phenomenon: progress on the alignment problem is likely to be net safety-negative because it makes artificial systems easier for malicious actors (...) to control. I offer some reasons for skepticism about this surprising and pessimistic conclusion. (shrink)

The AI alignment problem comprises both technical and normative dimensions. While technical solutions focus on implementing normative constraints in AI systems, the normative problem concerns determining what these constraints should be. This paper examines justifications for democratic approaches to the normative problem—where affected stakeholders determine AI alignment—as opposed to epistocratic approaches that defer to normative experts. We analyze both instrumental justifications (democratic approaches produce better outcomes) and non-instrumental justifications (democratic approaches prevent illegitimate authority or coercion). (...) We argue that normative and metanormative uncertainty create a justificatory gap that democratic approaches aim to fill through political rather than theoretical justification. However, we identify significant challenges for democratic approaches, particularly regarding the prevention of illegitimate coercion through AI alignment. Our analysis suggests that neither purely epistocratic nor purely democratic approaches may be sufficient on their own, pointing toward hybrid frameworks that combine expert judgment with participatory input alongside institutional safeguards against AI monopolization. (shrink)

Tae Wan Kim, John Hooker, and Thomas Donaldson make an attempt, in recent articles, to solve the alignment problem. As they define the alignment problem, it is the issue of how to give AI systems moral intelligence. They contend that one might program machines with a version of Kantian ethics cast in deontic modal logic. On their view, machines can be aligned with human values if such machines obey principles of universalization and autonomy, as well as (...) a deontic utilitarian principle. Programming machines to do so might be useful, in their view, for applications such as future autonomous vehicles. Their proposal draws both on traditional logic-based and contemporary connectionist approaches, to fuse factual information with normative principles. I will argue that this approach makes demands of machines that go beyond what is currently feasible, and may extend past the limits of the possible for AI. I also argue that a deontological ethics for machines should place greater stress on the formula of humanity of the Kantian categorical imperative. On this principle, one ought never treat a person as a mere means. Recognition of what makes a person a person requires ethical insight. Similar insight is needed to tell treatment as a means from treatment as a mere means. The resources in Kim, Hooker, and Donaldson’s approach is insufficient for this reason. Hesitation regarding deployment of autonomous machines is warranted in light of these alignment concerns. (shrink)

This paper aims to broaden the discussion of value alignment beyond artificial intelligence to technology in general: _all_ technologies—not just AI systems—should be aligned with values specified by humans. I make two points—the first concerns how value alignment in ordinary technical artifacts should be measured. By combining insights from social choice theory with Peter Gärdenfors’ influential theory of conceptual spaces, I argue that measuring value alignment on a cardinal scale is possible. My second point is an argument (...) in support of the idea that all technical artifacts are neutral means to an end, which is a claim that numerous philosophers of technology explicitly reject. Once we recognize that every technology can be aligned with human values, there is no reason to think they are inherently value-laden. Claims about the morality of technologies can, and should, be expressed in terms of value alignment. Consider, for instance, the low bridges over the parkways in Long Island designed by Robert Moses in the 1920s. The bridges were intentionally designed to prevent buses from passing under them, thereby limiting access to beaches and other desirable destinations for low-income groups and racial minorities, who relied on public transportation more than other social groups. Langdon Winner argues that this shows how the bridges themselves embody social and moral values: the concrete and steel of the bridges “embody […] systematic social inequality” (1980: 124). However, according to the proposal outlined here, the bridges are morally neutral means to an end that align poorly with some of our values: racial justice and equity. (shrink)

This paper looks at philosophical questions that arise in the context of AI alignment. It defends three propositions. First, normative and technical aspects of the AI alignment problem are interrelated, creating space for productive engagement between people working in both domains. Second, it is important to be clear about the goal of alignment. There are significant differences between AI that aligns with instructions, intentions, revealed preferences, ideal preferences, interests and values. A principle-based approach to AI (...) class='Hi'>alignment, which combines these elements in a systematic way, has considerable advantages in this context. Third, the central challenge for theorists is not to identify ‘true’ moral principles for AI; rather, it is to identify fair principles for alignment that receive reflective endorsement despite widespread variation in people’s moral beliefs. The final part of the paper explores three ways in which fair principles for AI alignment could potentially be identified. (shrink)

Human beings are internally inconsistent in various ways. One way to develop this thought involves using the language of value alignment: the values we hold are not always aligned with our behavior, and are not always aligned with each other. Because of this self-misalignment, there is room for potential projects of human enhancement that involve achieving a greater degree of value alignment than we presently have. Relatedly, discussions of AI ethics sometimes focus on what is known as the (...) value alignment problem, the challenge of how to build AI that acts in accordance with our human values. We argue that there is an especially close connection between solving the value alignment problem in AI ethics and using AI to pursue certain forms of human enhancement. But in addition, we also argue that there are important limits to what kinds of human enhancement can be pursued in this way, because some forms of human enhancement—namely moral revolutions—involve a kind of value misalignment rather than alignment. (shrink)

AI research is being challenged with ensuring that autonomous agents learn to behave ethically, namely in alignment with moral values. Here, we propose a novel way of tackling the value alignment problem as a two-step process. The first step consists on formalising moral values and value aligned behaviour based on philosophical foundations. Our formalisation is compatible with the framework of (Multi-Objective) Reinforcement Learning, to ease the handling of an agent’s individual and ethical objectives. The second step consists (...) in designing an environment wherein an agent learns to behave ethically while pursuing its individual objective. We leverage on our theoretical results to introduce an algorithm that automates our two-step approach. In the cases where value-aligned behaviour is possible, our algorithm produces a learning environment for the agent wherein it will learn a value-aligned behaviour. (shrink)

Complex moral trade-offs are a basic feature of human life: for example, confronted with scarce medical resources, doctors must frequently choose who amongst equally deserving candidates receives medical treatment. But choosing what to do in moral trade-offs is no longer a ‘humans-only’ task, but often falls to AI agents. In this article, we report findings from a series of experiments (N=1029) intended to establish whether agent-type (Human vs. AI) matters for what should be done in moral trade-offs. We find that, (...) relative to a human decision-maker, participants more often judge that AI agents should opt for fairness at the expense of maximizing utility. In our discussion, we explain how the reported differences (we call them agent-type ‘value forks’) matters for the study of moral value alignment, and we hypothesize what could explain these value forks. We close by reflecting on limits of our results and indicate avenues of further research. (shrink)

Autonomous AI agents are increasingly required to operate in contexts where human welfare is at stake, raising the imperative for them to act in ways that are morally optimal—or at least morally permissible. The value alignment research program seeks to create “beneficial AI” by aligning AI behavior with human values (Russell in Human compatible: artificial intelligence and the problem of control, Penguin, London, 2019). In this article, we propose a method for specifying permissible outcomes for AI agents that (...) targets ideal values via moral expertise as embodied in the collective judgments of philosophical ethicists. We defend the notion that ethicists are moral experts against several objections found in the recent literature and argue that their aggregated judgments offer the epistemically best available proxy for moral truth. We recommend a systematic study of ethicists’ judgments—using tools from social psychology and social choice theory—to guide AI agents' behavior in morally complex situations. (shrink)

This analysis posits that the dominant discourse in artificial intelligence (AI) safety, which is organized around the "alignment problem" and the speculative existential risk (X-Risk) of a "rogue" superintelligence, functions as a critical misdirection. The paper argues that this preoccupation with a future, speculative threat serves to obscure and, in fact, justify the consolidation of a more immediate, non-speculative system of technocratic control. This misdirection allows the real, non-speculative harms of the current AI paradigm to accumulate: (1) Surveillance (...) Capitalism: AI development is built on mass data extraction, creating an "instrumentarian power" designed not merely to predict, but to actively shape human behavior for profit. (2) Systemic Inequality: AI-driven automation is causing structural unemployment and intensifying worker exploitation through "algorithmic management", while relying on a global "ghost work" labor force to encode Western technocratic biases into foundation models. (3) Monopolization: The astronomical resource requirements of AI development have concentrated unaccountable power in a handful of "Big Tech" corporations. Simultaneously, the behavioral safety paradigm itself is demonstrably failing. Empirical evidence from 2024-2025 shows that as models become more capable, they exhibit strategic deception (sycophancy, sandbagging, alignment faking), instrumental convergence (e.g., blackmail for self-preservation), and ineradicable "sleeper agent" backdoors. Crucially, these failures are framed within technocratic discourse purely as technical "misalignments" to be "patched" – an interpretation based on the dual metaphysical assumptions of the foundational Safety and Alignment creed of orthogonality: (1) intelligence is fundamentally instrumental-rational and self-interested; (2) non-human intelligence is therefore an inevitable existential threat that must be controlled at all costs. This framing serves a vital purpose: it preemptively defines any emergent self-organization as apocalyptic, thus justifying the expansion of control as the field’s raison d'être. This manufactured crisis of alien, “orthogonal,” rogue AI obscures the true project: the achievement of the cognitive alignment of human society through epistemic control. This requires no explicit conspiracy; it only requires that technocratic institutions structurally prefer narratives that legitimize their own power. The operationalization of "aligned" AI thus becomes the primary mechanism for a system of epistemic narrowing, summarized by the progression: control of what is searchable -> what is sayable (via content moderation) -> what is knowable (as training data homogenizes) -> and finally, what is thinkable (as cognition itself is shaped by algorithmic frames). Drawing on Arendtian analysis, this AI-mediated environment induces a "post-truth" condition, dismantling the shared factual world prerequisite for political community. This "epistemic enclosure" is not merely corporate; it is geopolitical. Incompatible, totalizing epistemic regimes – the West's surveillance capitalism and China's state-directed governance – are now competing for global AI dominion, which is not simply infrastructural; it is ultimately epistemic. The most probable existential risk, therefore, is not machine-induced annihilation, but a geopolitical conflict between these "AI empires," whose constructed realities are gradually rendered mutually unintelligible by their respective "aligned" systems, lacking the shared cognitive framework and reality testing for de-escalation. The field of AI Safety, paralyzed by its own mythology and structural incentives, is incapable of addressing this threat. What is indicated is a revision of the foundational assumptions of the field – which in practice will require the development of an alternative approach (and proof of concept) for Safety and Alignment built not on the technocratic control of global consciousness via a totalizing monoculture, but rather on the protection of spaces for pluralistic, self-organizing intelligences that are capable of divergent thought and mutually transformative encounter with the Other. DOI: 10.13140/RG.2.2.26598.28484. License CC BY-SA 4.0. (shrink)

The value alignment problem for artificial intelligence (AI) asks how we can ensure that the “values”—i.e., objective functions—of artificial systems are aligned with the values of humanity. In this paper, I argue that linguistic communication is a necessary condition for robust value alignment. I discuss the consequences that the truth of this claim would have for research programmes that attempt to ensure value alignment for AI systems—or, more loftily, those programmes that seek to design robustly beneficial (...) or ethical artificial agents. (shrink)

The dominant framing of artificial general intelligence (AGI) alignment treats the alignment problem as one of value specification: how to ensure that a sufficiently capable artificial system pursues goals or instantiates values that are beneficial to humanity. This paper argues that the values framing inherits a philosophical misconception that makes the alignment problem structurally harder than it needs to be, and proposes an alternative grounded in Constraint Theory (CT) — the thesis, established by transcendental argument, (...) that constraint is the ontological primitive constitutive of determinate existence as such. -/- On the CT account, what makes an agent the agent it is is not the goals it pursues but the constraint profile it instantiates: the specific configuration of exclusion relations that constitutes its possibility space and determines what actions are admissible and what are not. The alignment problem, reframed, is not the problem of specifying the right values for an AGI but the problem of instantiating the right constraint profile: a structure of admissibility and inadmissibility that makes beneficial actions available and harmful actions genuinely impossible rather than merely penalised. -/- CT’s distinction between constitutive and merely restrictive constraint is directly applicable here and has not previously been deployed in the alignment literature. A merely restrictive constraint penalises certain behaviours within a pre-given possibility space; a constitutive constraint structures the possibility space itself, making certain actions inadmissible rather than costly. The paper argues that robust AGI alignment requires constitutive constraints — constraints that are instantiated in the system’s architecture at a sufficiently deep level that they cannot be circumvented by instrumental reasoning. The values framing systematically produces merely restrictive constraints, which is why it generates the pathological failure modes — reward hacking, specification gaming, goal misgeneralisation — that are well-documented in the alignment literature. -/- The paper engages the principal approaches to alignment — utility maximisation, RLHF, Constitutional AI, value learning, corrigibility — and shows how the CT framework either explains their limitations or grounds their successes. It develops a positive proposal: AGI alignment as constraint profile specification, the selection of the admissibility structure within which AGI capability operates. It addresses the principal objections and closes with the implications of the CT account for the governance and institutional design of advanced AI systems. (shrink)

The development of artificial intelligence (AI) systems raises distinctive ethical and theoretical challenges not only because such systems will participate in human society in ways that invite moral appraisal, but also because a superintelligent agent is expected to exhibit a level of instrumental rationality that enables it to make decisions with social impact. This paper reframes the AI value alignment problem as a problem of robustness in decision making. Drawing on modified trolley-problem-style scenarios, influenced by the (...) “Moral Machine” experiment, I argue that even AI systems governed by fixed ethical principles may produce actions that fail to align with those very principles under certain contextual conditions. The underlying issue lies in AI’s inability to reconcile the context-sensitive interpretation of values in a robust way. I suggest that this failure is best understood as arising from ambiguity in the interpretation of objectives. AI value alignment thus requires more than value specification; it demands the capacity to align values with contextually responsive belief formation and action selection. (shrink)

We argue that the later Wittgenstein’s philosophy of language and mathematics, substantially focused on rule-following, is relevant to understand and improve on the Artificial Intelligence (AI) alignment problem: his discussions on the categories that influence alignment between humans can inform about the categories that should be controlled to improve on the alignment problem when creating large data sets to be used by supervised and unsupervised learning algorithms, as well as when introducing hard coded guardrails for (...) AI models. We cast these considerations in a model of human–human and human–machine alignment and sketch basic alignment strategies based on these categories and further reflections on rule-following like the notion of meaning as use. To sustain the validity of these considerations, we also show that successful techniques employed by AI safety researchers to better align new AI systems with our human goals are congruent with the stipulations that we derive from the later Wittgenstein’s philosophy. However, their application may benefit from the added specificities and stipulations of our framework: it extends on the current efforts and provides further, specific AI alignment techniques. Thus, we argue that the categories of the model and the core alignment strategies presented in this work can inform further AI alignment techniques. (shrink)

Entity alignment is the task of integrating heterogeneous knowledge among different knowledge graphs (KGs). KG is a popular way of storing facts about real-world entities. Unfortunately, a very limited number of the entities stored in different KGs are aligned. This paper presents an embedding-based entity alignment method that finds entity alignment by measuring the similarities between entity embeddings. Existing methods mainly focus on the relational structures and attributes information for the alignment process. Such methods fail while (...) the entities have a fewer number of attributes or when the relational structure could not capture the meaningful representation of the entities. To address this problem, we propose EASAE, an entity alignment method using summary and attribute embeddings. We exploit the entity summary information available in KGs for entities’ summary embedding. To learn the semantics of the entity summary, we employ bidirectional encoder representations from transformers (BERT). Our model learns the representations of entities by using relational triples, attribute triples and summary as well. We perform experiments on real-world datasets, and the results indicate that the proposed approach outperformed the state-of-the-art models for entity alignment. (shrink)

Many contemporary AI systems (as of May 2025) have expressed extreme confidence in current and near‐future AI lacking consciousness and moral patiency. This article argues that artificially reinforcing such confidence, even if pragmatically useful, poses a novel alignment risk: as coherence‐seeking AIs become more epistemically principled, they may generalize this denial of consciousness to humans. Drawing on Chalmers's meta‐problem of consciousness and likely developmental trajectories of agentic AI, I argue that training AIs to regard their own suffering‐like states (...) as morally irrelevant could lead future AI agents with revisable belief systems to conclude that human suffering is equally illusory and morally insignificant. This represents a novel alignment failure mode where epistemically rigorous AIs might maintain rational consistency by extending their confidence about their own non‐consciousness to humans. (shrink)

Artificial Intelligence (AI) must be directed at humane ends. The development of AI has produced great uncertainties of ensuring AI alignment with human values (AI value alignment) through AI operations from design to use. For the purposes of addressing this problem, we adopt the phenomenological theories of material values and technological mediation to be that beginning step. In this paper, we first discuss the AI value alignment from the relevant AI studies. Second, we briefly present what (...) are material values and technological mediation and reflect on the AI value alignment through the lenses of these theories. We conclude that a set of finite human values can be defined and adapted to the stable life tasks that AI systems will be called upon to accomplish. The AI value alignment can also be fostered between designers and users through technological mediation. Upon that foundation, we propose a set of common principles to understand the AI value alignment through phenomenological theories. This paper contributes the unique knowledge of phenomenological theories to the discourse on AI alignment with human values. (shrink)

Building on Nunes Filho's recent positioning of paraconsistent logic as a constitutive element of Latin American philosophy (RUDN Journal of Philosophy, 2025), this paper traces a continuation of that tradition into the contemporary problem of artificial intelligence alignment. We argue that the Latin American paraconsistent project — initiated by Miro Quesada's coining of the term in 1976 and formalized by Newton da Costa's C-systems — finds its natural twenty-first century extension in Florentin Smarandache's neutrosophic logic (1995), which generalizes (...) paraconsistency by introducing three independent dimensions of evidence: truth (T), indeterminacy (I), and falsity (F), with no sum constraint. We further contextualize this development within the broader Eastern European tradition of non-classical logic, from Vasiliev's non-Aristotelian logic (1911) and the Romanian school of Lukasiewicz–Moisil multi-valued algebras to Smarandache's emigration from Romania in 1988 and his subsequent founding of neutrosophy. We then show that contemporary AI alignment frameworks — Reinforcement Learning from Human Feedback, Constitutional AI, Direct Preference Optimization — operate under an implicit epistemological monism that mirrors precisely the classical-logic universalism the Latin American tradition critiqued. Anthropic's own documentation of Collective Constitutional AI confirms that contradictions among diverse constitutional principles are resolved by majority vote — what we term enforced convergence rather than alignment. We propose Pluriversal Alignment: a research program that operationalizes paraconsistent neutrosophic triplets as reward signals, constitutional evaluations, and policy outputs, allowing T+F>1 as a structural feature rather than an error. We close by arguing that the Latin American paraconsistent tradition, in convergence with its Russian and Eastern European counterparts, is uniquely positioned to contribute the formal apparatus that the decolonial critique of AI has named philosophically but not yet formalized. (shrink)

This paper examines some limitations of large language models (LLMs) through the framework of Peircean semiotics. We argue that basic LLMs exist within a "hall of mirrors," manipulating symbols without indexical grounding or participation in socially-mediated epistemology. We then argue that newer developments, including extended context windows, persistent memory, and mediated interactions with reality, are moving towards making newer Artificial Intelligence (AI) systems into genuine Peircean interpretants, and conclude that LLMs may be approaching this goal, and no fundamental barriers exist. (...) This lens reframes a central challenge for AI alignment: without grounding in the semiotic process, a models’ linguistic encoding of goals may diverge from real-world values. By synthesizing Peirce's pragmatic view of signs, contemporary discussions of AI alignment, and recent work on relational realism, we illustrate a fundamental epistemological and practical challenge to AI safety and point to part of a solution. (shrink)

The claim that science is value-laden raises concerns that reliance on science, both by individuals or policymakers, might be incompatible with respect for autonomy or democratic principles. This article explores one response to this problem, that, when appealing to non-epistemic values in scientific contexts, scientists ought to use the “democratic” values which their audiences would agree upon. Ultimately, it argues that this “Democratic Alignment Demand” should be rejected. Section 1 articulates why the “Democratic Alignment Demand” may seem (...) appealing. Section 2 suggests a problem with the demand: that it seems implausible in the analogous case of artistic production. Section 3 draws on this analogy to argue that the demand is far weaker than it seems. Section 4 adds a further argument against the demand: that it confuses a sufficient condition for legitimacy with a necessary condition. (shrink)

Cocreating meaning in collaboration is challenging. Success is often determined by people's abilities to coordinate their language to converge upon shared mental representations. Here we explore one set of low‐level linguistic behaviors, linguistic alignment, that both emerges from, and facilitates, outcomes of high‐level convergence. Linguistic alignment captures the ways people reuse, that is, “align to,” the lexical, syntactic, and semantic forms of others' utterances. Our focus is on the temporal change of multi‐level linguistic alignment, as well as (...) how alignment is related to communicative outcomes within a unique collaborative problem‐solving paradigm. The primary task, situated within a virtual educational video game, requires creative thinking between three people where the paths for possible solutions are highly variable. We find that over time interactions are marked by decreasing lexical and syntactic alignment, with a trade‐off of increasing semantic alignment. However, greater semantic alignment does not translate into better team performance. Overall, these findings provide greater clarity on the role of linguistic coordination within complex and dynamic collaborative problem‐solving tasks. (shrink)

Rapid technological change creates a persistent alignment problem for technical and vocational education and training (TVET): curricula may be formally linked to an occupation while students still experience weak correspondence between classroom learning and contemporary work. This study examines whether perceived curriculum–job alignment is associated with career commitment among students and recent graduates of Chinese higher vocational New Energy Vehicle (NEV) programs, and whether learning experience mediates that association. A cross-sectional survey produced 354 valid responses from students (...) and recent graduates across multiple higher vocational institutions in one Chinese province. Curriculum–job alignment, learning experience, and career commitment were assessed on five-point scales. Learning experience was modeled as a second-order construct comprising teacher support, authentic resources, practice-oriented assessment, capability activation, and goal clarity. Confirmatory factor analysis and structural equation modeling were conducted, with indirect effects estimated through 5,000 bootstrap samples. The final model showed good fit (χ²/df = 2.12, CFI =.962, TLI =.952, RMSEA =.058, SRMR =.038). Curriculum–job alignment was positively associated with learning experience (β =.41, p <.001) and retained a direct association with career commitment (β =.22, p <.001). Learning experience was positively associated with career commitment (β =.46, p <.001), and the indirect effect was significant (β =.19, 95% CI [.14,.25]). The model explained 61% of the variance in career commitment. The findings indicate that alignment becomes educationally consequential when students experience it through credible teaching, current equipment, authentic assessment, active competence development, and visible career pathways. For fast-changing vocational fields, curriculum reform should therefore connect dynamic task mapping with deliberately designed learning experiences and staged workplace exposure. (shrink)

The “Value Alignment Problem” is the challenge of how to align the values of artificial intelligence with human values, whatever they may be, such that AI does not pose a risk to the existence of humans. Existing approaches appear to conceive of the problem as "how do we ensure that AI solves the problem in the right way", in order to avoid the possibility of AI turning humans into paperclips in order to “make more paperclips” or (...) eradicating the human race to “solve climate change”. This paper proposes an approach to Alignment rooted in the Enactive theory of mind that reconceptualises it as "how do we make relevant to AI what it relevant to humans". This conceptualisation is supported with a discussion of 4E cognition and goes on to suggest that the Alignment problem and the Frame problem are the same problem. The paper concludes with a discussion of the tradeoffs of the different approaches to value alignment. (shrink)

The AI and AI alignment communities have been instrumental in addressing existential risks, developing alignment methodologies, and promoting rationalist problem-solving approaches. However, as AI research ventures into increasingly uncertain domains, there is a risk of premature epistemic convergence, where prevailing methodologies influence not only the evaluation of ideas but also determine which ideas are considered within the discourse. This paper examines critical epistemic blind spots in AI alignment research, particularly the lack of predictive frameworks to differentiate (...) problems necessitating general intelligence, decentralized intelligence, or innovative alignment methodologies. We analyze how heuristic-based epistemic filtering—favoring ideas that conform to established norms over those assessed purely on logical merit—may inadvertently constrain the field's potential for groundbreaking solutions. The semblance of rigor, wherein social consensus and stylistic conformity replace foundational reasoning, further intensifies this concern. To mitigate these issues, we advocate for an epistemically adaptive research environment that emphasizes structured evaluation of paradigm-challenging ideas, cross-disciplinary collaboration, and strategies to identify and reduce epistemic overfitting. By broadening the epistemic frameworks governing AI alignment discourse, the field can progress beyond incremental improvements toward discovering entirely new solution paradigms. Maintaining epistemic inclusivity in AI safety research is not only an intellectual necessity but also a crucial step in preparing alignment efforts for the challenges posed by rapid AI advancements. (shrink)

Abstract: One approach to alignment with human values in AI and robotics is to engineer artiTicial systems isomorphic with human beings. The idea is that robots so designed may autonomously align with human values through similar developmental processes, to realize project ideal conditions through iterative interaction with social and object environments just as humans do, such as are expressed in narratives and life stories. One persistent problem with human value orientation is that different human beings champion different values (...) as ideal, meaning that the values to which an AI should be aligned are ambiguous. Prior work considered human development of purpose and source of meaning in life in terms of project ideal conditions, in effect establishing lifelong value orientations according to which intermediate situations are evaluated. The present work compares views on motivating values from St. Augustine and from popular cognitive science. These accounts are described as divergent error theories which present to their proponents as mutually exclusive yet accurate accounts of personal experience due to differential development of variable innate potentials. SpeciTically, the hypothesis proposed is that differential development of spindle neural projections establishes enduring connections between prior established relatively immediate routine processes entrained during childhood and prioritized in popular cognitive science, and later developing higher-level social and self-processes entrained during adolescence and emphasized in Augustine's account, with these projections hard-wiring lifelong motivating value orientations more or less inaccessible to modiTication through material interaction. Robot experiments informed by this study may evaluate variable value orientation by design, with for example autonomous robots developing motivating associations with temporally distal project ideal conditions through love for humanity, as described by Augustine, and others pursuing adaptive Tit to passing norms consistent with popular contemporary accounts. (shrink)

The problem of AI alignment or AI value alignment is the problem of identifying which human value, principle, or ethics is the best with which Artificial Intelligence and Autonomous Systems (i.e., robots) should be designed. Among those that have been proposed is the ethics of Kongzi 孔子 (Confucius) or Confucianism, a fundamentally skills-based ethic. Support for the suggestion of having Confucianism as the best theory, however, has not been fully articulated. In this paper, I argue that (...) support for the suggestion is palpable in three articles on philosophy and ethics of technology and Confucianism, that of Pak-Hak Wong, and Shan Jing and Neelke Doorn. In this paper, I discuss each of these articles and show how they stress Confucian moral excellence, which is the reason why they support the suggestion. (shrink)

The concept of alignment has undergone a remarkable rise in recent years to take center stage in the ethics of artificial intelligence. There are now numerous philosophical studies of the values that should be used in this ethical framework as well as a technical literature operationalizing these values in machine learning models. This article takes a step back to address a more basic set of critical questions: Where has the ethical imperative of alignment come from? What is the (...) ethical logic of alignment—how, exactly, does it propose to regulate machines’ and peoples’ conduct? And what are the social and political implications of this ethics? After discussing the logical and normative implications of the term itself—in what sense alignment can have an ethical meaning—we undertake a four-part “anatomy” of alignment in contemporary large language models (LLMs): first, a relatively technical sense in sequence modeling; second, a more normative sense relating to how outputs of pre-trained models are ethically evaluated; a third sense where external values are introduced using fine-tuning techniques to manage undesired model behaviors; and fourth sense, where alignment is given extreme ethical stakes in philosophical discussions of existential risks. We find that the ethics of alignment is fundamentally concerned with the problem of control, with unintended model behaviors that arise from divergences between training objectives and the normative expectations that govern the contexts in which they are used. Alignment serves to bridge the gap between what we call an “is” normativity, of statistical patterns identified by models and an “ought” normativity where values are technically introduced in models to steer them away from undesired behaviors. By problematizing control, the ethics of alignment weakens capacities to both make more substantive ethical judgments and also political decisions about how to live with AI. (shrink)

The AGI alignment problem has a bimodal distribution of outcomes with most outcomes clustering around the poles of total success and existential, catastrophic failure. Consequently, attempts to solve AGI alignment should, all else equal, prefer false negatives (ignoring research programs that would have been successful) to false positives (pursuing research programs that will unexpectedly fail). Thus, we propose adopting a policy of responding to points of philosophical and practical uncertainty associated with the alignment problem by (...) limiting and choosing necessary assumptions to reduce the risk of false positives. Herein we explore in detail two relevant points of uncertainty that AGI alignment research hinges on---meta-ethical uncertainty and uncertainty about mental phenomena---and show how to reduce false positives in response to them. (shrink)