Abstract

Hallucination in Large Language Models (LLMs) is a persistent and challenging phenomenon. These hallucinations manifest when LLMs produce outputs that are plausible but factually inaccurate or logically inconsistent. Old hallucinations in LLMs could be categorized into three categories: factual hallucination, contextual hallucination, and logical hallucination. Key contributing factors include limitations in training data, the absence of ground-truth verification mechanisms, and tendencies toward overgeneralization and extrapolation. To address these issues, researchers have proposed various mitigation strategies, including learning-based techniques, post-training interventions and knowledge transfer, and safety and alignment enhancements. Despite these efforts, residual hallucination risks persist, thereby raising profound challenges from an epistemological perspective. This paper examines these challenges through the lens of contemporary epistemology. Specifically, it explores how residual hallucinations in LLMs pose difficulties for knowledge-first and anti-Gettier accounts of knowledge. It analyzes the epistemic limitations of LLMs by engaging the debate between internalism and externalism in model justification, and addresses the epistemic boundedness of residual hallucinations in relation to grounding and post-training strategies. Additionally, it evaluates the epistemic constraints inherent in aligning LLMs with safety goals in the absence of justification, and considers an embodied epistemology of hallucination risks. Finally, the paper highlights the role of sycophancy in contributing to these residual epistemological risks. By integrating philosophical analysis with empirical developments in LLMs design, this study aims to clarify the epistemological implications of hallucination mitigation and propose conceptual frameworks for better understanding and evaluating LLMs outputs.