There’s a missing link in artificial intelligence. According to Tom Zahavy, discovery team co-lead at Google DeepMind, today’s AI systems are not yet capable of making the leap from number-crunching and prediction to the kind of intuition that produces breakthrough insights grounded in real-world experience.
In a new paper, Zahavy argues that current large language models (LLMs) may be unable to make original discoveries on their own. While the paper focuses on the limitations of AI in scientific discovery, its conclusions also have implications for AI’s creative capabilities in business settings.
Zahavy dissected the process that went into Albert Einstein’s formulation of his General Relativity Theory, noting that the scientist’s thinking evolved through three phases – deduction, induction, and finally, abduction.
AI excels at induction, based on statistical pattern matching, and is rapidly conquering deduction, which is formal proof, Zahary illustrated. However, it doesn’t clear the third of three processes, which is abduction, “the generation of novel explanatory hypotheses.”
If scientific discovery were merely the sum of deduction and induction, "modern large language models (LLMs) should theoretically be capable of inventing theories like General Relativity given sufficient compute,” he said.
Einstein’s discovery process was outlined in his letter to Maurice Solovine, where he “conceptualized discovery as a cyclical process involving an intuitive ’jump’ from sensory experience to axioms, followed by logical deduction,” Zahavy explained.
Generative AI is capable of learning statistical relationships from training data, and then making logical deductions about the implications of that information. But it can’t tell you why the information it has assembled for you is new or exciting.
The three stages of discovery include the following:
- Deduction (rule + case = result): “The analytic application of a rule to a case to predict a result. It is the only mode that guarantees truth, e.g., executing code to verify output.”
- Induction (case + result = rule): “The synthetic derivation of a rule from the accumulation of cases and results. It validates hypotheses through statistical frequency, e.g., generating a function to satisfy unit tests.”\
- Abduction (rule + result = case): "The inference of a case or a new rule to explain a surprising result."
The bottom line is LLMs are great tools for assembling information, but it still takes humans to add new dimensions or perspectives to what the output is telling us. It takes humans to make the breakthrough observations.
Again, there is a connection to any original thinking that we may hope to see from AI in non-scientific settings. A separate study out of Duke University questions the creative output of LLMs in a different sense – the conclusions tend to be homogenous, and, again unoriginal. “The creative outputs of commercial LLMs are more similar to each other than users might hope. When challenged with three standard tasks assessing creativity, answers from commercial LLMs are much more alike than their human counterparts.”
“People might wonder if different LLMs will take them in different directions with the same prompts for creative projects,” said Emily Wenger, assistant professor of electrical and computer engineering at Duke. “This paper basically says no. LLMs are less creative as a population than humans.”
Zahary emphasized that LLMs, in his experience, have been valuable in assisting humans in developing new theories or finding answers to problems. But LLMs will not do this on their own. “Unlike deduction, which guarantees truth, or induction, which finds pattern that generalize in data, abduction is a creative leap that invents a cause for a singular phenomenon,” Zahary explains. LLMs are not capable of making this leap.
“Einstein did not discover General Relativity by searching over symbols; he discovered it by simulating the sensual experience of a falling observer,” Zahary explained. “To build an AI capable of true invention, we must move beyond systems that merely read scientific literature to systems that can perceive the physical world. The emergence of physically consistent World Models offers a pathway to a synthetic laboratory.”