r/analytics • IOh We just found out our AI has been making up analytics data for 3 months and I'm gonna throw up. Support So we've been using an Al agent since November to answer leadership questions about metrics. It seemed amazing at first fast answers, detailed explanations, everyone loved it. I just found out it's been hallucinating numbers this entire time. Our VP of sales made territory decisions based on data that didn't exist. Our CFO showed the board a deck with fake insights. The Al was just inventing plausible sounding percentages. I only caught it by accident when someone asked me to double check something. I started digging, and holy shit, it's bad.
"Agentic AI" is your automation script, created by natural language input, running on someone else's computer, having all your data.
There is a benefit for people who can't code, but I'm absolutely unsympathetic to software engineers who are full of praise for them. You should know how it works. You should be aware that there's nothing new in it.
And once again, it's not intelligent, it doesn't think or reason. It generates an output that statistically most likely is what you are looking for.
Reasoning models by the way just use a multi-step approach, where they take the generated output as an additional context to generate a maybe better fitting answer. #GenAI#AgenticAI