Every once in a while, the AI debate produces a sentence that is so obvious that it should not need to be written down, and yet the fact that it does need to be written down tells you quite a lot about the state of the industry. “Statistical approximation is not general intelligence” is one of those sentences. It is the title of a recent response by Walter Quattrociocchi, Valerio Capraro, and Gary Marcus to the argument that current AI systems already show human-level intelligence. And while I am not particularly interested in spending another afternoon arguing whether the thing in front of us is AGI, proto-AGI, nearly-AGI, fake-AGI, or a very confident spreadsheet with a literature degree, I do think the paper is useful because it points at a confusion that matters far beyond the philosophical question.
The confusion is this: because a system performs well on tasks that look intelligent to us, we start treating the performance as evidence for intelligence itself. That sounds reasonable at first, because humans are also mostly judged by their behaviour. If somebody solves a difficult maths problem, writes a competent essay, passes an exam, or explains a concept well enough that even your uncle stops pretending he understood it already, we usually infer some form of intelligence from that. Like people assumed me to be intelligent after I wrote that piece about explaining an LLM to a 12 year old or your dad. But the paper’s point is that this inference becomes dangerous when we forget that similar output does not imply similar mechanism. A human and a model can both produce the same correct answer, but the process that led there can be very different, and when we move from impressive demonstrations to systems that are supposed to operate in the real world, the mechanism matters more than the screenshot.
This is especially important because the AGI discussion has become a wonderful place to hide sloppy engineering thinking behind large words. If a model passes a benchmark, someone calls it generalisation. If it solves a task from a domain it was not explicitly trained for, someone calls it reasoning. If it produces a convincing answer, someone calls it understanding. And if enough people do that often enough, we eventually end up in a place where “it worked in the demo” becomes the empirical foundation for a claim about general intelligence. Which is quite a leap, and not the fun kind. More the kind where the bridge was generated by a model and nobody checked the load-bearing parts.
The authors push back against exactly that move. Their argument is not that current systems are useless, nor that the progress is fake, nor that benchmark performance does not matter. It clearly does matter. If a model becomes better at coding, translation, summarisation, mathematical problem solving, planning, or tool use, then something meaningful has improved. The problem starts when we take performance on a collection of tasks and treat it as sufficient evidence that the system has the kind of robust, adaptive, reliable intelligence we historically meant when we used the term AGI. That is not a small definitional quibble. That is the difference between observing that a system can do many useful things and deciding that the system can be trusted as a general actor in open-ended environments.
And this is where the paper connects very directly to the enterprise AI problem. In enterprise systems, nobody should care whether the model’s output looks intelligent in isolation. That is not the bar. The bar is whether the system can be operated. Can I explain what happened? Can I reproduce it? Can I test it before it goes live? Can I prevent it from taking actions outside its authority? Can I detect when it has failed? Can I repair the process without turning every incident into a séance where five developers stare at a prompt and try to infer what the model was thinking? Those are the questions that determine whether AI becomes useful infrastructure or a very expensive way to produce new incident categories.
The uncomfortable truth is that statistical approximation is a fantastic interface to messy reality and a terrible runtime contract for critical work. LLMs are very good at dealing with the parts of work that classical software has always hated: ambiguity, incomplete language, weird phrasing, inconsistent descriptions, half-structured documents, human intent that arrives wrapped in three layers of organisational folklore, and all the little cases where a regular expression looks at the input and quietly gives up on life. That is a real breakthrough. It is not marketing. It is genuinely useful that a model can read a messy description, infer structure, propose categories, explain trade-offs, translate between business language and technical language, and help a human get from “this is roughly what I need” to something that can be specified.
But none of that means the model should own the execution of the work forever. Quite the opposite. If the paper is right that we should not confuse statistical approximation with general intelligence, then we definitely should not confuse statistical approximation with governance. A model can produce the next plausible token, the next plausible step, the next plausible tool call, the next plausible explanation. Sometimes that plausible continuation is correct. Often enough it is extremely useful. But plausibility is not the same thing as truth, and it is absolutely not the same thing as accountability. Enterprises are not built on plausible continuations. They are built on processes, approvals, constraints, logs, retries, audit trails, access rights, boring controls, and all the other things that make software less magical and therefore actually usable.
This is the part that the agent hype keeps trying to skip. The story usually goes like this: instead of building workflows, we give an autonomous agent access to tools, tell it what outcome we want, and then it figures out the rest. Which sounds beautiful, until you remember that “the rest” is where most of the work lives. The rest is the ERP returning a value that is technically valid and semantically nonsense. The rest is the customer using a term that only makes sense inside their department because that department has developed its own private language over the last decade. The rest is a policy document contradicting another policy document, a missing field being optional in the schema but mandatory in reality, an approval limit changing because someone restructured procurement, or a supplier invoice that looks normal until you notice that the legal entity changed halfway through the process. Real systems are not made of happy paths. They are made of exceptions that survived long enough to become business rules.
Now, one can of course argue that a sufficiently capable agent can handle those cases. It can inspect the error, reason about the context, try a different path, ask another model, reflect on its own answer, call a tool, update its plan, and continue. Some of that is true, and some of it is genuinely useful. But it does not remove the architectural problem. It mostly moves it into a darker room. If the agent repairs itself silently while operating your business, then the system may become more impressive and less governable at the same time, which is not what I would call progress unless your definition of progress is “we now have incidents that write their own postmortems”. The ability to recover from uncertainty is not the same as the right to improvise around controls.
This is why I keep coming back to Compiled AI. Not because the word itself is magical, although it is nice when a term finally appears for something you have been trying to explain for a while. The useful idea is that the LLM should be involved where probability is helpful: during understanding, design, generation, specification, validation, and repair. It should help translate intent into a Plan. It should help identify missing information. It should help propose steps, map messy human language into structure, generate executable artifacts, and explain what will happen. But once the work is understood, the execution should become explicit. The Plan should be inspectable, testable, versioned, approved, monitored, and repairable. In other words, the probabilistic system helps build the machine, but the machine does not have to remain probabilistic at every transaction.
Compiled AI is the word I wish we had a year ago
Every once in a while, a paper appears that does not necessarily tell you something completely new, but gives you much better words for something you have been trying to explain for a long time. Which is both wonderful and mildly annoying. Wonderful, because now there is a name, a definition, a few tables, and the calming feeling that people far away fr…
That distinction sounds small, but it changes the failure mode completely. In a runtime-agent system, every execution is a new interpretation. The model reads the context, decides what matters, selects tools, chooses arguments, evaluates responses, and continues. Every run is a little performance, and performances can differ. In a compiled execution model, the interpretation happens before deployment. The resulting Plan is not a vague intention floating inside a prompt. It is a governed artifact. It can be reviewed. It can be tested against known cases. It can be logged. It can fail at a specific step for a specific reason. And when it fails, the model does not need to reinvent the whole process like a confused oracle with admin permissions. It can look at the failed step, the actual error, the tool response, the surrounding context, and the allowed boundaries for repair.
This is also where the paper’s critique of AGI claims becomes practical. If task performance is not enough to establish general intelligence, then task performance is also not enough to justify operational autonomy. A benchmark can tell us that a system often produces the right answer for a certain class of inputs. It cannot tell us that the system is safe to put in charge of a business process that has legal, financial, reputational, or operational consequences. A Turing-test-style interaction can tell us that a system can sound human enough under certain conditions. It cannot tell us that the system understands organisational accountability, or that it will stop at the right boundary, or that it knows when the correct answer is not to continue.
The annoying thing is that this should not be controversial. We already know this from software. We do not deploy systems into production because they once produced a nice output in a meeting. We test them, we constrain them, we monitor them, we define ownership, and then reality still finds a way to make us feel humble. Somehow, when AI enters the conversation, people become willing to forget decades of operational discipline because the demo can explain itself in a pleasant tone. That is not engineering. That is being seduced by autocomplete with better manners.
And yes, humans are also unreliable. Humans forget, misunderstand, hallucinate, overfit to their own experience, approve things because the meeting starts in three minutes, and generally behave like biological distributed systems with caffeine dependencies. But organisations do not deal with human unreliability by pretending it does not exist. They wrap humans in process. Four-eyes principles, approval limits, audit trails, training, policies, escalation paths, segregation of duties, incident reviews. None of that makes humans perfect. It makes the work governable. It creates a structure in which mistakes have a place to surface and a path to be corrected. If we are serious about using AI in enterprises, the same principle applies. The goal is not to pretend that the model has become a perfect worker. The goal is to design a system in which the model’s usefulness is amplified and its failure modes are contained.
That is why “statistical approximation is not general intelligence” is not an anti-AI argument. It is a pro-architecture argument. It says: use the thing for what it is good at, but do not confuse the interface with the runtime. Use it to read the messy world, but do not let it silently rewrite the rules of the business while doing so. Use it to generate Plans, but make those Plans concrete. Use it to repair failures, but make repair an explicit governed action. Use it to accelerate the creation of workflows, but do not confuse faster workflow creation with permission to run uncontrolled workflows.
There is a very boring sentence hidden underneath all of this, and boring sentences are usually where enterprise value lives: AI systems need boundaries. Not because they are weak, but because they are useful enough to become dangerous without them. A weak tool does not need much governance because nobody gives it important work. A powerful tool absolutely does. The more capable LLMs become, the more important it becomes to separate semantic understanding from operational authority. Otherwise we will keep building systems that look intelligent in the parts humans can easily observe and fail in the parts organisations actually depend on.
Enterprise IT Can’t Afford MCP Chaos—Here’s the Fix
If you've tried integrating Model Context Protocol (MCP) servers into your enterprise workflows, you know the frustration. Current tools are either too basic—built for single users or local demos—or overly complex, requiring extensive engineering resources to maintain. It's like choosing between a tricycle or a spaceship for grocery shopping. Neither fi…
So no, I do not think the most interesting question is whether current models are AGI. Maybe they are, under some definition. Maybe they are not, under a better one. Maybe the word has been stretched so far that it now mostly tells you which conference panel someone wants to be invited to. What matters for enterprise AI is much simpler and much harder to fake: can the system finish the work correctly, repeatedly, observably, and with enough evidence that someone can trust the result later?
That is the bar. Not fluency. Not benchmarks. Not the feeling that the system “kind of gets it”. The bar is operability. Statistical approximation can help us get there, but only if we stop asking it to be the whole machine. Probabilistic where the world is messy. Deterministic where the business needs repeatability. Adaptive where the Plan needs improvement. Governed where the change has consequences.
That is not as exciting as declaring AGI. Good. Excitement is not a control mechanism.