Compiled AI is the word I wish we had a year ago

When the LLM stops being the runtime, enterprise AI starts to look like software again.

Every once in a while, a paper appears that does not necessarily tell you something completely new, but gives you much better words for something you have been trying to explain for a long time. Which is both wonderful and mildly annoying. Wonderful, because now there is a name, a definition, a few tables, and the calming feeling that people far away from your own whiteboards arrive at a very similar conclusion. Mildly annoying, because obviously I would have liked to have had those words earlier. It would have saved us a lot of conversations in which we tried to explain that INXM is not “just another agent platform”, that deterministic orchestration is not a boring implementation detail, and that letting a chatbot improvise your business process end-to-end is not a sign of technical ambition, but mostly a sign that someone has not spent enough time operating production systems.

The paper is called Compiled AI: Deterministic Code Generation for LLM-Based Workflow Automation, and the core idea is refreshingly simple: use the LLM during a compilation phase to generate executable artifacts, validate those artifacts, and then execute the workflow deterministically without asking the model again at transaction time. The authors define compiled AI through three properties: one-time LLM invocation, zero-token deterministic execution, and mandatory multi-stage validation before deployment.

Or, in less academic language: let the probabilistic thing help you build the machine, but do not make the probabilistic thing be the machine.

That is pretty much the sentence I wish we had one year ago.

I wrote before that non-deterministic AI agents are a terrible foundation for enterprise workflows. Not because AI is useless. It is not. But because enterprises are not built on “probably”. They are built on approvals, audit trails, retries, access control, accountability, compliance boundaries, and the very unglamorous ability to explain why something happened after it already happened and everybody suddenly cares about the answer. If each step in a workflow has a probability of being correct, those probabilities do not magically become enterprise-grade because the demo had nice animations. They multiply. 0.99^10 is already uncomfortable. 0.99^100 is not a business process, it is a risk register with a prompt attached to it. (blog.inxm.ai)

And yes, before someone writes the obvious comment: good agentic systems can self-correct, plan, reflect, use tools, run feedback loops, ask another model, and do all the other things we now pack into YAML files until they begin to look like distributed systems designed by a very enthusiastic intern. Some of that is useful. Some of it is necessary. But it does not remove the architectural question. Are you asking an LLM to interpret the work at runtime, step after step, transaction after transaction? Or are you using AI to turn intent into something explicit, validated, and executable, and then running that thing like software?

That distinction is the whole point. And I’m happy to say that INXM has been building the first Compiled AI Process Execution Engine for enterprises. For work too critical to improvise. At the center of that is Orchestrator. Orchestrator is a Process Execution Engine. It uses Compiled AI to turn user intent into executable Plans: governed workflows that complete repeatable, auditable, very small transactions across existing systems. Generated, validated, and deployed once; then run reliably across your existing tools.

That sounds less magical than “autonomous agents will automate your entire business”. Good. Magic is not what you want near your billing reconciliation, quality deviation handling, procurement approvals, regulatory checks, engineering validation, or anything else where “the model felt confident” is not a satisfying incident report.

The paper makes the same trade-off explicit. Compiled AI trades runtime flexibility for predictability, auditability, cost efficiency, and reduced security exposure. It is aimed at well-specified, high-volume, compliance-sensitive workflows where the work needs intelligence to design, but not intelligence to execute again and again. The authors also show why this matters economically: in their function-calling benchmark, compiled AI had a one-time generation cost, zero execution tokens, broke even against direct runtime inference after roughly 17 transactions, and reduced token consumption by 57x at 1,000 transactions.

The cost numbers are nice, of course, but the determinism is nicer. Because cost can be optimized later. Determinism is an architectural decision. If your workflow is interpreted by an LLM every time it runs, then every run is a new little performance. The prompt is read, the context is considered, the tool selection is made, the answer is generated, and everyone hopes the model does not discover a creative interpretation of “approve invoice”. With compiled execution, the clever part happens before deployment. The resulting Plan can be inspected, tested, versioned, approved, rolled back, and explained. Not perfectly, because software is still software and therefore a machine for converting assumptions into tickets, but at least it fails in the way software fails: visibly, locally, and with something concrete to fix.

This is the part I think is still underappreciated.

The paper says deterministic execution yields observability, and that when accuracy degrades you can pinpoint which code segment underperformed and trigger targeted regeneration. It describes compiled AI not as a one-shot process, but as an evolutionary system: deterministic execution with adaptive improvement. (arXiv)

That sentence is very important, because “compile once” alone is not enough. If all you do is generate a giant block of code from a vague specification and then declare victory, you have not solved enterprise AI. You have created a new place for bugs to hide. The real question is how you operate the compiled artifact. What happens when the Plan fails? What happens when a tool schema changed? What happens when the ERP returns a weird response because of course it does? What happens when the human intent was underspecified, or a compliance check requires an additional approval, or the system reaches a branch that looked theoretical during design and very real at 4:17pm on a Friday?

And here is where I will allow myself a little bit of German: we have about one year Vorsprung in this. We did not just arrive at “LLM generates workflow, workflow runs”. That is the easy sentence. We spent the year dealing with the uncomfortable runtime reality around it. In INXM, Plans are not one giant agentic blob that either succeeds or explodes into a tragic stack trace. They are made of small executable steps. Each step has a job. Each step has inputs. Each step has expected outputs. Each step can fail in a way that the system can actually understand. That changes the failure mode completely. When a Plan fails, the LLM does not have to stare at the entire enterprise process like a confused oracle and invent a new strategy from scratch. It sees the exact step that failed, the actual error, the tool response, the surrounding context, and the boundaries of what it is allowed to change. It can then propose a repair to the Plan instead of improvising around the error at runtime. And if the repair changes the governed workflow, the system can come back to the user in the way enterprise systems should come back to users:

“Your Plan failed, but we can repair it. I prepared the following changes. Are you fine with this?”

This sounds like a small UX detail, but it is the difference between an agent silently correcting itself while operating your business on one end, or requiring a 20-person developer team on call on the other, and a governed execution engine that can say: here is where the Plan failed, here is the proposed change, here is why, here is what will happen if you approve it. The user is not asked to debug a vague AI failure. The user is asked to approve a concrete repair to a concrete Plan. That is how AI becomes operable. And reduces cost, because you get predictability without the operating costs of keeping those predictable workflows alive.

Now the paper also has a limitations section, and I like that, because without limitations every architecture becomes a religion. Compiled AI assumes good specifications. Not every workflow can be reduced to deterministic code. Some tasks need runtime semantic understanding. Generation can fail. Generated code quality depends on the model. Broader evaluation is still needed. All true.

But that is also why “Compiled AI” alone is not enough. The specification problem is not solved by telling business users to write perfect YAML. That is just moving the pain from code into a slightly different file format, which is not progress, it is mostly typography. The better approach is to let people describe intent in the language they already use, then turn that intent into an explicit Plan that can be reviewed, validated, deployed, operated, and repaired. Natural language is a perfectly good creation interface, given enough guidance. But it is really really not a good runtime contract.

This is where Orchestrator matters. It turns enterprise intent into executable Plans: governed workflows that complete repeatable, auditable transactions across existing systems. The LLM helps build and improve the Plan. The Process Execution Engine runs it. And when reality disagrees with the Plan, the system does not quietly improvise. It stops at the failed step, prepares a repair, and brings the decision back into governance.

Now, Compiled AI does not mean “no LLMs at runtime ever”. The paper itself highlights this as well. Some tasks are too semantic, too noisy, or too dependent on ambiguous input to be solved with pure deterministic extraction. Their document intelligence example uses a Code Factory variant, where focused LLM calls are wrapped inside compiled code artifacts. That is the right middle ground: bounded invocation, schemas, validation, fallback logic, and deterministic control flow around the probabilistic part. The LLM can help extract meaning from a messy clinical note or invoice, but it should not own the whole process like a tiny drunk process manager. But critical enterprise work is different. The goal is not to maximize how autonomous the AI feels. The goal is to finish the work correctly, repeatedly, and with enough evidence that someone can trust the result later.

That is why I am genuinely happy about this paper. Not in the “excited to announce” way where every minor API wrapper somehow changes the future of humanity. I mean happy in the engineering sense. Someone put a clean name and scientific framing around the thing we have been building toward: AI should compile enterprise intent into governed execution, not improvise business operations forever. And yes, I also like that we have a year Vorsprung in the ugly part. The part where Plans fail. The part where tools return bad data. The part where generated artifacts need tests. The part where users need to approve repairs. The part where auditability, permissions, replayability, and deterministic execution are not slideware, but product requirements.

INXM. AI that finishes the work.