We did everything the agentic-coding playbook told us to do. We wrote skills, we curated a beautiful CLAUDE.md , and we documented our architecture RFCs in crisp markdown, linked them from every corner of the repository, and even set up agentic judges to review the agents’ work. AI watching AI, like a security guard hired to watch another security guard, both of whom occasionally hallucinate.
And then, roughly one commit after we increased the harnessing regarding our architectural layers, an agent took a fat slab of domain business logic and dropped it straight into our application wiring layer. While it was there, it reached past the domain and grabbed the persistence layer directly. The exact two things our RFC said, in bold letters, must never happen.
The agent had read the RFC. It could quote the RFC. It just didn’t obey the RFC.
That’s the moment we stopped writing better prompts and started writing better laws. Okay, we did not stop writing better prompts, we still do that. But if we really care about things, we now make it a law.
Non deterministic harnesses are pleas, deterministic ones are laws
The problem with non-deterministic guardrails like skills, prompt engineering, style guides, or LLM-as-judge is a shared one: each one is a probability, not a deterministic guarantee.
For a deeper dive into the differences between deterministic and non-deterministic and a look into what inspired us, check out my former colleague Birgitta Böckeler’s post on martinfowler.com or her chat with Chris Ford on the topic. But let’s start with a simplified example and get into it:
Say your markdown guideline is respected 95% of the time. On its own that sounds good. Now the agent makes forty decisions in one task where the guideline applies, and it becomes:
P(all respected) = 0.95^40 ≈ 13%I want to give a disclaimer in the form of a quote of my colleague Matthias from his article Why non-deterministic ai agents are the ultimate doom for enterprises here:
Some may say that a good agentic system can correct its own errors, making the steps not truly independent. Others (rightly so) will tell you this is an oversimplification, and I cannot simply apply multiplicative laws of probabilities to an agentic system like a math punk. While that’s true, it just means the math gets more complicated; the fundamental problem of decaying reliability for long tasks remains. And while, indeed, work is done to make models “less stochastic” in their output, ie via Feedback Loops, grounding, planning, task decomposition, the problem is both foundational and multilayered, and it’s not solved now, nor will it ever be fully solved.
None of this means the guideline was badly written, nor that writing it was wasted effort. It’s what we were told to do: You noticed agents need guidance, and you gave it to them in the clearest form the LLM vendor tooling offered. The problem isn’t the idea or quality, it’s the medium. Anything an agent interprets will inherit the probabilistic nature, so even an excellent guideline enters the compliance product above as a factor below one. And stacking more probabilistic layers on top, for example a judge agent that itself works 90% of the time reviewing output that’s 95% compliant, can’t push that product back to one. The math is working against any non deterministic harness, no matter how well it’s written.
Additionally the harnesses shipped with the LLMs, as well as our own guidelines, constantly change. This means that at times your agent based harnessing will yield wildly different results, or stop working all together. This is akin to traditional dependency management where you need to keep up with new contracts. However with dependency management you have a) a control over the timing and b) a deterministic impact.
Even worse is that the more rules you have the less important each individual rule becomes to the agent. This is known as the “Lost in the middle problem”. (see Lost in the Middle: How Language Models Use Long Contexts) So the more time you invest into your non deterministic harness rules, the less important each one might become. This might be non problematic for small isolated tasks that only need a few of your rules, but if you want agents to implement complex features across your whole stack, this becomes a very real risk.
A deterministic check inverts this completely. A test that fails the build when a rule is broken is respected 100% of the time. On the 1st run and on the 10,000th. It doesn’t get tired, doesn’t get creative, and doesn’t decide that this particular case is surely an exception.
The agent doesn’t need to want or remember to follow the rule. It just needs to be physically unable to progress without following it.
Why Rust turned out to be an agent harness for us
At INXM we use Rust for most of our services. And we didn’t pick Rust to be fashionable. We picked it because, viewed through this lens, a strict compiler is a deterministic harness. One somebody else already spent years building.
In Rust:
There is no implicit null. If a value can be absent, it’s an
Option<T>, and the compiler forces the agent to handle both cases. No agent can “forget” a null check, because forgetting is a compile error.Errors are values.
Result<T, E>means error handling isn’t a best practice the agent might skip under pressure; unhandled results are loud, visible, and linted.The compiler talks to agents like a good colleague. Rust’s error messages don’t just say what broke, they usually say how to fix it. That’s not a human luxury anymore. It’s a feedback loop. The agent proposes, the compiler rejects with a precise explanation, the agent corrects. Deterministically. Every time.
Clippy nudges toward idiom. Instead of a markdown file saying “please prefer iterators over index loops,” a lint says it — and can fail the build if we mean it. Better yet the “Clippy” tool can usually adjust the code itself, which we added as a step before committing anything to our agents’ workflows.
In interpreted and / or weakly typed languages every one of these would have been a paragraph in our guidelines. As paragraphs, they’d each be a 0.xx factor in the compliance product above. As compiler rules, they’re 1.0.
Of course you can get similar or even higher levels of support from the tooling in other languages as well. Examples here could be Haskell or Kotlin, where most of our developers made good experiences with their strict modeling capabilities. The general rule of thumb and the main takeaway from this paragraph should be:
The more rules your compiler can statically check with high confidence, the better of a baseline harness it provides
When the language isn’t strict enough, extend it
But languages don’t ship with your architecture rules. No compiler knows that your app wiring layer may construct repositories and clients but must never contain business logic. No borrow checker knows that nothing may enter your domain layer without authorization. And that is the point. A language is general purpose, whilst your architecture rules should fit for your domain and your environment.
So we wrote the rules we cared about and were missing as code. Here at two (very simplified) examples:
Architectural layers
We built a small crate of architecture fitness tests. Deterministic checks that parse the codebase’s AST and fail the build when a structural rule is violated. Simplified, the wiring-layer rule looks like this:
#[test]
fn app_layer_only_wires() {
let violations = analyze_crate("app")
.functions()
.filter(|f| {
!f.is_constructor_call_of(&["Repository", "Client"])
&& !f.is_domain_type_construction()
})
.collect::<Vec<_>>();
assert!(
violations.is_empty(),
"app layer must only wire dependencies and build domain types:\n{violations:#?}"
);
}The app layer may build repositories and clients in approved wiring files. It may construct domain types and hand them over. Anything else, like an actual call of functionality from the persistence crate, and the build goes red with a message pointing at the exact offending line.
We have similar tests demanding our domain layer does not know any of the other crates besides importing the wrapper types for authorization (see next paragraph), and all the other rules we care about in this repository.
Authorization
Our rule that any domain function needs to run in an authorized context got the same treatment, but one level deeper: we encoded it into the type system itself. Domain entry points don’t accept raw requests; they demand proof:
pub fn approve_order(
auth: Authorized<ApproveOrderRequest>, // can only be obtained via the auth crate
) -> Result<Approval, DomainError> {
// ...
}There is no way to construct an Authorized<ApproveOrderRequest> except by going through our authorization crate. An agent that tries to shortcut it doesn’t produce a subtle security bug that a judge-agent might catch on a good day. It produces code that does not compile.
Our RFCs stopped only being documents and became deterministic tests.
What actually changed
After the fitness tests landed, the agents didn’t just get caught more often. They violated the rules less.
In hindsight it makes sense. An agent in a tight loop with a deterministic harness converges fast. It writes the shortcut once, the build screams with a precise error, it learns what this codebase’s physics are. The red build is a far better teacher than the markdown file ever was, because it’s incapable of being ignored.
The practical effect: we now hand agents significantly more complex tasks with a straight face. Not because the agents got smarter, but because the blast radius of their creativity got smaller. The architecture can’t drift, the domain can’t be entered without authorization, and null can’t sneak in, no matter how the model is feeling today.
Okay that last sentence was a bit of an overstatement. We still review the code our agents spit out. And they still make mistakes we did not anticipate. But now we do what good development practices taught us to do when we encounter a bug: We write tests that prevent a regression. And it makes our experience a little better every time.
The takeaway
If there’s one thing to take away from this post, it is this:
Every rule you actually care about should be a failing build, not a paragraph.
Keep the markdown as agents and humans both benefit from context and intent. But treat textual guidelines as what they are: suggestions with a compliance probability strictly below one. The moment a rule matters, like architecture boundaries, authorization, or error handling, promote it. Into a type. Into a lint. Into a fitness test that parses your AST and says no.
Non-deterministic workers are here to stay. The trick isn’t making them promise to behave.
It’s building a world where misbehaving doesn’t compile.