A security tool that can't reproduce yesterday's verdict on identical inputs can't be accredited today. Nobody who has sat through a real accreditation would argue with that sentence. And yet most of the AI security tools being sold into accreditable environments right now break it quietly, every day, without the buyer or the accreditor noticing.
So here's my case. Reproducibility should be treated as a non-negotiable property of any AI system that produces a security-relevant decision. That's a stronger claim than the field is making at the moment. I think it's the right claim, and I think the buyers who internalize it will sidestep a whole class of accreditation failures that everyone else is about to walk straight into.
What reproducibility means in this context
In classical security tooling, reproducibility meant something simple. Same rule, same input, same configuration, same alert. Every time. Nobody argued about it. Signature-based detection is reproducible by construction, because the signature either matches or it doesn't.
AI-based detection behaves differently. The model carries internal stochasticity. Its output depends on a sampling temperature, a top-k or top-p constraint, a randomized initialization that creeps into some inference paths, and the exact runtime version of the inference engine sitting underneath it. Run the same model twice on the same input and you can get two different verdicts.
In research, that's fine. In accreditation, it isn't. The accreditor's job is to certify that the system, given a documented input, produces a documented output. A system that produces variable output can't be certified the same way. It can be certified at a probabilistic level, and that's a weaker, genuinely different kind of certification than the one the buyer thinks they're getting.
The accreditation pathways for federal and critical-infrastructure environments were never built for probabilistic certification. ATO, FedRAMP, IEC 62443, the analogous frameworks, all of them assume deterministic behavior from a certified system. Non-deterministic AI tools are getting certified anyway. Often the accreditor signing the package has no idea they're certifying something the framework was never designed to handle.
Where stochasticity hides in AI security tools
Three places. I've ordered them by how often I watch teams miss them.
The model's sampling temperature. Default temperature on commercial APIs is usually non-zero, because non-zero temperatures give you more varied, more human-sounding output. For a chat assistant that's a feature. For a security tool that has to return the same verdict on the same input, it's a defect. Setting temperature to zero is the first and easiest fix, and it gets skipped constantly, because the people building these tools came out of a research culture where non-zero was just the default you never thought about.
The judge model in semantic predicates. A lot of AI security tools lean on a small judge model to evaluate semantic conditions. Does this email match the intent of a phishing campaign? Does this script match the intent of a credential dump? The judge returns a yes or a no. But the judge is itself a model, with its own stochasticity, its own context window, its own inference-engine version. Two calls to the same judge on the same input can disagree. The fix is to cache the judge's verdict, version-stamp it, and treat that cache as the authoritative answer for the lifetime of the session.
The model version itself. The vendor updates the underlying model. The customer doesn't always hear about it. The system that returned verdict A on input X yesterday returns verdict B on input X today, because the API quietly routed to a different model. This is the nastiest source of non-reproducibility, because it's invisible to the customer. The fix is explicit model-version pinning, with the version written into every audit row, plus vendor-side change control that stops silent rotation cold.
Those three account for the overwhelming majority of practical non-reproducibility I see in the wild. None of the fixes are exotic. They get skipped because the field hasn't yet decided reproducibility is a first-class property worth engineering for.
What reproducibility actually requires
A reproducible AI security tool has four engineering properties.
Temperature zero on every decision-making path. Every model call that produces a verdict, a score, or a decision runs at temperature zero. Sampling parameters are explicit, documented, identical across runs. No production path uses non-zero temperature, full stop. Research and development can crank the temperature up all they want. Production can't.
Version stamping at the moment of decision. Every audit row carries the model version, the inference-engine version, the policy-library version, and anything else that touches the output. The stamps have to be complete enough that, given the inputs and the stamps, the system can reproduce the original verdict on demand.
Cached intermediate results, scoped to the decision. Semantic predicates, judge calls, every other non-deterministic intermediate step gets cached at the granularity of the decision and keyed on that step's inputs. Re-evaluate the same step inside the same decision and you get the cached result back. That's what keeps probabilistic noise from leaking up out of the intermediate steps and into the final verdict.
Replay as a first-class feature. The system ships an explicit replay mode. Hand it a decision identifier and it reproduces the decision and shows you the reproduction is identical. Replay is documented, supported, and exercised on a regular basis. It is not a forensic afterthought the engineering team grudgingly builds the week the auditor asks for it. It's a product capability the auditor can verify themselves.
A vendor whose product has all four is shipping a reproducible AI security tool. A vendor whose product has some subset is shipping something that needs to be evaluated on probabilistic grounds, with the qualifications written into the certification.
Why this matters now
Two things make reproducibility urgent today that weren't urgent eighteen months ago.
The first is volume. The number of AI security tools deployed in accreditable environments has grown by an order of magnitude in the last year, and the accreditation backlog has grown faster than the capacity to clear it. Accreditors are pushing AI tools through without the framework updates that would let them certify probabilistic systems honestly. The reproducibility shortfall is being absorbed quietly into the pipeline. That won't hold. The first audit failure that traces back to non-reproducibility is going to trigger a wave of recertification that lands on every vendor in the category at once.
The second is regulatory pressure. Several jurisdictions are drafting AI-specific regulations with explicit reproducibility requirements baked in. The EU AI Act has provisions that map closely onto what I've laid out above. The US executive orders on AI safety carry similar language. The frameworks aren't finalized. The direction is obvious. Vendors who ship reproducible products today will pass these rules on engineering they already did. Everyone else is looking at substantial rework.
A closing observation
The temptation in AI security is to wave all this off. Perfect reproducibility is too expensive. Probabilistic certification is good enough. The customer will live with variability if the model is excellent. None of those arguments survive a conversation with an accreditor whose own job depends on certifying behavior that's actually reproducible.
The vendors who treat reproducibility as non-negotiable will own the certified market. The ones who file it under nice-to-have will hand those deals to the vendors who built it in. Buyers who write reproducibility into their RFPs will get tools that meet the bar. The buyers who don't are going to find out in the next certification cycle that the tool they bought has a property they never knew to ask about, and by then it's somebody's problem to explain.
A reproducible AI security tool can absolutely exist. There's no engineering law standing in the way. There are only choices about whether to build one, and the category has reached the point where those choices decide which vendors are still selling in 2028.