Run a fine tuning cycle every two weeks and you're probably solving the wrong problem most of the time. That's an uncomfortable thing to tell a team. They're working hard. The pipeline is real engineering. The metrics they track are the metrics the whole field agreed were the right ones. None of that makes the claim wrong. The work is real and it's also pointed at a big chunk of failures it was never going to fix.
I want to be specific about when fine tuning earns its keep, when it doesn't, and what you'd reach for instead. I've built both pipelines in production. I've also watched expensive fine tuning runs fix something a structured rule library would've handled in an afternoon, so I have opinions.
When fine tuning is the right answer
Fine tuning works when the failure is dense in the training distribution and the corrective signal lives in the gradient. A few cases qualify.
The model is missing a capability that only comes from moving many parameters at once. You're adding a language. You're teaching a domain vocabulary it never saw. You want it to internalize a style or a formatting convention it can't fake with a few examples.
The corrections are too numerous and too tangled to write down as discrete rules. The model has to absorb a continuous gradient over its own behavior, not a handful of patches.
And inference cost has to stay low. Fine tuning trades training compute for inference compute, so the corrected model runs at the same price as the original. That saving is real once the correction has to apply across millions of calls.
In those cases the MLOps pipeline is doing exactly what it's for. No shortcut exists. The gradient lands in the weights and the weights ship.
When fine tuning is the wrong answer
Now flip it. The failure is sparse, the corrective signal is discrete, and somebody downstream needs an audit trail.
The model fails on a small set of recurring patterns that drive most of the impact. We're talking five to fifty patterns, not five thousand. Each one is identifiable and you can name it.
The fix is a discrete rule you can write in a sentence or two. "Don't produce code that uses deprecated API X without a deprecation warning." "Escalate to a human reviewer for any task touching customer financial records." "Cite line numbers only when the source actually contains them."
The customer needs to audit which corrections are live and trace each one back to a specific incident or policy call. Regulated industries demand this. Procurement environments demand it more every year.
Here fine tuning is a tax. It burns training compute, it eats engineering attention, and it hands you a model whose behavior changed but whose reasoning for the change is opaque. Nobody on the team can point at a weight and explain why it is what it is. The customer can't audit the corrections without poking at the model's behavior across a test suite, and that test suite is just a worse, implicit version of the rule library you should have built in the first place.
The alternative
The alternative I keep coming back to is a failure driven memory loop. The shape is plain.
A critic runs after every session and scores the output against the evidence the model actually saw. The critic is itself a model, but it's small and the call costs almost nothing.
When the score comes back low, a post mortem generator fires and writes a structured rule. The rule carries a name, a justification grounded in evidence, a description of when it applies, and a trigger intent so future sessions can match it semantically.
Before any rule lands, a verifier checks whether it's grounded in the evidence and whether it contradicts existing policy. Hallucinated rules die right here.
On every new task, a trigger evaluator matches the live rule library against the incoming work and injects the relevant rules into the prompt. Lexical predicates run first because they're fast. Semantic predicates fire only when they're needed.
The whole loop costs a few cents per session in extra inference. What you get back is an explicit, auditable rule library. New rules go live in minutes. Old rules get revoked in seconds. The audit trail is the rule itself.
The cost analysis
Let me put numbers on this, because the gap is wider than the field has internalized.
Fine tune a 30 billion parameter model on 6,000 examples and you'll spend roughly $400 to $1,200 in compute, depending on the platform and how long training runs. The cycle takes 6 to 24 hours. Somebody had to prepare the corpus, which is another 8 to 40 hours of human labeling. Then the deployment gets tested against regression suites, call it another 4 to 16 hours of engineering. End to end the cycle runs roughly $2,000 to $8,000 and 2 to 5 days of wall-clock time.
A failure driven memory loop processes one failed session for roughly $0.10 to $0.40 in inference. The cycle takes 30 to 90 seconds. The output is a candidate rule that passes a 10 to 30 second verifier check before it goes live. From failure to live policy is under five minutes.
A fine tuning cycle absorbs maybe 20 to 50 corrective signals at a time. The memory loop absorbs them one at a time, in real time, the moment they show up.
Cost per absorbed correction in the fine tuning case lands somewhere between $40 and $400. In the memory loop case it's between $0.10 and $0.50. That's a 100 to 1,000 times differential in favor of the memory loop, for the class of failures it was built to handle.
Fine tuning still wins the dense corrections that genuinely need gradient updates. The memory loop wins the sparse, discrete, auditable ones. A sane MLOps strategy runs both and stays honest about which class each failure belongs to. The trap is running fine tuning for everything because you built a fine tuning pipeline and that's the hammer you own.
The audit angle
Fine tuning hands you a model, and the model is the audit artifact. A customer can examine its behavior. They can't easily examine the reasoning behind any one behavior change. The audit is empirical, never explanatory.
The memory loop hands you a rule library, and the library is the audit artifact. Every rule has a name, a justification tied to a specific incident, and a documented trigger condition. The customer can read the library, trace any rule to its origin, and roll back anything that turns out wrong.
For regulated industries this isn't a nuance. It's the line between an auditable system and one nobody can vouch for. Vendors who ship the rule library as a first class artifact are going to win the regulated procurement conversations. Vendors who ship only a fine tuned model will hand those deals to whoever can produce the audit artifact on demand.
What MLOps teams should be doing differently
Two changes do most of the work, and a third reinforces them.
One. Triage failures by class before you pick a fix. Not every failure deserves a fine tuning cycle, and most don't. The triage is fast, getting it right costs almost nothing, and getting it wrong costs several thousand dollars per misallocated run.
Two. Build the rule library infrastructure if you don't have it. A rule library with a verifier gate, a trigger evaluator, and a versioned audit trail is real engineering. It's also cheaper than the fine tuning pipeline most teams already run. The infrastructure pays for itself in the first month of fine tuning cycles you didn't waste on the wrong class of failure.
Three. Measure both pipelines on the metrics that matter. Cost per absorbed correction. Time from failure to live correction. How auditable the corrected behavior is. The two pipelines serve different failure classes and the numbers will say so. Let the data point each pipeline at the work it's good at.
A closing observation
The field has spent five years building fine tuning pipelines. They work. The people running them are good at it. The mistake was deciding the pipeline answers every problem that involves changing how a model behaves.
It doesn't. There's a class of corrections, large and getting larger, where a different mechanism is dramatically faster, cheaper, and easier to audit. Teams that adopt the alternative for the right workloads will sit on much lower MLOps costs and a far better audit posture by the end of 2027. The teams that keep defaulting to fine tuning for everything are paying a tax the auditors will eventually catch.
Building the rule library infrastructure is the easy part. The hard part is cultural, the moment a team accepts that not every failure earns a fine tuning cycle. That's the real bottleneck. The engineering never was.