Most AI architecture writing assumes the cloud is reachable. Most operational AI in the environments that actually matter runs without that assumption. The mismatch produces vendor offerings that sound right in a conference room and then fail in the field, because the field includes ships at sea, military forward deployments, classified networks, energy facilities with strict data sovereignty rules, hospitals under HIPAA constraints that rule out cloud inference, and a long tail of regulated industries that simply can't send their data to a third-party API.
I want to sketch what an air-gapped AI reference architecture actually looks like, not as a degraded mode of a cloud architecture but as the normal operating mode for a meaningful and growing share of production AI workloads. The architecture is achievable today. The components exist. Integrating them is non-trivial but not exotic. What's missing is a vendor consensus on what the reference looks like, so every customer ends up reinventing the same wheel and paying for the lesson.
The five things that have to live on the metal
Air-gapped means the deployment can't reach the public internet, and that's more restrictive than it sounds. Five categories of artifact have to live entirely inside the air-gapped boundary, with no runtime dependency on anything outside it.
One. Model weights. All of them. The full-precision weights, the quantized weights for the production inference path, and the variant weights for any fallback or specialty role. No model gets loaded from a public bucket at runtime. The weights sit on local storage at deployment time and they stay there.
Two. Embeddings. Any precomputed embeddings the system uses, including the embeddings for the policy library, the documentation corpus, and any other reference material. The embedding model itself has to live inside the boundary too, because computing new embeddings on incoming data has to happen locally.
Three. Tools. The tool definitions, the tool implementations, and any binaries the tools call. If the agent needs a code interpreter, the interpreter is inside the boundary. If it needs a search tool, the search index is inside. The boundary is the execution environment for everything the agent might ever reach for.
Four. Training data. This is where a lot of implementations quietly fail. The training data the model was tuned on, the evaluation suites, the regression tests, all of it has to be available inside the boundary if the deployment ever needs to be diagnosed or recertified. Without it, troubleshooting a model behavior question means breaking the air gap, which usually means the question never gets troubleshooted at all.
Five. Audit log infrastructure. The audit storage has to sit inside the boundary, sized for the deployment's full retention requirement, with rotation and cold tiering that works without cloud object storage. People miss this one, because most cloud architectures just assume audit goes to S3 or GCS. The air-gapped architecture has to stand up an on-premises equivalent.
Together these five represent a substantially larger storage and compute footprint than a cloud-connected deployment carries, and the architecture has to budget for all of them at design time. Adding any one of them later is harder than including it from the start.
The update problem
The hardest sustained operational problem in air-gapped AI is updating the system. The model gets a new release. The vulnerability database gets a new signature. The training data gets a new corpus. The cloud-connected case handles every bit of that transparently. The air-gapped case handles none of it for you.
So the reference architecture has to specify the update mechanism explicitly. Three patterns work in practice, and each one trades something away.
Pattern A. Periodic physical media. Updates ship on a removable drive, and a human carries it through the boundary on a defined cadence. This is the highest-assurance pattern and the lowest cadence. It fits environments where crossing the boundary is itself a regulated activity. Cadence usually runs monthly to quarterly.
Pattern B. One-way data diode. Updates arrive through a hardware device that physically permits traffic in one direction only. Higher cadence than physical media, and still high assurance. The diode has to be certified for the environment, and the update content has to be validated on arrival.
Pattern C. Scheduled cloud window. The boundary opens on a schedule, the updates are pulled, the boundary closes. This is the lowest-assurance pattern and the highest cadence. It fits environments where the air gap is policy rather than physical, and where the temporary connection can be tightly monitored.
The reference architecture supports all three, and the customer picks based on their threat model. The vendor's job is to make the update payload behave identically across all three patterns.
The security tradeoffs
Air gapping changes the security posture in ways that aren't always intuitive.
The boundary itself cuts the attack surface dramatically. An attacker can't reach the deployment over the public internet at all. That's the central benefit of the architecture and the reason organizations choose it in the first place.
The boundary also concentrates the risk. Everything that gets through it has to be inspected, because there's no second chance waiting downstream. An update carrying a malicious payload, a piece of training data with a hidden backdoor, a tool that's been tampered with, any of these reaches the deployment with no further filtering. The threat model shifts from what can reach the system to what arrives at the system.
That shift has supply-chain implications. The vendors of every component the deployment uses turn into high-trust dependencies. The audit pathway for those components has to be more rigorous than it would be in a cloud-connected deployment. The customer's security team has to be able to validate the integrity of every artifact that crosses the boundary.
It has implications for the model too. A cloud-connected deployment can lean on the model vendor to silently patch vulnerabilities. The air-gapped deployment can't. The customer has to know about every model vulnerability that affects them, understand which patches exist, and schedule the updates by hand. That's a real operational burden, and the cloud-connected architecture absorbs it invisibly.
What is missing in current vendor offerings
Most AI vendors, including the ones that explicitly market to defense and regulated industries, ship products that are partial fits for air-gapped deployment. The gaps are familiar.
The model is air-gapped, the tooling isn't. The agent loads the model from local storage but reaches out to a cloud-hosted vector database for retrieval, or a cloud-hosted MCP server for tool execution. The deployment looks air-gapped right up until the first time the network actually goes down.
The audit log is local, the audit query interface is cloud-hosted. The deployment writes audit records locally and then serves dashboards from a cloud SaaS. So the customer's compliance team is querying their own data through a cloud service, which defeats the entire point of the air gap.
The model is air-gapped, the model updates aren't. The deployment requires regular check-ins to a cloud licensing server to validate the model's authorization to run. The first network interruption triggers a license expiry that takes the whole deployment offline.
None of these gaps are technical limitations. They're business-model artifacts. The vendors found it easier to monetize through cloud-connected components than through pure air-gapped offerings. Customers who care about the air gap as more than a marketing claim have to read the deployment requirements carefully and reject the partial fits.
A closing observation
Air-gapped AI is a real architecture, with real components, real tradeoffs, and a real customer base. That base is growing, because the regulatory environment is moving in this direction faster than the vendor ecosystem is. There's a meaningful procurement opportunity in 2026 and 2027 for vendors that ship genuinely air-gappable products without partial cloud dependencies buried in the architecture.
The customers who recognize the opportunity will get better products. The vendors who recognize it will win deals their cloud-first competitors can't touch. The architecture isn't difficult. It just demands the discipline of designing against the constraint from the beginning instead of retrofitting it later.
The constraint is real, and it isn't going away. The architectures that work under it are the ones that will run a meaningful share of the AI workloads that genuinely move the needle in regulated industries over the next five years.