Air-Gapped LLMs & Agents on Your Own Cluster
The AI half of an air-gapped deployment, in detail: where the weights live, how retrieval grounds itself without an embedding API, what an agent's tools connect to when there is nothing to connect to, and how any of it stays current.
Weights Live on Your Cluster
Open-weight models are served from hardware inside your perimeter. There is no external inference endpoint to reach, so there is no external inference endpoint to lose.
Retrieval Grounds Itself Locally
Embeddings, indexing and retrieval all run inside the environment, grounded in your own ontology and documents — no hosted embedding service anywhere in the path.
Nothing Phones Home
No licence check, no telemetry, no model-provider callback. Updates arrive when you carry them in, through a channel you control and audit.
"It runs offline" is easy to say and hard to mean. Most AI stacks have at least one thing that quietly needs the internet — the model endpoint, the embedding call, a licence check, a telemetry ping — and you find out which one on the day the cable comes out. This page is the inventory: every part of the AI that would normally reach outside, and what it does instead when it cannot.
Read this if you're the architect who has to certify that an AI deployment genuinely has no outbound dependency — and prove it before accreditation.
Air-gapped AI runs the model serving, retrieval and agent execution layers entirely inside a network with no route to the public internet. Open-weight models are served from your own cluster, embeddings and retrieval are computed locally against your own ontology and documents, agent tools resolve only to systems inside the perimeter, and model and software updates arrive through a controlled channel you operate.
A platform is only air-gapped at the point where every dependency has somewhere local to go. The infrastructure half of that — the cluster, the storage, the isolation — is covered under [Sovereign Foundations](/platform/sovereign-infra/air-gapped). This page is the other half: what happens to the AI itself. Inference stops being an API call and becomes a workload you schedule. Retrieval stops assuming a hosted embedding service. Agent tooling stops resolving to anything outside. Evaluation, updates and licensing stop phoning home. None of these are degraded modes — they are how the platform is built, which is why the capability set on a disconnected network is the same one you would get connected.
Air-Gapped AI in the Scrydon platform
One integrated, sovereign architecture. Here is where Air-Gapped AI sits — highlighted against the full stack it works with.
The AI OS for Humans & AI Agents
Ontology & Semantic Layer, one connected model for your data, knowledge & processes
Combining the best of data lakes, data warehouses and search
AI agents, workflows & automations that execute across your systems
Integrate across A2A, MCP, legacy systems and data sources
Secure domain federation, trusted data sharing, and cross-boundary intelligence
Sovereign Foundations
Air-Gapped AI in depth
Human + AI Orchestration
The AI OS for Humans & AI Agents
The Human + AI Orchestrator is the operational runtime at the heart of the AI OS — also called the Agentic OS — scheduling, routing, and governing every task across your enterprise, whether executed by an AI agent, an existing system, or a human.
Most organisations have broken processes: encoded in siloed systems or locked in people's heads. The AI OS makes them visible and executable. It captures intent, synthesises context, acts — then feeds every result back into the ontology so the next run is smarter. All of it inside your perimeter.
The AI OS only works if it can be trusted. Every layer of the platform rests on a zero-trust infrastructure and identity foundation that operates consistently from fully air-gapped on-premises deployments through to hyperscale cloud environments. Sovereignty is not a feature added on top — it is the condition under which everything else operates.
- Zero-trust architecture: Continuous verification for every request, every user, and every workload — no implicit trust, even inside the perimeter.
- Federated identity: Seamless integration with your existing IdP (SAML, OAuth 2.0, OIDC) for unified, policy-enforced access control.
- Air-gapped deployment: Run the complete platform with no external network dependencies — ideal for defence, critical national infrastructure, and classified workloads.
- Confidential computing: Hardware-level encryption of data in use via AMD SEV-SNP and Intel SGX, protecting workloads even from infrastructure administrators.
Deployment Options: From Air-gapped to Cloud
Deploy the Scrydon platform where it makes sense for you — from air-gapped environments to public cloud — with sovereignty, compliance, and auditability built in.
No data leaves your jurisdiction. No black-box AI. No compromises on control.
This is sovereignty by design.
Four things that normally need the internet — and what they do instead
Take any AI stack and ask what it reaches for. There are usually four answers, and each one has to be replaced rather than disabled before "air-gapped" means anything. Inference is the obvious one: open-weight models are served from hardware inside the perimeter, so generation is a workload the cluster schedules rather than a request that leaves it. Retrieval is the one that catches people out — a stack can serve its own chat model and still send every document chunk to a hosted embedding endpoint, so embedding, indexing and search all run locally here, grounded in your ontology and citable back to records that never moved. Agent tooling changes less than you would expect: agents reach systems over the same Model Context Protocol and reach each other over the same A2A they would use anywhere, and the only difference is which destinations resolve. A runtime whose egress is default-deny to begin with does not need special handling for a network that has no egress at all. What is left is lifecycle — evaluation, observability, audit and updates — and all of it is produced, stored and promoted inside the environment.
Inference — Open-weight models served on your own cluster rather than called over an API. The platform is model-agnostic, so which weights you run is your decision and can change without rebuilding what sits on top.
Retrieval — Embedding, indexing and search execute inside the perimeter against your own ontology and documents, so answers stay grounded and citable with nothing leaving the network.
Agent tooling — Agents reach systems over the same protocols as anywhere else, but the only resolvable destinations are the ones inside your environment. Default-deny egress in a network with no egress is simply the ordinary case.
Lifecycle — Evaluation, observability and audit are produced and stored locally. Model and software updates arrive through your approved channel — no background download, no silent version drift.
The dependencies that only reveal themselves once the cable is out
Almost nobody sets out to build a stack that needs the internet. It happens by default, one convenient dependency at a time, and the bill arrives on the day the environment is sealed. The hosted embedding call is the classic: the chat model was brought in-house, the retrieval path was not, and every document chunk still leaves. Licence heartbeats are the second — entitlement checks that validate over the network work flawlessly in a lab and fail on day thirty inside a vault, which is why entitlement here does not depend on reachability. Telemetry and crash reporting are usually on by default and are a data-egress question long before they are a privacy one; in an accredited environment they are a finding. And the model registry quietly assumes it can fetch weights or a tokeniser on first use, which turns a cold start into an outage. The reason these do not bite here is not that each was found and switched off. It is that disconnected was the design case rather than a configuration of it, so there was never a fetch to disable — a distinction covered from the infrastructure side under Sovereign Foundations.
The hosted embedding call — A stack can serve its own chat model and still send every document chunk to a hosted embedding endpoint. Retrieval is the most commonly overlooked outbound path in an otherwise local deployment.
The licence heartbeat — Software that validates entitlement over the network works perfectly in a lab and fails on day thirty in a vault. Entitlement here does not depend on reachability.
Telemetry and crash reporting — Defaults that quietly report usage are a data-egress question long before they are a privacy one. In a classified environment they are also an accreditation finding.
The model registry — Pulling weights or tokenisers on first use turns a cold start into an outage. Everything needed to serve a model is present before the environment is sealed.
How a disconnected environment keeps up
The fair objection to disconnected AI is not whether it runs — it is whether it ages badly. Models improve quickly, and an environment that cannot reach a registry is an environment that could sit on a year-old model without anyone deciding to. The answer is to make updating deliberate rather than automatic. Model and platform updates move through the channel you already operate for everything else — approved removable media or a data diode — staged, checked and promoted on your schedule. Because the platform is model-agnostic, adopting a newer open-weight model is a deployment step rather than a migration of the agents, retrieval and processes sitting on top of it, so the cost of keeping current stays low enough that you actually do it. Before anything is promoted, it is evaluated inside the environment against your own data and your own tasks, which is a better test than a public benchmark and the only one available here anyway. The evidence that comes out of that — traces, evaluations, audit records — is generated and retained locally, so the material an accreditation review asks for is already in the room rather than something that has to be collected from elsewhere.
Updates you carry in — Model and platform updates move through your own controlled, audited channel — approved removable media or a data diode — on your schedule rather than a vendor's.
Swap models without re-architecting — Because the platform is model-agnostic, adopting a newer open-weight model is a deployment step, not a migration of everything built on top of it.
Evaluate before you promote — Evaluation runs against your own data inside the environment, so a new model is judged on the work it will actually do rather than on a public benchmark.
Evidence stays inside — Traces, evaluations and audit records are generated and retained locally — the accreditation evidence never has to leave to be collected.
Frequently asked questions
Can an LLM really run air-gapped?+
How does RAG work with no internet connection?+
Do AI agents still work without a network?+
How do you update models and software on an air-gapped network?+
Which models can we run offline?+
Does an air-gapped deployment need its own hardware?+
Is anything missing compared with a connected deployment?+
How is this different from the Sovereign Foundations air-gapped page?+
Explore the platform
Prefer to write? Email hello [at] scrydon.com and we will get back to you.
Partners
