Self-hosted or hosted LLMs? A decision framework for regulated teams

This decision gets made badly in both directions.

Some organisations reach for self-hosting reflexively, on the assumption that sovereignty demands it, and end up maintaining GPU infrastructure to summarise press releases. Others rule it out on the assumption that open-weight models are hopelessly behind, using a mental benchmark that was accurate two years ago and is not now.

Here is how to decide properly.

Start with classification, not capability

The instinct is to begin by comparing model quality. That is the wrong end.

Begin instead with a table. List every use case you are seriously considering, and for each one write down the highest data classification that will touch the system in production, not in the pilot, not in the demo, but in the version people actually use.

That table usually decides most of the architecture on its own:

Highest classification in production	Realistic options
Public / already published	Anything, including hosted frontier models
Internal, non-personal	Hosted under enterprise terms, or private UK deployment
Personal data, moderate sensitivity	Private UK deployment; hosted only with a completed DPIA and enforced controls
Special category, privileged, or safety-critical	Self-hosted, or private deployment with strong isolation and a documented legal position

Only once you know which row you are in does the capability comparison become meaningful, because you are now comparing options that are actually available to you.

The capability gap is smaller than you think, and narrower than it looks

Frontier hosted models remain ahead of open-weight models on the hardest reasoning tasks. That gap is real and it is not closing as fast as enthusiasts claim.

But it is also mostly irrelevant to the work regulated organisations actually want done. The bulk of real enterprise demand is:

• retrieval and question answering over an internal corpus
• extraction of structured fields from unstructured documents
• classification and routing
• summarisation against a defined template
• drafting in a house style, for human review

On these, well-configured mid-size open-weight models perform close enough to frontier models that the difference is usually smaller than the variance introduced by your retrieval quality, your prompt design and your document preparation. Teams routinely attribute to model capability what is actually a chunking problem.

The test that matters: build an evaluation set of 100-200 real examples from your own corpus, with known-correct answers, before you choose anything. Run both options against it. In our experience this exercise reverses the team’s prior assumption about half the time, in both directions, and it costs a few days.

Anyone who tells you which model is best for your use case without having seen your data is guessing.

Real cost comparison

The cost conversation is routinely distorted in both directions, because the two options are usually compared on different bases.

Hosted costs scale with usage and start near zero. Genuinely cheap at low volume. The failure mode is a successful rollout: per-token costs that were trivial in a pilot with 20 users become material at 2,000, and you have no lever except usage restriction, which undermines the thing you built.

Self-hosted costs are mostly fixed and mostly up front: accelerators, or dedicated cloud GPU capacity, plus the engineering time to run it. Expensive at low volume, and increasingly favourable as usage grows, because your marginal cost per request approaches electricity.

The crossover point depends heavily on your workload, but the shape is consistent: hosted wins on low and unpredictable volume; self-hosted wins on high and steady volume. Model your projected volume at full adoption, not at pilot, and compare there.

Two costs consistently get left out of the self-hosted side of the model:

• Operational burden. Someone has to own capacity, updates, monitoring and incident response. If that person does not exist and you are not hiring them, self-hosting is more expensive than your model says.
• Model refresh. Open-weight models improve; you will want to move. Budget for periodic re-evaluation and migration rather than treating deployment as one-off.

And one cost consistently gets left out of the hosted side:

• The architectural lock-in. Building tightly around one provider’s API, tooling and quirks makes leaving expensive. Keep the boundary clean (an internal interface you control, with the provider behind it) and this cost stays manageable.

The hybrid pattern most organisations land on

In practice, the mature answer is rarely one or the other. It is a routing layer that sends requests to different backends based on data classification:

1. A classification step determines what kind of data is in the request.
2. Sensitive material routes to the private or self-hosted model.
3. Everything else can use the hosted frontier model.
4. The routing decision is logged, so you can prove the boundary held.

This gets you frontier capability where it is safe and sovereignty where it is required, and, importantly, it produces the audit evidence that a boundary existed and was enforced by a control rather than requested by a policy.

The pattern’s weak point is the classifier. If it is wrong, sensitive data goes to the wrong place, silently. Build it conservatively: default to the private path, require positive evidence to route externally, and sample the decisions for review. A routing layer that fails open is worse than no routing layer, because it produces confident evidence of a boundary that was not held.

What actually decides it

After the classification table, three questions:

Do you have, or will you hire, someone who can own inference infrastructure? If not, self-hosting is a commitment you cannot honour. Be honest about this. It is the single most common cause of abandoned private deployments.

Is your volume predictable and material? High steady volume favours self-hosting economically. Spiky or uncertain volume favours hosted, at least until the pattern is clear.

Does your regulator, insurer, or largest customer ask where the data goes? If that question is already in your tenders or your assurance reviews, the answer has commercial value beyond the compliance position, and that value belongs in the business case.

The summary

Do not choose an architecture because it sounds appropriately serious about security. Choose it because the classification table demands it, the evaluation set supports it, and the cost model works at the volume you actually expect.

Then keep the boundary clean enough that you can change your mind in a year, because you probably will.

---

We do this analysis as part of an AI readiness assessment, and build the result in private and self-hosted deployment.