When Is Self-Hosting an LLM Cheaper Than Bedrock?

Two questions send teams down this path: “our Bedrock bill is growing, should we run this on our own GPU?” and “we fine-tuned a Llama, where does it go?” For most teams the answer to both is no, and the reason is not token price. A GPU instance bills 24 hours a day whether or not anyone calls it. Your traffic does not work that way. The decision is about duty cycle, not volume — and there is a third option between the two that most comparisons skip entirely.

You Are Comparing Billing Shapes, Not Prices

The three options bill on fundamentally different axes, which is why comparing headline rates gets people to the wrong answer.

Option	Billed on	Cost when idle
Bedrock on-demand	Tokens processed	Nothing
Bedrock Custom Model Import	Active model-copy minutes, in 5-minute blocks	Nothing, after 5 minutes idle
Self-hosted on EC2	Instance-hours	Full rate, always

Only the third one charges you for the hours nobody is using it. That single asymmetry decides most real cases before token rates enter the picture.

The Option Most Comparisons Skip

If you have fine-tuned an open-weights model, the common assumption is that self-hosting is your only route. It is not.

Amazon Bedrock Custom Model Import takes your own fine-tuned weights and serves them through Bedrock’s managed, serverless On-Demand mode. Supported architectures include Meta Llama (2, 3, 3.1, 3.2), Mistral 7B, Mixtral 8x7B, Flan, and IBM Granite. You import from S3 or by referencing a SageMaker model ARN.

How it bills:

• Import is free. You pay for inference, based on the number of active model copies and how long they stay active.
• Billing runs in 5-minute increments, starting from the first successful invocation of each model copy.
• Capacity is measured in Custom Model Units (CMUs), determined by architecture, parameter count and context length. AWS’s published examples: Llama 3.1 8B at 128K is 2 CMUs; Llama 3.1 70B at 128K is 8 CMUs.
• With no invocations for 5 minutes, it scales to zero, and back up on demand — with a cold start of tens of seconds. It maintains zero to three copies by default, adjustable through Service Quotas.
• A monthly storage fee per CMU applies on top.

The constraints matter as much as the capability. Weights must be in Safetensors format at FP32, FP16 or BF16 — no 4-bit quantized weights. LoRA-style adapters must be merged into the base model before import. It is available in us-east-1, us-east-2, us-west-2 and eu-central-1, and it does not work with batch inference or CloudFormation.

That precision requirement is worth pausing on, because it exposes a flaw in most self-hosting cost comparisons — see below.

The Formula

Skip the token arithmetic. The question is what fraction of the time your model is actually serving.

Define duty cycle as the share of billed time your model spends active. Then:

Effective hourly cost, self-hosted = instance_hourly_rate ÷ duty_cycle
Effective hourly cost, per-use     = per_active_hour_rate  (idle is free)

Self-hosting wins when:

duty_cycle > instance_hourly_rate ÷ per_active_hour_rate

That is the whole decision. A GPU at $1/hour running at 25% duty cycle costs $4 for every productive hour, and that is the number to compare — not the sticker rate.

Working it through

Take AWS’s own published example: a fine-tuned Llama 3.1-type 8B model at 128K sequence length in us-east-1 needs 2 CMUs, priced at $0.1570 per minute, with $3.90 per month of storage.

That is $9.42 per active hour — far more than any single-GPU instance costs per hour. Read in isolation it looks like a bad deal.

Now apply the formula. Against an illustrative $1/hour GPU instance:

break-even duty cycle = 1.00 ÷ 9.42 = ~11%

Below roughly 11% duty cycle, Custom Model Import is cheaper. Above it, the dedicated instance is. For an internal tool used during business hours, or anything bursty, you are well under that line. For a customer-facing feature under steady load, you are well over it.

Substitute your own instance rate and current CMU pricing — the shape of the answer holds even as the numbers move, which is the point of using the formula rather than memorising a threshold.

The 5-minute trap

Duty cycle here is not measured in seconds of GPU time. Billing starts at the first invocation and runs in 5-minute blocks, and the scale-to-zero timer needs 5 minutes of complete silence.

So a workload receiving one request every four minutes never scales to zero. It has a duty cycle near 100% while doing almost no work. Ten requests spread across an hour can bill the same as continuous load.

Request spacing, not request count, drives the bill. Batching sporadic traffic into deliberate bursts is the single highest-leverage optimisation on this path, and it is invisible if you reason in tokens per month.

Where Model Size Changes the Answer

A common claim is that self-hosting economics improve with model size, because managed inference charges a premium that scales with model tier while hardware does not.

On the Custom Model Import path, it goes the other way. CMUs scale sub-linearly with parameters: 8B at 128K is 2 CMUs, 70B at 128K is 8 CMUs — 4× the units for 8.75× the parameters. Meanwhile the GPU capacity you need to serve a 70B model grows faster than 4× relative to an 8B, once you account for VRAM headroom for the KV cache.

The practical implication reverses the usual advice: the larger your fine-tuned model, the better Custom Model Import looks against a dedicated instance, not worse. Note that CMU counts also scale with context length, so a long-context configuration costs more units than the same parameter count at a shorter window.

The Comparison Almost Everyone Gets Wrong

Self-hosting cost estimates are usually built on 4-bit quantized weights, because that is how you fit a 70B model onto affordable hardware. The managed alternative is then priced at full precision.

Those are not the same product. A 4-bit quantized model produces measurably different output from the FP16 original, and a cost comparison that ignores this is pricing two different things against each other.

Custom Model Import accepting only FP32, FP16 and BF16 is AWS treating that gap as material. If your self-hosted plan depends on 4-bit quantization to make the arithmetic work, the honest comparison is: cheaper inference and a different model. That may well be an acceptable trade — but decide it deliberately, and evaluate quality before committing, not after.

What the Formula Leaves Out

Duty cycle decides the compute line. Three things sit outside it and regularly outweigh it for small teams.

Someone has to run it. Configuring the inference server, handling spot interruptions, monitoring GPU memory, and updating model versions is ongoing work, not a one-off setup. It does not appear on the bill, which is precisely why it gets left out of the comparison.

Spot changes the arithmetic and adds a failure mode. Spot capacity moves the break-even duty cycle sharply in favour of self-hosting, and brings two-minute reclamation notices with it. You need a fallback — a second capacity pool, an on-demand fallback, or request queuing — and that fallback is infrastructure you also own.

Scale-to-zero has a cold start. Custom Model Import takes tens of seconds to wake. If a real user waits on that first request, it is a product decision, not just a cost one. Keeping a copy warm removes the wait and removes the savings.

Choosing

Bedrock on-demand — you are using a base model as published, or you are early enough that traffic is unpredictable. No floor, no infrastructure. This is the default and most teams should stop here.

Custom Model Import — you have fine-tuned open weights on a supported architecture, in a supported Region, and your duty cycle is low. Bursty internal tools, business-hours workloads, anything spiky. Cold start must be acceptable, and you cannot need batch inference.

Self-host on EC2 — your duty cycle is genuinely high and sustained, or you need something the managed paths cannot serve: an unsupported architecture, quantization Bedrock will not accept, or a hard requirement that inputs never leave your own compute. You also need the team to absorb the operational load.

If you are landing on self-hosting mainly because you have fine-tuned a model, check Custom Model Import first. That was the strongest argument for running your own GPU, and it stopped being one.

Summary

• A GPU bills 24/7; the managed paths bill only while working. That asymmetry, not token price, drives the decision.
• Compute the duty cycle break-even: instance_hourly ÷ per_active_hour. Below it, per-use wins; above it, the instance does.
• Custom Model Import serves your own fine-tuned weights serverlessly and scales to zero after 5 minutes idle. Free to import; billed per active model-copy minute.
• Billing granularity is 5 minutes. One request every four minutes bills like continuous load. Batch sporadic traffic into bursts.
• CMUs scale sub-linearly with size — 2 for 8B/128K, 8 for 70B/128K — so bigger fine-tuned models favour Custom Model Import, not self-hosting.
• 4-bit self-hosted versus FP16 managed is not like-for-like. Custom Model Import accepts FP32/FP16/BF16 only.
• Constraints to check first: Safetensors, merged LoRA adapters, four Regions, and no batch inference.

Pricing, CMU values and Region availability change. Verify against the Amazon Bedrock pricing page and the Custom Model Import documentation before committing — the formula is the durable part, not the numbers.