# What local and hosted AI really costs

Every month, Lea and Mark prepare the same risk memo from contracts, spreadsheets, and public sources. Lea works on a 16 GB Mac and delegates repetitive extraction to a local model. Mark uses a hosted model for the hardest synthesis. Both deliver a useful document.

Who spent less?

The calculation includes the computer, tokens, elapsed time, retries, review, and the share of deliverables that are actually accepted. A cheap generation that needs three corrections can cost more than an expensive generation that works on the first attempt.

This article examines two separate decisions: **Qwen 3.5 4B on the Mac or Claude Haiku 4.5 through Anthropic**, then **DeepSeek V4 Flash 0731 through Umans or Claude Sonnet 5 through Anthropic**. The first comparison is about where inference runs. The second compares two hosted routes. Each comparison retains its own model, route, and task.

## The figures that matter on August 13, 2026

| Case | Model billing | Other decision factors |
| --- | --- | --- |
| Qwen 3.5 4B in loqy | No per-token invoice from a provider or loqy | Hardware, energy, memory, storage, and Mac time |
| Claude Haiku 4.5 | $1 per million input tokens, $5 per million output tokens | Task quality, cache, tools, network, and review |
| DeepSeek V4 Flash 0731 through Umans | $0.14 input, $0.28 output, $0.028 cache read | Route-specific measurement, service terms, and data boundary |
| Claude Sonnet 5 | $2 input, $10 output | Standard Anthropic API price |

Prices are in US dollars before tax per million tokens. They are volatile. The dated method and sources appear at the end of the article.

## Local inference shifts cost to the device

A local model adds no provider charge for each word sent or generated. Once the model is downloaded, another generation does not create a proportional API line item. This zero marginal token invoice can reshape the economics of frequent, predictable work that fits the model.

The computation still uses real resources:

- the Mac must exist and have enough unified memory;
- weights, the vision projector, and caches occupy storage;
- inference consumes energy and produces heat;
- memory and GPU capacity are occupied during generation;
- speed depends on the device, context length, and competing load;
- user time rises when the model needs more retries or correction.

Buying a Mac only for inference makes hardware cost prominent. Reusing an already depreciated device for short tasks produces a different result. Local cost is therefore specific to the existing fleet and its utilization.

![Two cost stacks compare local and hosted inference, from token billing to hardware resources and review time](/assets/blog/inference-cost-stack.svg "Local inference has a zero provider token charge and uses device resources. Hosted inference bills usage and carries the generation load.")

## Case one: local Qwen 3.5 4B or hosted Claude Haiku 4.5

The loqy V1 profile planned for 16 GB Macs uses a 2.55 GiB Q4_K_M Qwen 3.5 4B file and a separate 0.63 GiB vision projector. Its 16 GB profile reserves about 7.8 GiB for inference and launches the engine with a technical 131,072-token window. A V1 run receives a lower active limit: 16,384 tokens in Standard mode, 32,768 in Extended, and up to 65,536 in Deep. The original model is multimodal and Apache-2.0 licensed.

The exact Qwen artefact is a system of weights, quantisation, inference runtime, context profile, and hardware. Throughput and quality are measured on that complete configuration. [The local-model qualification method](/blog/how-loqy-evaluates-local-models/) fixes that unit of comparison precisely.

Claude Haiku 4.5 is a different service. Anthropic charges $1 per million input tokens and $5 per million output tokens. The loqy V1 hosted catalog exposes it as a text and image model with a published 200,000-token context. Anthropic supplies and operates the compute.

| Model | Input, per million tokens | Output, per million tokens |
| --- | ---: | ---: |
| Local Qwen 3.5 4B | $0 provider invoice | $0 provider invoice |
| Claude Haiku 4.5 | $1 | $5 |

Local inference becomes valuable when it avoids a recurring expense, keeps content on the device, works offline, or reduces reliance on a service. Haiku is valuable when its speed, context, or quality saves more time than the API costs.

This comparison covers marginal cost and compute placement. The quality of the exact Qwen artifact belongs to V1 qualification, with its own tasks, measurements, and admission criteria.

## Case two: DeepSeek V4 Flash 0731 or Claude Sonnet 5

Both models are hosted. The provider carries their computation and bills every request. The decision involves price, useful performance, speed, processing terms, and fit for the task.

loqy addresses the exact `umans-deepseek-v4-flash-0731` route through Umans. Umans is the provider shown to the user, and its policy documents possible routing to third-party inference providers for continuity. These were the public prices on August 13, 2026, before tax:

| Route | Input, per million tokens | Output, per million tokens | Simple average |
| --- | ---: | ---: | ---: |
| DeepSeek V4 Flash 0731 through Umans | $0.14 | $0.28 | $0.21 |
| Claude Sonnet 5 through Anthropic | $2 | $10 | $6 |

**On the simple average of input and output rates, DeepSeek V4 Flash 0731 through Umans is about 29 times less expensive than Claude Sonnet 5.** The billed amount then depends on the actual mix of input and output tokens.

## Price takes meaning on a task

The August 6, 2026 loqy lab ran a bounded fixture across several models and reasoning levels. On the max-effort profile selected for this comparison, the exact DeepSeek V4 Flash 0731 route produced a 0.6843 aggregate score. Claude Sonnet 5 produced 0.6902. Each cell contained 146 settled fixture results and the intervals overlapped widely.

The result has a precise scope: **on this loqy fixture, with this profile and two repetitions, the headline scores were close**. Writing, coding, vision, languages, very long contexts, and tool policies require separate qualification. DeepSeek’s best profile in this experiment used another effort level.

Speed adds another dimension. At 16:34 CEST on August 13, the Umans status page showed 299.6 output tokens per second as the five-minute median for the exact 0731 route. Its current daily median was 285.2 and the published daily peak was 318.0. Artificial Analysis separately measured Claude Sonnet 5 at high reasoning effort at 60.3 output tokens per second over the previous 72 hours with 10,000 input tokens.

The two published reference points each describe their own protocol. They show that a provider can serve an open model at high generation throughput, while complete response time also depends on first-token latency, internal reasoning, response length, and tool calls.

![Two panels compare Qwen 3.5 4B with Claude Haiku 4.5 for placement, then DeepSeek V4 Flash 0731 with Claude Sonnet 5 for price, published speed, and one bounded loqy evaluation](/assets/blog/model-cost-speed-comparison.svg "The first panel compares placement and marginal cost. The second separates price, speed measured under different protocols, and a score from one bounded loqy fixture.")

## The data boundary belongs in the price discussion

Local generation keeps inference context on the Mac. Hosted generation sends admitted context to the selected provider. A token price therefore pays for infrastructure, availability, and a data-processing contract as well as computation.

Umans states a zero data retention, or ZDR, boundary for request content: payloads such as prompts, code, and files are processed without being stored as product data by default. Its policy separately identifies retained technical and security metadata, and provides for continuity routing to third-party inference providers during high load or incidents. The ZDR boundary therefore covers request content.

In loqy, the user selects the model for the need. Confidential work that fits the local model can remain on the device. A task that justifies different intelligence, a larger context, or higher throughput can use a hosted model. That choice stays fixed for the run. [The path of a hosted request](/blog/where-ai-agent-work-runs/) shows exactly which data crosses that boundary.

## The denominator should be an accepted deliverable

An agent’s useful cost extends beyond the model response. Sources must be gathered, tools called, failures retried, outputs reviewed, and the deliverable corrected.

A practical comparison uses this formula:

> **Cost per accepted deliverable = model cost + device cost + review time + retries and failures, divided by the number of accepted deliverables.**

Suppose a $0.00336 route produces an acceptable document half the time and each result needs ten minutes of correction. A $0.06 route that succeeds nine times out of ten and needs two minutes of review can be economically better. For a routine extraction that Qwen handles reliably on an existing Mac, the conclusion can reverse.

Evidence also lowers cost. A memo that preserves sources, the exact model, tool results, and decisions is faster to inspect. [That evidence chain](/blog/why-ai-agents-need-evidence/) makes review time measurable instead of letting it disappear into a fresh investigation.

## A workable selection rule

Cost changes with the work. Four questions are enough to begin:

1. Does the local model produce an acceptable result for this task on this device?
2. May the content leave the Mac under the project’s requirements?
3. Does a hosted model save enough time, retries, or context to justify its price?
4. Can sources and evidence validate the result without repeating the entire task?

Lea might keep monthly extraction local and choose Sonnet for a difficult negotiation. Mark might use DeepSeek through Umans for high-volume analysis and reserve Sonnet for cases where its behavior has a measured advantage. The common rule is simple: placement and model follow the need, not a universal ranking.

## Method and scope

Figures checked on August 13, 2026:

- public standard prices in US dollars before taxes and discounts, per million tokens;
- simple calculations centred on model rates;
- the exact named Umans route for DeepSeek V4 Flash 0731;
- Umans throughput from its status page and Sonnet throughput from Artificial Analysis, using different protocols;
- the DeepSeek and Sonnet score from one bounded loqy evaluation;
- a Haiku and Qwen comparison centred on placement and marginal cost;
- local cost completed by hardware, energy, and human time for each use case.

Prices and throughput can change after publication. The method remains useful: compare identical units, name the exact route, and measure the cost of an accepted result.

## Sources

- loqy managed local-model catalog
- loqy hosted-model catalog
- [Official Qwen 3.5 4B model card](https://huggingface.co/Qwen/Qwen3.5-4B)
- [Anthropic API pricing](https://platform.claude.com/docs/en/about-claude/pricing)
- [Umans DeepSeek V4 Flash 0731 route status](https://status.umans.ai/status/umans-deepseek-v4-flash-0731)
- [Umans public pricing](https://umans.ai/pricing)
- [Umans Code privacy policy](https://app.umans.ai/offers/code/legal/privacy-policy)
- [Artificial Analysis measurement for Claude Sonnet 5](https://artificialanalysis.ai/models/claude-sonnet-5-high/providers)
- August 6, 2026 bounded loqy evaluation