Explanation
Hybrid AI economics
Why local and cloud together beats either one alone.
Most AI tools send every request to a large cloud model. It is simple to build and expensive to run, and the expense lands on you: in money if you pay per token, in privacy if you do not.
The assumption underneath it is that every request deserves a frontier model. Almost none of them do. Summarizing a folder, drafting a reply, pulling a total out of a price list, answering a question about a document you already have: these are well within what a small model on your own Mac does properly.
OOMU's position: do the work on the machine you already own, and pay for the cloud when it genuinely earns it.
What running locally actually costs
Nothing, per request. You already bought the Mac.
| Local | Cloud | |
|---|---|---|
| Marginal cost per request | $0 | Per token |
| Works offline | Yes | No |
| Data leaves your Mac | No | Yes |
| Rate limits | None | Provider's |
| Available at 2am with no signal | Yes | No |
The last row is not a joke. A tool that stops working on a plane, in a basement, or when a provider has an outage is a tool you cannot build a habit on.
The decision has to be automatic
Asking a person to choose local or cloud per message fails, for a predictable reason: the choice arrives before you know how hard the question is. So it gets made once, badly, and stays.
OOMU makes it per request, with a model whose only job is that decision. The understanding model is a 0.6-billion-parameter Qwen3 fine-tuned specifically to route. It is small enough to run before every turn without you noticing, and it has one output: a decision.
It has four answers available, and one of them costs nothing at all:
| Route | Cost |
|---|---|
| Deterministic | No model runs. Native code answers. |
| Local | Free. Runs on your Mac. |
| Cloud | Metered, on your key. |
| Clarify | One question, before spending anything on a guess. |
Deterministic deserves attention. A calculation does not need a language model, and OOMU does not use one for it. Nor does asking for the runtime's own status. Every one of those is a request that costs zero and returns instantly.
Clarify is an economic decision too. A wrong answer that runs a five-step workflow costs far more than a three-second question.
The ledger
On this Mac in the sidebar shows what actually happened: "Compare work completed on this Mac with work completed in the cloud." Choose a period: today, 7, 30, 90 days, or all time.
| Panel | Shows |
|---|---|
| On this Mac | "{local} of {total} completed responses" |
| How OOMU worked | A split bar, local against cloud |
| Private Data Kept on Mac | Megabytes of your input that stayed local |
| Model usage | Input and output tokens, local and cloud |
| Compute activity | Responses over time |
| Potential saved | An estimate: see below |
Read the estimate correctly
Potential saved takes the tokens that ran locally and multiplies them by prices you set. It defaults to $1.25 per million input tokens and $5.00 per million output tokens, and both fields are editable.
The panel labels it honestly: "Potential savings uses these editable prices as a comparison, not as a provider quote."
That is the right framing. It answers "what would this have cost at these rates?", not "what did I save." Set the prices to the model you would actually have used, and the number becomes meaningful. Leave the defaults and it is a rough scale.
Private Data Kept on Mac is similarly an estimate, computed from local input tokens at four bytes per token, and the panel says so.
And where it cannot measure
"{count} cloud responses did not include token totals."
Some providers do not return usage. OOMU reports the gap rather than estimating over it. A ledger that quietly filled in missing numbers would be more satisfying and less useful.
What your Mac can do
The economics only work if local is genuinely capable, which depends on your hardware. Settings → This Mac reports what OOMU has actually observed:
| Reading | |
|---|---|
| Metal GPU | Offload active, CPU active, or Available, not observed |
| Local speed | Tokens per second, with the model and date measured |
| Model memory | From the verified model's real footprint |
Available, not observed is a deliberately unglamorous state. Your Mac supports Metal, but OOMU has not yet seen an offload happen, so it does not claim acceleration. Not measured yet means the same for speed: "Complete a local response to measure actual speed."
Memory sets the ceiling on local context:
| Memory | Maximum local context |
|---|---|
| Under 32 GB | 4,096 tokens |
| 32 – 64 GB | 8,192 tokens |
| 64 GB and above | 16,384 tokens |
More memory buys you either a bigger model or more context. For most professional work, context is worth more.
When the cloud is worth it
Not never. The cloud earns its cost when:
- The task genuinely exceeds what a small model does well: long multi-source synthesis, hard reasoning, complex code.
- You need a frontier model's specific strengths.
- The project permits it, and the material is not sensitive.
OOMU makes that a per-request judgement rather than a standing default, and shows you the ratio afterwards. Set a project to Only this Mac and the question does not arise at all.
The compounding argument
The savings are real but secondary. What matters more is what free-at-the-margin changes about how you work.
When each question costs something, you batch. You self-censor. You ask the important ones and skip the ones that would merely have been useful. When local reasoning is free and instant, you ask the small questions: and the small questions are where most of the value in a working assistant actually lives.
The Quick Launcher exists because of this. ⌘⇧Space, a question, an answer, back to work. That interaction is only viable because it costs nothing and runs on your machine.
Related
- How OOMU picks a model: the routing decision.
- Choosing a local model: matching a model to your Mac.
- Privacy & security: the other reason local matters.