Explanation
Hybrid AI economics
Why local and cloud together beats either one alone.
Most AI tools send every request to a large cloud model. It is simple to build and expensive to run, and the expense lands on you: in money if you pay per token, in privacy if you do not.
The assumption underneath it is that every request deserves a frontier model. Almost none of them do. Summarizing a folder, drafting a reply, pulling a total out of a price list, answering a question about a document you already have: these are well within what a small model on your own Mac does properly.
OOMU's position: do the work on the machine you already own, and pay for the cloud when it genuinely earns it.
What running locally actually costs
Nothing, per request. You already bought the Mac.
| Local | Cloud | |
|---|---|---|
| Marginal cost per request | $0 | Per token |
| Works offline | Yes | No |
| Data leaves your Mac | No | Yes |
| Rate limits | None | Provider's |
| Available at 2am with no signal | Yes | No |
The last row is not a joke. A tool that stops working on a plane, in a basement, or when a provider has an outage is a tool you cannot build a habit on.
The decision has to be automatic
Asking a person to choose local or cloud per message fails, for a predictable reason: the choice arrives before you know how hard the question is. So it gets made once, badly, and stays.
OOMU makes it per request, with a model whose only job is sorting requests. The System 1 router is a 0.6-billion-parameter model that scores a short list of options and writes nothing. It is small enough to run before every turn without you noticing.
Most of what it sorts costs nothing at all:
| Outcome | Cost |
|---|---|
| An answer on this Mac | Free. Streams straight from the local assistant, with no planning pass. |
| One native action, such as adding a reminder or reading your battery | Free. The action runs in native code; the assistant only fills in its details. |
| A calculation in the Quick Launcher | Free, and no model runs at all. Native code answers, exactly. |
| A cloud answer | Metered, on your key, and only when you have allowed cloud use and the request calls for it. |
Automatic stays on this Mac until you turn on Use cloud models in Automatic. That default is an economic decision as much as a privacy one: nothing is spent until you decide it should be.
A question is an economic decision too. When a request is ambiguous, OOMU asks one question rather than spending a five-step plan on a guess.
The ledger
On this Mac in the sidebar shows what actually happened: "Compare work completed on this Mac with work completed in the cloud." Choose a period: today, 7, 30, 90 days, or all time.
| Panel | Shows |
|---|---|
| On this Mac | "{local} of {total} completed responses" |
| How OOMU worked | A split bar, local against cloud |
| On-device data estimate | About how many megabytes of your input stayed local |
| Model usage | Input and output tokens, local and cloud |
| Compute activity | Responses over time |
| Potential saved | An estimate: see below |
Read the estimate correctly
Potential saved takes the tokens that ran locally and multiplies them by prices you set. It defaults to $1.25 per million input tokens and $5.00 per million output tokens, and both fields are editable.
The panel labels it honestly: "Potential savings uses these editable prices as a comparison, not as a provider quote."
That is the right framing. It answers "what would this have cost at these rates?", not "what did I save." Set the prices to the model you would actually have used, and the number becomes meaningful. Leave the defaults and it is a rough scale.
On-device data estimate is similarly an estimate, computed from local input tokens at four bytes per token, and the panel says so.
And where it cannot measure
"{count} cloud responses did not include token totals."
Some providers do not return usage. OOMU reports the gap rather than estimating over it. A ledger that quietly filled in missing numbers would be more satisfying and less useful.
What your Mac can do
The economics only work if local is genuinely capable, which depends on your hardware. The This Mac card in Settings → Models & Providers reports what OOMU has actually observed:
| Reading | |
|---|---|
| Metal GPU | Offload active, CPU active, or Available, not observed |
| Local speed | Tokens per second, with the model and date measured |
| Model memory | From the verified model's real footprint |
Available, not observed is a deliberately unglamorous state. Your Mac supports Metal, but OOMU has not yet seen an offload happen, so it does not claim acceleration. Not measured yet means the same for speed: "Complete a local response to measure actual speed."
Memory sets the ceiling on local context:
| Memory | Maximum local context |
|---|---|
| 8 GB | 8,192 tokens |
| 16 GB and more | 32,768 tokens |
Beyond 16 GB, more memory buys you a bigger model of your own rather than more context for the built-in one.
When the cloud is worth it
Not never. The cloud earns its cost when:
- The task genuinely exceeds what a small model does well: long multi-source synthesis, hard reasoning, complex code.
- You need a frontier model's specific strengths.
- The project permits it, and the material is not sensitive.
OOMU makes that a per-request judgement rather than a standing default, and shows you the ratio afterwards. Set a project to Only this Mac and the question does not arise at all.
The compounding argument
The savings are real but secondary. What matters more is what free-at-the-margin changes about how you work.
When each question costs something, you batch. You self-censor. You ask the important ones and skip the ones that would merely have been useful. When local reasoning is free and instant, you ask the small questions: and the small questions are where most of the value in a working assistant actually lives.
The Quick Launcher exists because of this. ⌘⌃⇧Space, a question, an answer, back to work. That interaction is only viable because it costs nothing and runs on your machine.
Related
- How OOMU picks a model: the routing decision.
- Choosing a local model: matching a model to your Mac.
- Privacy & security: the other reason local matters.