How-to guides
Choosing a local model
Add and manage the models available to OOMU, on this Mac and in the cloud.
Everything in Settings → Models, and what your Mac can actually do with it.
The three system models
OOMU's own models sit at the top, under System models: "OOMU's router, local assistant, and optional vision model."
| Role | Model | What it does |
|---|---|---|
| Understanding model | Qwen 3 Router | Reads every request and chooses the path. Required. |
| Local assistant | Gemma 4 E2B | Everyday reasoning on this Mac. Required. |
| Vision | Gemma Vision Projector | Lets the assistant read attached images. Optional. |
Each shows a state: Ready, Missing, or Changed. Install adds one; Replace swaps the file.
Every one is verified by SHA-256 against OOMU's embedded manifest before it is accepted. Changed means the file on disk no longer matches what OOMU recorded: reinstall it rather than working around it.
Why the router is its own model
The understanding model is a 0.6-billion-parameter Qwen3 that OOMU fine-tuned specifically to make routing decisions. It is not a general assistant, and it is not asked to be one. It runs with a 4,608-token context, a 96-token output ceiling, and thinking disabled, which is how it returns a decision in tens of milliseconds. See How OOMU picks a model.
Adding your own models
Under On this Mac:
"Add other models you've downloaded. OOMU uses the default for work on this Mac unless a chat or agent chooses another."
Choose Add models and pick one or more .gguf files. OOMU runs on llama.cpp, so any GGUF that llama.cpp can load will work.
For each added model you can:
| Action | |
|---|---|
| Make default | Use it for local work unless a chat or agent overrides. |
| Rename | Change the display name. |
| Verify | Re-check the file against its recorded hash. |
| Edit / Reassign model | Change what points at it. |
| Remove | Delete it from the list. |
Each model card shows where the file came from and when it was last verified, and how many agents and chats use it: "Used by {agents} agents and {chats} chats."
You cannot orphan work. Removing a model that is in use requires reassigning its chats, agents, and default first. OOMU shows you the count and asks where to move them. The same rule applies to cloud services: "Reassign this service's chats, agents, and default before removing it."
Which models to look for
Gemma 4 in Google's QAT GGUF builds is the family OOMU ships against, and the E2B build is what the local assistant uses. Qwen and Llama GGUF builds also work. As a rule of thumb, a Q4 quantization of a small instruct model is the right starting point on 16 GB; larger memory buys you either a bigger model or more context, and usually the context is worth more.
What your Mac can do
Settings → This Mac reports "Live hardware facts and receipt-backed local performance." It is measured, not estimated from a marketing number.
| Reading | What it tells you |
|---|---|
| Unified memory | Total physical memory. |
| Model memory | Estimate from the verified local model's footprint. |
| Metal GPU | Offload active, CPU active, Available, not observed, or Unavailable. |
| Local speed | Tokens per second, with the model and date observed. |
Local speed starts at Not measured yet: "Complete a local response to measure actual speed." OOMU reports the speed it actually achieved, on your machine, with your model. It does not quote a benchmark from somewhere else.
Likewise Available, not observed for Metal means exactly what it says: your Mac supports Metal, but OOMU has not yet observed an offload happening. It will not claim GPU acceleration it has not seen.
Memory decides your context ceiling
OOMU sizes the local context window from your physical memory:
| Memory | Maximum local context |
|---|---|
| Under 32 GB, or unknown | 4,096 tokens |
| 32 GB to under 64 GB | 8,192 tokens |
| 64 GB and above | 16,384 tokens |
Local responses default to a 4,096-token context and 256 output tokens, up to a hard ceiling of 16,384 context and 8,192 output. A chat or agent can request more within your Mac's tier; it cannot exceed it.
Images are bounded too: up to five per turn, 8 MB each, 20 MB in total.
Cloud services
Cloud services in the same panel manages connected providers: "Connect as many services as you need. Your keys are stored in your Mac's Keychain."
Add service connects a new one; Change key replaces a key; Manage handles the rest. Adding or changing always re-verifies against the real provider before saving.
See Connecting services for the full provider list, including custom OpenAI-compatible endpoints.
Tuning per chat
The Tuning panel beside the chat composer exposes three dials for the current session:
| Dial | |
|---|---|
| Context budget | How many tokens of conversation this chat may use, and how much is in use now. |
| Reasoning | Off or On locally; on cloud models, whichever of Low, Medium, High, Extra, Max, and Ultra that model genuinely supports. |
| Compact automatically | "OOMU shortens older context before it reaches your limit." With a threshold you set, and Compact now. |
"How hard the model thinks before replying. The available levels depend on the selected model." OOMU offers the levels a model actually has rather than showing a full scale and quietly ignoring the top of it.
Related
- How OOMU picks a model: what happens before a model is chosen.
- Connecting services: adding cloud providers.
- Hybrid AI economics: what running locally is worth.