How-to guides

Choosing a local model

Add and manage the models available to OOMU, on this Mac and in the cloud.

Everything in Settings → Models & Providers, and what your Mac can actually do with it.

The three system models

OOMU's own models sit at the top, under System models: "OOMU's router, local assistant, and optional vision model."

Role	Model	What it does
Understanding model	OOMU System One (Bosun v3.1)	Reads every request and chooses the path. Required.
Local assistant	Gemma 4 E4B, or Gemma 4 E2B on a Mac with 8 GB of memory	Everyday reasoning on this Mac. Required.
Vision	The vision projector that matches your assistant	Lets the assistant read attached images. Optional, and not part of first-run setup.

OOMU chooses between E4B and E2B from your Mac's memory, once, and only lists the one it chose. Chats that used E2B before, for example ones imported from Beta 3, still open and read normally.

Each shows a state: Ready, Missing, or Changed. Download fetches the file from its official source, Locate File… uses a copy you already have, and Replace File… swaps it.

Every one is verified by SHA-256 against OOMU's embedded manifest before it is accepted. Changed means the file on disk no longer matches what OOMU recorded: reinstall it rather than working around it.

Why the router is its own model

The understanding model is Bosun v3.1, a 0.6-billion-parameter model built for one job: choosing between a short list of options. It is not a general assistant, and it is not asked to be one. It reads up to 8,192 tokens and writes nothing; it scores the options and OOMU takes the most likely, which is how it returns a decision in tens of milliseconds. It also updates on its own schedule, separately from the app. See How OOMU picks a model.

Adding your own models

Under On this Mac:

"Add other models you've downloaded. OOMU uses the default for work on this Mac unless a chat or agent chooses another."

Choose Add models and pick one or more .gguf files. OOMU runs on llama.cpp, so any GGUF that llama.cpp can load will work.

For each added model you can:

Action
Make default	Use it for local work unless a chat or agent overrides.
Rename	Change the display name.
Verify	Re-check the file against its recorded hash.
Edit / Reassign model	Change what points at it.
Remove	Delete it from the list.

Each model card shows where the file came from and when it was last verified, and how many agents and chats use it: "Used by {agents} agents and {chats} chats."

You cannot orphan work. Removing a model that is in use requires reassigning its chats, agents, and default first. OOMU shows you the count and asks where to move them. The same rule applies to cloud services: "Reassign this service's chats, agents, and default before removing it."

Which models to look for

Gemma 4 in Google's QAT GGUF builds is the family OOMU ships against: the E4B build on most Macs, and E2B on 8 GB Macs. Qwen and Llama GGUF builds also work. As a rule of thumb, a Q4 quantization of a small instruct model is the right starting point on 16 GB; larger memory buys you either a bigger model or more context, and usually the context is worth more.

What your Mac can do

The This Mac card in Settings → Models & Providers reports "Live hardware facts and receipt-backed local performance." It is measured, not estimated from a marketing number.

Reading	What it tells you
Unified memory	Total physical memory.
Model memory	Estimate from the verified local model's footprint.
Metal GPU	Offload active, CPU active, Available, not observed, or Unavailable.
Local speed	Tokens per second, with the model and date observed.

Local speed starts at Not measured yet: "Complete a local response to measure actual speed." OOMU reports the speed it actually achieved, on your machine, with your model. It does not quote a benchmark from somewhere else.

Likewise Available, not observed for Metal means exactly what it says: your Mac supports Metal, but OOMU has not yet observed an offload happening. It will not claim GPU acceleration it has not seen.

Memory decides your context ceiling

OOMU sizes the local context window from your physical memory, after setting aside room for macOS and for the model itself:

Memory	Maximum local context
Unknown	4,096 tokens
8 GB	8,192 tokens
16 GB and more	32,768 tokens

32,768 tokens is the ceiling for work on this Mac, and a single reply can run to 8,192 tokens. A chat or agent can ask for less; it cannot exceed your Mac's tier. The Tuning panel shows the figure for your Mac: "This Mac supports up to {count}K tokens of local context."

Images are bounded too: up to five per turn, 8 MB each, 20 MB in total.

Cloud services

Cloud services in the same panel manages connected providers: "Connect as many services as you need. Your keys are stored securely on this Mac."

Connecting and managing services happens in Connections; this panel links there. Adding or changing always re-verifies against the real provider before saving.

See Connecting services for the full provider list, including custom OpenAI-compatible endpoints.

Tuning per chat

The Tuning panel beside the chat composer exposes three dials for the current session:

Dial
Context budget	How many tokens of conversation this chat may use, and how much is in use now.
Reasoning	Off or On locally; on cloud models, whichever of Low, Medium, High, Extra, Max, and Ultra that model genuinely supports.
Compact automatically	"Shorten older context on this Mac before it reaches your limit." With a threshold from 50% to 90% (80% by default), and Compact now. Summaries are saved, so returning to a long chat does not redo the work. Compaction is for local models; it is off for cloud models.

"How hard the model thinks before replying. The available levels depend on the selected model." OOMU offers the levels a model actually has rather than showing a full scale and quietly ignoring the top of it.

How OOMU picks a model: what happens before a model is chosen.
Connecting services: adding cloud providers.
Hybrid AI economics: what running locally is worth.