Coming soonOOMU is launching soon. Downloads will open at launch.
OOMU
← All documentation

How-to guides

Choosing a local model

OOMU does its everyday thinking on your own Mac, using an open model you download once. Here is what to install, and how to match a model to your hardware.

Last updated: July 27, 2026.

OOMU keeps your routine work on your own machine. That work is done by an open model that lives on your Mac, so most of what you do is private and free to run. This page recommends which model to download and which to reach for as your needs grow.

Our official preference is the Gemma 4 family, in the QAT GGUF builds Google publishes. They are small, fast, genuinely capable, and free under the Apache 2.0 license.

Start here

Required. Download the official Gemma 4 E2B (instruction-tuned) model before anything else. OOMU uses it for its basic on-device work, and every install expects it to be there. google/gemma-4-E2B-it-qat-q4_0-gguf is about 3.35 GB and runs on essentially any modern Mac.

Everything else on this page is optional. Add a larger or specialized model only when you have a reason to, and only if your Mac has the memory for it.

What the names mean

The model names look cryptic, but each part is simple:

  • Gemma 4 is Google's open model family. It is free to download and runs on hardware you already own.
  • IT means instruction-tuned: trained to follow requests and hold a conversation. Always choose an IT model for OOMU.
  • QAT means quantization-aware training: the model was trained to stay accurate even after it is compressed, so a small file still gives good answers. This is the build we prefer.
  • GGUF is the file format local apps read. Q4_0 is the compression level inside it, and it is the balanced default.
  • E2B and E4B are the "effective" sizes. They use a memory-saving design that lets a small model punch above its weight. 12B is a larger, more capable model.

Match the model to your Mac

All three below are the official Google QAT GGUF builds. Each Hugging Face page holds a few files; choose the Q4_0 one unless you have a specific reason not to.

ModelBest forDownloadComfortable on
Gemma 4 E2B IT (required)The baseline OOMU needs. Fast and light.about 3.35 GBAny modern Mac, including 8 GB
Gemma 4 E4B ITA clear step up in quality, still light.about 5.15 GB16 GB and up
Gemma 4 12B ITThe most capable everyday choice.about 6.98 GB24 GB and up

These are download sizes; running a model needs some memory beyond the file itself, so leave headroom for the rest of your Mac.

Have a workstation-class Mac with 32 GB or more? Google also publishes larger 26B A4B and 31B Gemma 4 builds. You can browse the full official family on Hugging Face.

Tune it for your Mac

Which model you can run is only half the story. The other half is the context budget in the app's Tuning panel: how much text the model holds in mind at once. A larger budget lets OOMU work across longer documents, but it uses more memory and can slow the first reply. If answers start to lag or your Mac gets warm, lower the budget. If you have memory to spare and work with long files, raise it.

The pairings below are a starting point, not hard limits. Real speed depends on your chip, your memory, and what else is open, so treat them as a place to begin and adjust to taste.

Your MacEveryday modelStarting context budgetNotes
M1 or M2, 8 GBGemma 4 E2B IT4K to 8KE2B is a comfortable fit. Keep the budget modest so replies stay quick and memory stays free for the rest of your Mac.
16 GBGemma 4 E4B IT8K to 16KRoom for longer documents, or run E2B with a larger budget for snappier replies.
24 GBGemma 4 12B ITup to 16KThe most capable everyday setup, with headroom to work across long files.
32 GB and upGemma 4 12B ITup to 32KPlenty of room. Even an M1 with 32 GB runs 12B smoothly, so raise the budget when you want the full picture.

Two more notes from the same Tuning panel:

  • Reasoning lets the model think before it answers. Google's guidance is to leave it on in most cases; it is how Gemma reaches its best answers, and the extra moment is usually worth it. Keep it on by default, and consider lowering it only for the simplest, speed-critical tasks.
  • If a model ever feels slow to start, it is usually the context budget, not the model. Step it down a notch before you reach for a smaller model.

Optional variants for specific needs

These are alternatives to the official builds, useful in particular situations. None of them replaces the required E2B model above.

Faster and leaner: Unsloth

Unsloth republishes Gemma 4 as "Dynamic" GGUF quants that use roughly three times less memory while staying close to the original accuracy. The E2B build also ships a small speculative-decoding helper (mtp-gemma-4-E2B-it.gguf) that recent versions of llama.cpp pick up automatically for extra speed. There is a mobile-optimized build as well.

Apple Silicon native: MLX

MLX runs models natively on Apple Silicon and can be very fast on a Mac. Browse the Gemma 4 MLX collection, for example gemma-4-e2b-it-4bit and gemma-4-e4b-it-4bit.

One caveat worth knowing. At the time of writing, the compressed (4-bit) MLX ports of Gemma 4 had a known bug: quantizing the model's per-layer embedding produced garbage output. Use an mlx-vlm build that includes the fix, or the full-precision version, which works reliably. If you are not sure, the official GGUF builds above are the safe path.

Fewer guardrails: uncensored builds

Some community builds remove the model's built-in refusals; you will see these called "abliterated" or "uncensored." They can help with legitimate work the base model over-blocks, such as security research, mature fiction, or frank discussion of a sensitive topic.

Please read before using one. These are community projects, not official Google releases, and they are not a replacement for the required E2B model. Removing safety guardrails puts responsibility for the output entirely on you. Use them deliberately, and keep them out of shared or unattended workflows.

Where OOMU keeps your models

In OOMU, open Settings. Under Models you can see and manage what is installed, and General lets you choose the folder where your local models live. OOMU checks that folder when it starts, so a model you add is ready the next time you launch.