Which model for what — a reference matrix
One table to come back to when you choose a model for a task. A reference as of 11.10.2026 — our own observations from trials on our own machine, not a benchmark. New on 11.10.2026: a trial on real accounting documents and the lesson that how you call a model matters as much as which model you pick. The verdict is one of three: use, niche or avoid. A companion to the lessons on the trial method (I20) and on reading documents from a photo (I19).
01What you will learn
- Which local model we take as the universal worker, which for the riskiest tasks, and which we never put on data at all.
- Which search and vision models work on Bulgarian — and which fail.
- How the cloud Claude models line up and when we use which.
- Three settings that let big models run on a desktop machine, and why "raw" reasoning models are not for data.
- How to call "thinking" models (
think, the right request endpoint) for document extraction — and why label-based rules come before the model.
02Before you start
- Read this frame first: this is not a ranking. The numbers come from small trials on our own documents (6 leak-detection cases, 3 questions on a regulation, 10 tasks, 5 search queries). They give direction, not proof.
- All speeds ("tps" = tokens per second) were measured on one local machine of ours with 128 GB of shared memory; the cold start was discarded, 3 repetitions. On your machine it will differ.
- You know what a "guard" is (a check for leaking personal data) and what "compliance" means (questions on a regulation). The method is in I20.
- For the local models you need Ollama 🔒 local; for Claude — an account 🌐 global model.
03Steps
-
How to read the matrix
✅ use — proven for that role in our trials; ◐ niche — fits one specific case, with a caveat; ⛔ avoid — not for structured output or data. Models carry the exact names from the public catalogues, checked on 01.10.2026.
📅Shelf life: 01.10.2026Models change within weeks. This is a reference as of that date and our own experience, not an independent assessment. Before deciding on work with personal data or on a quote, run your own small trial and write the date down. -
Text models — the workers 🔒 local
Six trial passes between 14 and 19.09.2026. The key rules: the universal worker is small; the biggest are not necessarily better; reasoning models need a forced format.
Model Our experience Verdict gemma3:12b~25 tps · guard 6/6 · compliance 3/3 · dates ✓ · nomenclature 6/6 ✅ The universal worker — reader, guard, compliance, dispatcher qwen3:14b~22 tps · guard 5/6 · compliance 3/3 · dates ✓ · nomenclature 6/6 ✅ Second universal — an alternative to 12b qwen3:32b~11 tps · vertical 10/10 · guard 6/6, no false alarms · compliance 3/3+3/3 · 0 invalid JSON ✅ Premium guard/compliance — dense model, slow nemotron-3-super(87 GB in Ollama)~22 tps · vertical 10/10 · guard 6/6 · compliance 3/3+3/3 · runs with context 8k + flash attention + KV q8_0 · answers Bulgarian prompts in Russian ✅ with a caveat — perfect on the test but weak for Bulgarian; the official language list does not include Bulgarian Mistral-Small-4-119B(Apache-2.0)~50 tps · guard 6/6 · compliance 3/3+3/3 · invoice + EU ✓ · on our setup it loaded with Ollama 0.34.2; the older 0.23.2 did not recognise the architecture ✅ Premium compliance/guard — about 43 GB, for high-stakes work, not daily. Weights on Hugging Face; there is no official tag in the Ollama library (checked on 01.10.2026) deepseek-r1:32b~10 tps · guard 6/6 · compliance 3/3 — the strongest of the smaller ones ◐ Top compliance, but slow; raw, it spills its reasoning qwen2.5-coder:7b~44 tps · sums ✓ · guard 0/6 and 6 false alarms · date 2023 instead of 2026 ◐ Only calculations and code — not for judgment mistral:7b~44 tps · extraction ✓ · guard 5/6, but 3 false alarms ◐ Extraction ok; the guard is noisy gemma3:4b~73 tps · fast, but compliance 0/3 · dates ✗ ◐ Light routing only — not for data qwen3:30b-a3b~78 tps · raw: 0/10 valid JSON (reasoning leaks) · with format=json: 0 invalid out of 10 · guard 6/6 · in free text: 3 invented refusals◐ Tamed with format=json — fine for extraction, not for refusals or free text Qwen2.5-7B-Instruct-1M(Apache-2.0)~47 tps · vertical 8/10 · guard 4/6 (missed 2 leaks) · with unbounded context it froze the machine; we capped it at 8k ◐ Not for a guard — long context only with a cap. Weights on Hugging Face; there is no official tag in the Ollama library (checked on 01.10.2026) gemma3:12b-it-q8_0(13 GB)Q8 variant of 12b; not in the three-pass test ◐ More precise, heavier — Q4 is enough for everyday work (our choice, not a measurement) gemma3:27b~12 tps · guard 6/6 · nomenclature 1/6 · slow ⛔ No gain over 12b for structured data qwen3:8b~39 tps · guard 0/6 (missed everything) ⛔ Guard-blind — dangerous for judgment llama3.1:8b~41 tps · wrong total, wrong dates, guard broke the JSON ⛔ Silent errors — not for data gemma4:26b-a4b(q8 and q4)11.10.2026, document extraction: without an explicit think: false— 32–36% and up to 10 failures; with it — q8: 64%, 0 failures, median ~7 s✅ A reliable second opinion for extraction — only with an explicit think: falsegpt-oss:20b~58 tps · raw: 0/10 valid JSON (reasoning leaks) · 11.10.2026: via /api/chatwiththink: "low"— 55% on extraction (from 14%)◐ Only when called right — /api/chat+think: "low"; still not for Bulgarian (see the addendum of 23.09)deepseek-r1:70b~5 tps · raw: 0/10 · very slow · 11.10.2026: on extraction it timed out on most documents (~2 min each) ⛔ Not for extraction; heavy reasoning, 32b is enough ✅A big model is not the problem — unbounded context isThe biggest model in the table passed the whole battery once the context was capped and the cache quantized. The earlier crash came from a model with a context of up to 1 million tokens that ate the memory until the machine froze. Cap the context, don't fear the size. -
Search models — Bulgarian 🔒 local
A trial of 5 Bulgarian queries over our own documents. The criterion is which document comes first and how widely the right ones separate from the wrong ones.
Model Our experience Verdict snowflake-arctic-embed25/5 · all in first place · widest separation (0.08–0.25) ✅ Best for Bulgarian search bge-m35/5 · all in first place · 0 invalid vectors ✅ Excellent, on par granite-embedding:278m5/5 (one in third place) · tiny and fast ◐ Light fallback nomic-embed-text3/5 · blind on 2 Bulgarian queries ⛔ Rejected for Bulgarian -
Vision models and reading from a photo 🔒 local
The lesson on reading documents (I19) describes pass 3, when the four open vision models failed on Cyrillic. Pass 4 changed the picture for one of them: the problem with
qwen2.5vl:7bwas a missing smallnum_ctx, not the model.Model / solution What happened Verdict OCR (Tesseract, Bulgarian) → gemma3:12bCorrectly extracted the total and all 7 line items from an invoice ✅ This works — reading from a photo qwen2.5vl:7bWith a small num_ctxit read an electricity-meter value correctly ("018565,7") in ~4 s; the pass 3 failure (timeout) came from the missing context limit✅ OCR works (small context) minicpm-v:8bGarbled text ("СУМА" became "КВМА") ⛔ Fails on Cyrillic granite3.2-vision:2bMisread the meter digits ⛔ Fails on Cyrillic llava:34bRefused: "I don't see an image" ⛔ Fails on Cyrillic A scene narrator is not OCR. For describing a picture (not reading text), our trials on 16.09.2026 with a small
num_ctxgave good results:Model Our experience Verdict qwen2.5vl:7bDescribes scenes vividly and correctly in Bulgarian (scene, not text) ✅ Fast narrator by default gemma3:12bWarm, accurate Bulgarian narration ✅ Narrator with richer language gemma3:27bPoetic and emotional, but slow and heats the machine (~60 °C under our load) ◐ The special-occasion narrator — not for the marathon -
The Claude cloud models 🌐 global model
The current lineup per Anthropic's official page, checked on 01.10.2026. Prices are not in this reference — see the official page; if you write a quote, calculate in euro with VAT on the day of the quote. The "How we use them" column is our experience, not the vendor's recommendation.
Model Official description How we use them Claude Fable 5.1For demanding reasoning and long-horizon agentic work · context 1M · output up to 128K The hardest work: architecture documents and the final check Claude Opus 5.5For long-running agentic coding and knowledge work · 1M · 128K The main "thinking" model in working sessions Claude Sonnet 5.5The best combination of speed and intelligence · 1M · 128K Executing a clearly given address and task Claude Haiku 4.5The fastest model with near-frontier intelligence · 200K · 64K Checks and routine 🧭Our delegation ruleThe smallest suitable model for routine; the stronger one only where thinking happens. Anything irreversible waits for a human. This arrangement is our own choice, not a vendor ranking. -
Three settings that saved the big models 🔒 local
You cap the context, quantize the cache and force the format. The two variables are set on the Ollama server; read them in the official FAQ before touching them.
server environment · less memory for the cacheOLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0request body · context cap, no keep-alive, forced JSON{ "model": "<model>", "prompt": "<task>", "stream": false, "format": "json", "keep_alive": 0, "options": { "num_ctx": 8192 } }⚠️Rule 00One heavy model at a time,keep_alive: 0, load at or below 80%. A forgotten service once caused an out-of-memory crash — before a heavy task, check what else is hanging around.🏆The two golden rulesDefinitions beat the model — a clean vocabulary in the prompt lifts accuracy more than a pricier model. An imposed schema rescues — even a weak model returns valid JSON when the format is set. The full method is in I20. -
Addendum of 23.09.2026 — discipline in Bulgarian
In a council run in Bulgarian (one task, strict rules)
gpt-oss:20binvented numbers and drifted into Russian — don't use it for tasks in Bulgarian.gemma3:27binvented nothing, but added nothing new either.gemma4:26bis a candidate for a fast classifier — not yet a verdict. No local model proved fit to be a critic or a judge (our experience, a small sample; details in the lesson on the council).🗣️An adjacent finding: Voxtral TTSVoxtral-4B-TTS-2603from Mistral has open weights, voice cloning and 9 languages — Bulgarian is not on the list. The licence is CC-BY-NC-4.0 — non-commercial use only. As of 19.09.2026 the self-hosting stack had no image for our ARM machine, so for now we use it through the provider's API (our choice).📄An adjacent finding: Mistral OCR 4A strong model for documents, but as of 19.09.2026 self-hosting it was offered only to enterprise customers on request — so it was dropped. The local path (qwen2.5vl:7bwith a smallnum_ctx, or OCR → text model) stays. ⚠️ The provider's terms were not re-checked on 01.10.2026. -
Addendum of 10–11.10.2026 — extraction from accounting documents 🔒 local
We tried 11 models on 22 real accounting documents of one company — utility and telecom invoices, an insurance document, online-payment receipts and the company's own outgoing invoices. The documents were processed only in our machine's memory and deleted after every round.
📏What we measure — and what we don'tThe number is the share of documents whose extracted fields pass the deterministic checks: the company-ID checksum, the arithmetic "base × rate = VAT" and the dates. It is not field accuracy — for the incoming documents we had no ready answer key. It says "the numbers are internally consistent", not "everything was read correctly".Model · how it is called Passed the checks Failed Median per document qwen3:14b·/api/generate68% 1 2.5 s gemma3:12b·/api/generate73% 1 30 s gemma4:26b-a4bq8 ·think: false64% (without it: 32%) 0 7 s mistral:7b59% 1 9 s gpt-oss:20b·/api/chat·think: "low"55% (the old way: 14%) 2 7 s Mistral-Small-4-119B50% 2 10 s qwen3:14b·/api/chat·think: false41% 5 46 s qwen3:32b·gemma3:27b59% · 32% 1–6 25–36 s Full thinking ( think: true, qwen3)27–41% 7–12 41–113 s deepseek-r1:70b·think: truetimes out on most documents (~2 min each) ✅How you call a model matters as much as which model you pickgemma4has a "thinking" capability, and ifthinkis not set explicitly it thinks silently and returns nothing — an explicitthink: falsedoubled the result and removed every failure.gpt-osswantsthink: "low"as a string (notfalse— that breaks its template) and structured output only through/api/chat. Forqwen3it is the opposite:/api/chatmade it worse, so it stays on/api/generate. Full thinking did not help extraction and was 5–15 times slower.gemma4 · extraction · explicitly without thinkingPOST /api/generate { "model": "gemma4:26b-a4b-it-q8_0", "prompt": "<task + document text>", "think": false, "format": <JSON schema of the fields>, "stream": false, "options": { "num_ctx": 8192, "num_predict": 2048, "temperature": 0 } }gpt-oss · extraction · "low" thinking as a stringPOST /api/chat { "model": "gpt-oss:20b", "messages": [{ "role": "user", "content": "<task + text>" }], "think": "low", "format": <JSON schema of the fields>, "stream": false, "options": { "num_ctx": 8192, "num_predict": 900, "temperature": 0 } } // read message.content; message.thinking is not kept🏆Rules before the modelWith a known invoice layout the fields are read by their labels ("VAT number", "Tax base", "Amount due") and checked deterministically: 100% correct in 0.13 s per document, without a single model call. The model only fills what is missing. Hence the pattern we use: a fast model first → a second opinion only when the checks fail → a per-supplier template, confirmed by a human, from the second month on.🧪Real documents catch what tests missEvery real-data run in this round found a defect that hundreds of synthetic tests did not: a truncated model answer, a date with slashes instead of dots, a model writing the field name instead of a value. Measure on your own documents — in memory, nothing written to disk — before you trust a number.A model with a new architecture may need a newer Ollama release than the one you run. Instead of upgrading the working server, run a second, newer one side by side.
04Check
1. What is this matrix?
2. Which is the universal local worker in our trials?
3. How does a reasoning model (for example gpt-oss:20b) behave "raw" when asked for JSON?
4. What caused the crash with the model that had a context of up to 1 million tokens?
5. What doubled gemma4's result on document extraction?
05What's next
06Sources
- The Ollama library — names and tags of the local models: gemma3, gemma4, qwen3, qwen2.5vl, nemotron-3-super, bge-m3, snowflake-arctic-embed2.
- Mistral-Small-4-119B-2603 — the weights and the Apache-2.0 licence.
- Qwen2.5-7B-Instruct-1M — a long-context model, Apache-2.0 licence.
- Voxtral-4B-TTS-2603 — open weights, CC-BY-NC-4.0 licence.
- Claude — models overview — the official lineup, context and output limits.
- Ollama — FAQ — the cache settings and capping the context.
- Ollama — releases — when each architecture was added.
- Ollama — thinking — the
thinkfield and its levels. - Ollama — structured outputs —
formatwith a JSON schema. - /api/generate and /api/chat — the two request endpoints.
- KAGAMI's own trials, 14–19.09.2026: small samples of Bulgarian documents on our own machine. Our experience, not a benchmark.
- KAGAMI's own trials, 10–11.10.2026: 22 real accounting documents, 11 models, three rounds, processed in memory only. Our experience, not a benchmark.