We run a real business workload on local models: document digestion, code review, report drafting, agent-style task planning . Our machine is a single ASUS ProArt PX13 laptop with AMD's Strix Halo APU (Ryzen AI Max), 128 GB of unified memory of which ~119 GB is addressable, and a Radeon 8060S iGPU that Vulkan reports with a 132 GB window. Everything below was measured on this box, on one day, under llama.cpp with the Vulkan backend, identical 40,960-token context, identical five-question gauntlet with planted verifiable answers.
| Model | Quant | Disk | Architecture notes |
|---|---|---|---|
| Qwen3-Next-80B-A3B | Unsloth UD-Q6_K_XL | 69 GB | MoE, 3B active, GatedDeltaNet hybrid attention |
| gpt-oss-120b | MXFP4 | 61 GB | MoE, harmony format with a separate reasoning channel |
| Qwen3-235B-A22B | Unsloth UD-Q3_K_XL | 98 GB | MoE, 22B active |
| Metric | 80B-A3B | gpt-oss-120b | 235B-A22B |
|---|---|---|---|
| Generation, tok/s | 44 | 51 | 17 |
| Prompt ingest at ~6.3k tokens, tok/s | 530 | 644 | 108 |
| Time to first token, short prompt | 0.5 s | 0.8 s | 2.1 s |
| Time to first token, 6.3k prompt | 12.2 s | 9.6 s | 60.1 s |
| RAM while resident | ~76 GB | ~70 GB | ~90 GB |
| Peak thermal under sustained load | 74 °C | — | similar |
| Quality gauntlet (5 planted-answer tasks) | 5/5 | 4/5* | 5/5 |
* The asterisk is the most interesting finding in this report.
reasoning_content channel and returned an empty content string. For an interactive chat this is a retry; for an unattended worker pipeline that parses content, it is a silent no-op that looks like success. If you orchestrate agents over harmony-format models, either grant generous budgets, cap reasoning effort, or treat empty content as a hard failure.By accident, one of our gauntlet questions contained inconsistent numbers (a linear system whose solution was non-integer for an integer-only quantity). All three models detected the inconsistency and said so rather than fabricating a plausible integer. We had planted facts to catch retrieval hallucination; we ended up with an unplanned honesty test, and a 3/3 pass. Worth knowing: at Q3–Q6 quantisation on consumer hardware, refusal-to-fabricate behaviour held.
The 235B's 17 tok/s generation is usable. Its 108 tok/s prompt ingest is what actually throttled our workday: feeding it a 90 KB dashboard file meant a half-hour of reading before the first output token. The A3B-active 80B reads the same material five times faster. If your workload is "read a lot, write a little" (summarisation, code review, log digestion), prompt throughput dominates wall-clock and active-parameter count is the number to shop by.
ExecStartPre loop that waits for /dev/dri/renderD128 before launch. If your tokens-per-second dropped 4x after a reboot, check this before blaming the model.-ub) between 65 and 256 collapse performance. We run -ub 2048. The model is then fully stable on GPU.With mmap loading, "server healthy" arrives in ~15 seconds for every model, because the weights page in lazily during the first requests. First-prompt latency after a cold start is therefore the real load time. For a resident service this is a feature: restarts are cheap and the page cache does the rest.
The everyday seat went to the 80B, not the fastest model: 90% of gpt-oss's speed, a perfect gauntlet, and no empty-answer channel risk. We now run a three-tier auto-router: task keywords select FAST (80B, resident), DEEP (gpt-oss, deliberately large budgets), or MAX (235B, final reviews), the switch is a systemd drop-in swap that announces itself and is verified by an independent truth-checker that compares what our dashboard claims against what the inference server actually serves. That last habit, instruments that can disagree with reality, caught more defects this week than any code review.