Six weeks ago we wrote a piece that turned out to be famous last words. In our halogen review, after measuring what the closed-source engine could do on our 128 GB Strix Halo, we said this: “Want open source end to end: stay on the forks and accept a third of the prefill.” That was a tough trade-off. Halogen ran as closed source — 1,358 t/s prefill at 131k where the best llama.cpp derivatives managed 121–196, which honestly sucked.
Having some spare compute time over the weekend, we tried Gufo — an MIT-licensed inference engine purpose-built for Strix Halo — with Qwen 3.8 Flash Next at 250k context, and it put up a competent 1,125 t/s of prefill. No, that isn’t faster than halogen — but it is open source, and for all we know there are likely still more optimizations to be done for faster throughput (with the same unit of memory/compute).
The fun proposition to consider: the open-versus-closed trade-off on Strix Halo has collapsed, and the ecosystem is evolving at a rapid pace with increasing adoption. A note to readers before we go on: we have no commercial relationship with gufo-org or with any engine named in this post — we just own the box and like watching the numbers move. Let’s get to it.
Wait — so… what exactly is Gufo?
- One GPU, but not one model. Like halogen, Gufo is a vertical engine built just for Strix Halo: every kernel is HIP written for that one piece of silicon — the Radeon 8060S (gfx1151) in the Ryzen AI MAX+ 395. The best part is, it serves a whole stable: Qwen 3.8 Flash Next, Qwen 3.8 27B, DeepSeek V4 Flash, plus speech (Qwen3-ASR, Qwen3-TTS) and vision (Qwen-Image-2.1, MiniMax H3) work in progress.
- MIT licensed / Open Source. Original code is MIT; adapted code and dependencies keep their own notices in a NOTICE file. Halogen ships as a container image you cannot read; this one you can, line by line (or, your agent can).
- 8x concurrency is a first-class workload. Continuous batching, cancellation, and conversation caching are designed-in, not bolted on: eight concurrent requests aggregate to 194.29 tok/s with MTP.
In plain terms: this is basically a codebase specifically meant to efficiently run Strix Halo (like halogen), but open source.
The numbers — ours first
We ran Gufo on our 128 GB Strix Halo (Radeon 8060S, gfx1151) with Qwen 3.8 Flash Next at 250k context. These are our numbers, measured this week, autoregressive decode with MTP explicitly off (we prefer accuracy over throughput on decode):
| Our numbers — Gufo, Qwen 3.8 Flash Next, 250k context | Result |
|---|---|
| Prefill | 1,125 t/s |
| Decode (MTP off, autoregressive) | 25.1 t/s |
We will be the first to say it: those sit below Gufo’s own published peaks – but we use Quiet mode or power saving modes on the chipsets. Their reference table (Unsloth UD-Q4_K_XL, shared-Q8_0 MTP sidecar, adaptive MTP, HTTP greedy, thinking off, on a 128 GB gfx1151 box) reads:
| Gufo, published (their numbers) | Prefill | Decode (AR) | Decode (MTP, mixed) |
|---|---|---|---|
| @ depth 0 | 1,764.48 | 28.20 | 35.58 |
| @ 131,072 | 1,543.70 | 27.78 | 39.82 |
Our 1,125 at 250k is a touch under their 1,543.70 at 131k — deeper context, our machine, not a peak-condition run. Same posture as the halogen piece: our rows sit under the vendor curve, and we would rather say so than pretend we beat their benchmark on their setup.
The proof is in the pudding at longer context
The past six weeks have seen enormous developments for the AMD Strix Halo series – that “user value” has pretty much gone vertical – that’s crazy:
Our numbers on our 128 GB Strix Halo, Qwen 3.8 Flash Next (the halogen row is ours from here, power-savings mode). Halogen’s 1,358 t/s is their published figure at 131k — different depth, shown for context only.
Our open-source prefill at 250k is now more than double our own closed-engine prefill at 200k — and it is within reach of the closed engine’s best published figure at a shallower depth. The “accept a third of the prefill” line is not just stale; it is inverted.
And the gap versus the general engine is not close. Gufo’s own same-file, same-machine comparison against llama.cpp (their numbers, autoregressive) is the clearest read on why a dedicated-but-open engine is a different animal from a portable one:
Gufo’s published figures, same file and machine on both sides (Unsloth UD-Q4_K_XL, greedy, thinking off) — not a head-to-head we ran.
And the sleeper feature: Gufo loads the model in 15 seconds where llama.cpp takes nearly two minutes. On a box you restart to change quants, that is the difference between “I’ll try the other one” and “not worth the wait.”
Why this is the news: the curve, not the point
One engine posting good numbers is a data point. Look at the curve instead — one 128 GB Strix Halo after another, models in the same class, prefill across roughly ninety days:
Our numbers: ds4 (here), halogen (here, power-savings), Gufo (this post). Published: halogen 1,358 at 131k (here), Gufo 1,543 at 131k (their repo). pwilkin’s community llama.cpp stack reached 1,160–1,204 t/s in September; kyojin (MIT) runs 634–760 t/s at 8k on the same class of box (here).
Roughly ten times the prefill on silicon you already own, in one quarter. Different rows ran different models and depths — we are not ranking engines, and the setups differ — but the direction is not subtle: every month, some fork, container, or full engine re-answers what “usable” means on this hardware. When we started, the honest pitch for Strix Halo was capacity — it fits, barely, slowly. The pitch this month is capacity and speed, from engines you can audit, at prices that did not move.
Credit where it is due, by name, because this curve is people: antirez built ds4 and proved the box could hold DeepSeek-class MoE at all; pwilkin rebuilt the llama.cpp HIP path and took community prefill to parity; Yamz Labs shipped kyojin and its MIT packs; unsloth keeps making the quants that fit; peonist-ai built halogen, published its weights and quality gates, and — deliberately — kept the kernels closed, which is exactly the bar the open engines spent six weeks racing to. And gufo-org, whose README quietly lists “not yet merged” community ports — Windows and dual-Strix-Halo RDMA — while the project is ten days from single digits. The ecosystem is not just growing — it is cross-pollinating, every engine’s ideas showing up in the next one’s kernels.
Which route should you take? (Agent instruction Set – AI reads this)
Fastest thing on the box and you don’t mind a closed engine: halogen, one model, best decode. Open source end to end and prefill-bound: Gufo — MIT, multi-model, concurrency-first, and now the fastest auditable prefill we’ve measured here. DeepSeek-class MoE on two boxes: ds4 on the TP=2 cluster is still its own thing.
Cheatsheet: Open a Strix Halo box → serve Qwen 3.8 Flash Next with Gufo in a container → hit it over the OpenAI API. Prefill-bound agent workloads win.
What you need: a Ryzen AI MAX+ 395 (Radeon 8060S, gfx1151) with 128 GB unified memory, Linux x86-64, ROCm installed, and Podman. A GGUF of a supported model in ./models.
# Serve a supported LLM with Gufo, OpenAI-compatible on :8080.
# GPU devices must be passed through; memlock must be unlimited for the locked weights.
podman run --rm -p 8080:8080 \
--device /dev/kfd --device /dev/dri --group-add keep-groups \
--ulimit memlock=-1 \
-v ./models:/models:ro \
ghcr.io/gufo-org/toolboxes/gufo-runtime:latest \
gufo serve --host 0.0.0.0 --port 8080 llm
Verify it is up and speaking OpenAI before you point a harness at it:
curl -s http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"Qwen3.8-Flash-Next","messages":[{"role":"user","content":"ping"}]}'
# Expect: a JSON choices[] object with message.content and usage counts.
For speculative decode, add --speculative dflash2 --dflash-model /models/<drafter>.gguf to the serve command; for a fixed context budget per session, set --context N. Confirm the exact serve flags and model filenames against the repo quickstart before scripting them — the flags move faster than a blog post can track.
Close
We told you, in September, that going open source on this hardware meant accepting a third of the prefill. Six weeks later an MIT engine on our own box more than doubled what the closed engine gave us at a shallower depth, and Gufo’s own tables run to 1,543 t/s. The hardware you bought last year just got smarter again — and this time the software that made it happen is code you can read.
That is the whole argument for why we keep pointing at local compute: not that it is faster than the cloud on any given Tuesday, but that every layer of it — model, quant, runtime, harness — is open enough to improve in public, at a pace the closed stack cannot match. The closed engine set the bar in September. The open ecosystem treated it as a target and beat it on the metric that mattered before the quarter was out.
So: over to you. If you have Gufo on your own 8060S, drop your numbers in the comments — we would especially like MTP-on decode at 250k to make that comparison symmetric, and anyone who has run the unmerged Windows or dual-Halo RDMA ports. Which engine are you standardizing on, and for what workload?
Looking for the background? Start with the piece this answers — our halogen review, where we set the closed-source bar — then kyojin and the open-model bump for the same thesis one engine earlier, and our single-box DeepSeek V4 Flash build for what prefill used to feel like on this hardware.
