GLM 5.3 Flash on one 128GB URAM machine, or even Two for higher quality
Here’s the headline that should make you sit up: a 320-billion-parameter multimodal model — one that trades blows with Claude Opus 4.8 on coding and agentic benchmarks — now fits in a laptop or Mini PC’s worth of unified memory. No cloud account, no API key, no per-token meter running (just electricity — and yes, hyper-expensive unified memory).
We’ve spent a lot of words on this site arguing that local compute is a real alternative to the cloud. Today we’re going to prove it with the strongest case yet (as of writing): GLM 5.3 Flash running on a 128 GB machine you already own — or, if you’ve been following our Strix Halo series, on two of them for the even better GLM 5.3.
The catch is that “easiest way” isn’t one thing. There isn’t a single blessed path — we think there are two decently simple routes, and which one is right depends on how you got here. So we’re going to give you both, keep it simple, and then take the obvious next step: what happens when one 128 GB box isn’t enough, and when two is better.
Let’s get to it.
First, what exactly is GLM 5.3 Flash?
GLM 5.3 Flash (codename ox-alpha) is Z.ai’s open multimodal model. The specs that matter:
In plain terms: this is a frontier-grade coding and agentic model compacted enough to run at home. That wasn’t true six months ago at this quality level. So if you’re wondering — yes, that memory you bought earlier has arguably appreciated in “intellectual terms”: same RAM, but a lot smarter than last quarter.
But 320B parameters is a lot of weights. Let’s talk about how it fits in 128 GB at all.
The quant math: why 128 GB is suddenly enough
Unquantized, GLM 5.3 Flash is an ultra-chonky 641 GB in BF16. That’s a datacenter box. What changed is aggressive quantization — and a very clever kind of it.
Here’s the ladder, straight from Unsloth’s model page:
| Quant | Size | Accuracy vs BF16 | Realistic RAM |
|-------|------|------------------|---------------|
| BF16 (raw) | 641 GB | 100% | ~650 GB |
| UD-Q4_K_XL | 199 GB | 92% | ~210 GB |
| UD-IQ3_XXS (3-bit) | 120 GB | 82% | 128–150 GB |
| UD-IQ2_XXS | 101 GB | 76% | ~115 GB |
| UD-IQ1_M | 97.5 GB | 73% | ~110 GB |
| **UD-IQ1_S (1-bit)** | **93 GB** | **71%** | **~100 GB** |
Read that bottom row again: a 1-bit quant at 93 GB. That’s the smallest you can go with this model and still call it GLM. 71% of the accuracy at a fraction of the size (kudos to the Unsloth team for their ongoing effort).
The practical pointer: if your machine has ~128 GB of unified memory, you’re in. Unsloth explicitly notes the 3-bit quant runs on 128 GB Macs and the Nvidia DGX Spark. Strix Halo isn’t named on their page, but it’s the same category — a 128 GB unified-memory APU. If a Mac fits it, a Strix Halo or DGX Spark fits it.
Now — and this is important — the “1-bit” number and the “ds4 Q2” number are not the same artifact. They’re two different quantization approaches from two different teams, both landing at roughly the same ~90–96 GB footprint. That’s exactly why we’re giving you two routes instead of pretending there’s one blessed path.
Route A: The Unsloth way (easiest, Mac/Spark/Halo-friendly)
If you want the absolute lowest barrier to entry — and you’re on macOS or a DGX Spark — Unsloth Desktop is your friend. It’s a one-click desktop app (macOS/Windows/Linux) that downloads the GGUF and runs it for you: just load it and go, or tweak the advanced settings for KV-cache quantization or a larger context.
The 1-bit UD-IQ1_S at 93 GB is the smallest footprint, and it needs only ~100 GB of RAM (you want a little headroom for context). If you want a bit more fidelity and have a full 128 GB, the 2-bit or 3-bit quants (UD-IQ2_XXS, UD-IQ3_XXS) are the sweet spot.
The Unsloth route also works through llama.cpp if you prefer the command line — their PRs tend to stay more up-to-date than LM Studio’s. (Advanced users can build their own custom forks.)
When to pick Route A: you’re on a Mac or Spark, you want the fewest moving parts, and you don’t want to compile anything. Download, point, run.
When to skip it: you’re on a Strix Halo running Linux, you want the coding agent + server built in, or you want to go distributed. That’s Route B.
Route B: The DS4 way (best if you’ve been reading this site)
This is the one that should feel familiar. If you’ve been following our DeepSeek V4 Flash posts and the AMD 395+ AI MAX Two-Node Cluster Guide, you already know DwarfStar (ds4) — antirez’s self-contained native inference engine. His continued effort has let ordinary users — you, me — enjoy the latest open-source mini-Opus/Fables of sorts.
Here’s the news: ds4 has recently been updated to serve GLM 5.3 Flash. It ships its own asymmetric Q2 GGUF — a single file at ~96 GB, and a glm53-q2 download ID that maps to exactly the 96/128 GB class of machine. This is the same quantization philosophy we wrote about with DeepSeek V4 Flash: only the routed MoE experts get quantized hard, so quality stays surprisingly high for the size.
And because it’s ds4, you get more than a bare runner. It includes:
ds4-agent — a native coding agentds4-server — an OpenAI/Anthropic-compatible server you can point real tools atOne Strix Halo, the copy-paste version (sorry, no GB10s here for research)
On a single 128 GB Strix Halo running Linux with ROCm, this is the whole setup:
# 1. Pull the GLM 5.3 Flash Q2 weights (~90 GiB)
./download_model.sh glm53-q2
# 2. Build for Strix Halo / ROCm
make strix-halo
# 3. Run it
./ds4 --rocm -m ./ds4flash.gguf
That’s it. Three commands and you’ve got GLM 5.3 Flash resident on your own APU. The --rocm flag is the Strix Halo-specific bit — the README points to STRIXHALO.md for the reproducible ROCm 10.0 container build and GTT config if you want the fully tuned path.
For a coding-agent setup on that same single box, fire up the server with a context sized to your RAM:
./ds4-server --ctx 100000 --kv-disk-dir /tmp/ds4-kv --kv-disk-space-mb 8192
A 1M-token context eats ~26 GB of RAM, so on a 128 GB machine you’ll want a context in the 100k–300k range unless you’re deliberately streaming (we suspect you can go higher — DeepSeek’s KV-cache architecture typically peaks around 10–12 GB when quantized properly). Then point your favorite agent at it — ds4 documents configs for opencode, Pi, Codex CLI (Responses API), and even Claude Code (via its Anthropic endpoint). If you’ve ever wanted a local GLM agent with real tool-calling, this is the cleanest on-ramp we’ve found.
Okay, and what if (gasp!) I got TWO Strix Halos?
Here’s where this site’s whole thesis comes together. We’ve written about distributed inference before — that a pair of boxes can beat a single bigger one at load-bearing larger/better models, and that local clusters are the real cloud alternative. GLM 5.3 Flash is the perfect excuse to revisit it, because the model gives you two very different reasons to want a second machine.
Reason 1: Fit a bigger quant (quality)
On one 128 GB box, you’re locked to the ~90–96 GB Q2/1-bit class. That’s great for, say, 71–73% of full fidelity.
With two 128 GB Strix Halos, the math changes. ds4 does distributed inference over pipeline and tensor parallelism — it can span TCP (two boxes over a network) the way it spans RDMA (two Mac Studios) and CUDA GPUs. Split the model across both machines and you can step up to the glm53-q4 at ~178 GB, or even the full GLM 5.3 (211 GB IQ2) that needs more than any single 128 GB box can hold.
More parameters resident means better accuracy (read it as an intellect proxy for coding or other use cases). It’s the cleanest quality upgrade you can buy without touching a datacenter, at a fraction of the wattage.
Reason 2: More speed, not just more size
Even if you stay at Q2, splitting the MoE experts across two APUs means two memory buses doing the work. Our earlier two-node benchmarks on the AMD 395+ AI MAX cluster showed exactly this pattern — the same model, more prefill per second, which is the part that matters for AI agentic harnesses (they read a lot more input than they generate). If you care about interactive coding-agent speed (and you should), two boxes is often the difference between “usable” and “smooth” — especially on lower-memory-bandwidth hardware.
The two-node recipe
This is the same shape as our AMD 395+ AI MAX Two-Node Cluster Guide, so if you’ve built that, you’re already 80% there. On each Strix Halo:
./download_model.sh glm53-q2 # or glm53-q4 for the bigger quant
make strix-halo
./ds4 --rocm -m ./ds4flash.gguf --distributed ...
Then join them over your network fabric — TCP for a plain pair, RDMA if you have the low-latency link. The README’s distributed section walks through the tensor/pipeline parallelism flags; the two-node cluster guide on this site has the hardware and network setup you’ll want in front of you. We’ll leave SSD streaming for a separate post.
One Strix Halo: the easy Q2/1-bit route, ~90 GB resident, a genuinely strong agentic model.
Two Strix Halo: step up to Q4 or the full model, or keep the same quant and go faster. The quality ceiling lifts, the token rate climbs, and you’re running a model that simply cannot fit on a single consumer box.
Wait — can two of them run the full GLM 5.3?
Two 128 GB Strix Halos pool into 256 GB of unified memory — and that changes the ceiling again. It’s enough for the full, chunkier GLM 5.3 (not the Flash): ds4’s glm53-full-q2 weighs in around ~197 GB, a quant that simply can’t sit in any single 128 GB box. Two APUs, one model, no SSD streaming required. That’s the upgrade path we’re genuinely excited about — and the subject of a deeper post soon. Consider this your spoiler.
Which route should you take?
Here’s the one-sentence cheat sheet:
> On a Mac or Nvidia Spark, take Route A (Unsloth) — fewest parts, done. On a Strix Halo — especially if you’ve already got a ds4 setup or a two-node cluster from our earlier guides — take Route B (ds4) and unlock the distributed upgrade path.
Either way, you now have a frontier-class 320B model running on hardware you own, for the cost of your electricity. That’s the whole point of this site.
If you build either route — or the two-node version — we’d love to hear your tokens-per-second numbers. Drop them in the comments; we’re genuinely curious how far a pair of 128 GB APUs can push this thing.
Looking for the background? Start with our DeepSeek V4 Flash Q2 on Strix Halo 128GB guide for the ds4 basics, and the AMD 395+ AI MAX Two-Node Cluster Guide for the distributed build. And if you’re on Apple Silicon, our GLM 5.2 at 4-Bit With MTP piece is the natural sibling.
