Quantization Archives

DwarfStar Strikes Again: Mac Studio M3 Ultra runs Qwen 3.8 Flash Next Q4 at 810 t/s Prefill

Qwen 3.8 Flash Next Q4 on a 96 GB M3 Ultra Mac Studio: 810 t/s prefill flat to 200k context, 53–61 t/s decode, 89.2 GB resident — and the wired limit to raise.

Benchmarking, ds4, Engram, LLM Inference, Local AI, Mac Studio, Quantization, Qwen

GLM 5.3 Flash running on a 128GB PC — featured banner

AMD Distributed Inference

GLM 5.3 Flash on 128GB: Two Easiest Ways to Run It — and When Two Strix Halos Beat One

GLM 5.3 Flash on one 128GB URAM machine, or even Two for higher quality Here's the headline that should make you sit up: a 320-billion-parameter multimodal model — one that…

ds4, GLM, local LLM, Quantization, Strix Halo

GLM 5.2 at 4-bit with MTP on Mac Studio — featured banner

Apple Silicon

GLM 5.2 at 4-Bit With MTP: The Easiest Way In

And the 512GB Mac Studio that Apple just stopped selling. Apple discontinued the 512GB Mac Studio. Not the chip — the M3 Ultra is still on the shelf — but…

GLM, LLM Inference, Local AI, Mac Studio, Quantization

DeepSeek V4 Flash Q2 running on Strix Halo 128GB — featured banner

AMD Distributed Inference

Running DeepSeek V4 Flash Q2 on Strix Halo 128GB

AIfinitee believes in local compute as an alternative to cloud and that the most capable open models should be runnable on hardware you actually own. This post is the recipe for…

Cluster Computing, DeepSeek, LLM Inference, Local AI, Quantization, Strix Halo