ds4 Archives - AIFinitee

DwarfStar Strikes Again: Mac Studio M3 Ultra runs Qwen 3.8 Flash Next Q4 at 810 t/s Prefill

Qwen 3.8 Flash Next Q4 on a 96 GB M3 Ultra Mac Studio: 810 t/s prefill flat to 200k context, 53–61 t/s decode, 89.2 GB resident — and the wired limit to raise.

Benchmarking, ds4, Engram, LLM Inference, Local AI, Mac Studio, Quantization, Qwen

Banner: two purple-haired chibi anime girls affectionately hugging two AMD mini-PCs with glowing cyan accents, dark purple background

AMD Distributed Inference

Repeat after me: Good things come in TP=2 (DeepSeek V4 Flash Q4 on 2x Strix Halos RDMA)

Models continue to improve in quality for a similar memory size. Sometimes, though, one box just isn't enough room for those that want to run higher quality/intelligent models. DeepSeek V4…

DeepSeek, ds4, local LLM, RDMA, Strix Halo, tensor parallelism

GLM 5.3 Flash running on a 128GB PC — featured banner

AMD Distributed Inference

GLM 5.3 Flash on 128GB: Two Easiest Ways to Run It — and When Two Strix Halos Beat One

GLM 5.3 Flash on one 128GB URAM machine, or even Two for higher quality Here's the headline that should make you sit up: a 320-billion-parameter multimodal model — one that…

ds4, GLM, local LLM, Quantization, Strix Halo