Distributed Inference Archives

Aimee, a chibi anime girl with purple twin-tails, cheering triumphantly atop two AMD Strix Halo mini PCs with glowing whale glyphs on their grilles, against a purple cosmic nebula

The Whale Fits: DeepSeek V4.1 Flash Q2 on Two Strix Halos

DeepSeek V4.1 Flash Q2 (340.6 GiB) on two Strix Halo mini-PCs over a 25 GbE RoCE link: 404–448 t/s prefill, a 210k context window, and 13–15 t/s decode — no discrete GPUs.

Chibi anime character before a giant mech bearing GLM and MiMo logos with a glowing AMD core - Kyojin runs GLM 5.3 Flash (320B) and MiMo V2.6 (309B) on one Strix Halo machine

AMD Distributed Inference

Same URAM, Bigger Brain: Kyojin Brings GLM 5.3 Flash (320B) and MiMo V2.6 (309B) to One Strix Halo

Kyojin runs GLM 5.3 Flash (320B) and MiMo V2.6 (309B) on one 128 GB Strix Halo — real numbers, setup commands, and when a bigger brain beats a faster model.

Chibi Aimee with purple cat-clip pigtails and yellow eyes rides a dark mini-PC like a horse at full gallop down a floodlit stadium track, a huge plume of dust and speed lines trailing behind at golden hour

AMD Distributed Inference

Strix Halo: Value Laggard becomes competent Qwen 3.8 Flash Next with Halogen (450+ prefill & 35+ tps)

We have wanted to write about Qwen 3.8 Flash Next for quite a while, and yet, we haven't been doing that — because nothing we ran it on made it…

Banner: two purple-haired chibi anime girls affectionately hugging two AMD mini-PCs with glowing cyan accents, dark purple background

AMD Distributed Inference

Repeat after me: Good things come in TP=2 (DeepSeek V4 Flash Q4 on 2x Strix Halos RDMA)

Models continue to improve in quality for a similar memory size. Sometimes, though, one box just isn't enough room for those that want to run higher quality/intelligent models. DeepSeek V4…

DeepSeek, ds4, local LLM, RDMA, Strix Halo, tensor parallelism

GLM 5.3 Flash running on a 128GB PC — featured banner

AMD Distributed Inference

GLM 5.3 Flash on 128GB: Two Easiest Ways to Run It — and When Two Strix Halos Beat One

GLM 5.3 Flash on one 128GB URAM machine, or even Two for higher quality Here's the headline that should make you sit up: a 320-billion-parameter multimodal model — one that…

ds4, GLM, local LLM, Quantization, Strix Halo

DeepSeek V4 Flash Q2 running on Strix Halo 128GB — featured banner

AMD Distributed Inference

Running DeepSeek V4 Flash Q2 on Strix Halo 128GB

AIfinitee believes in local compute as an alternative to cloud and that the most capable open models should be runnable on hardware you actually own. This post is the recipe for…

Cluster Computing, DeepSeek, LLM Inference, Local AI, Quantization, Strix Halo

ComfyUI artwork generated on AMD Ryzen AI MAX 96GB — featured banner

AMD Distributed Inference

ComfyUI on AMD Ryzen AI MAX: 96 GB Unified Memory vs 16 GB NVIDIA

At AIFinitee, we have spent months benchmarking LLM inference on our dual-node AMD Ryzen AI MAX 395+ cluster. We have measured tokens per second across MiniMax-M2 and Qwen3.5-397B. We have…

ComfyUI, LLM Inference, Local AI, Strix Halo

Speed versus smarts: choosing the right local model size — featured banner

AMD Distributed Inference

Speed vs. Smarts: When Bigger Models Win for Local AI Coding

At AIFinitee, we've spent months chasing tokens per second. Our two-node AMD Ryzen AI MAX cluster hits 17-20 tok/s with MiniMax-M2. Our Linux-vs-Windows benchmarks showed how your OS quietly taxes…

Benchmarking, LLM Inference, Local AI, Strix Halo

Linux vs Windows for LLM inference — featured banner

AMD Distributed Inference

Linux vs Windows for LLM Inference: More Tokens/Second on Linux

Same GPU, different OS = different performance. See why Linux delivers 5-30% more tokens/sec than Windows across NVIDIA, AMD, and Intel GPUs.

Benchmarking, Linux, LLM Inference, Local AI, Strix Halo

AMD Distributed Inference MiniMax

Strix Halo Cluster

One of the beliefs we hold at AIfinitee is that LLMs will only get bigger. Bigger LLMs will mean more memory for usage in either cloud compute or local compute.…

Benchmarking, Cluster Computing, LLM Inference, Local AI, Strix Halo