An inference engine written for a single chip: Gufo, with two Framework users, got 1,000 stars in six weeks

Two Italian engineers who each owned a 128GB Framework Desktop posted Gufo, their own local inference engine, to r/StrixHalo on 2026 Sep 24 and got 123 stars that day. It supports exactly one chip (AMD Ryzen AI Max+ 395, aka Strix Halo) and only a handful of models. By Nov 6 it had 1,096 stars and 188 forks. This piece looks at why a project like that can earn stars on llama.cpp's home turf, what part of that playbook you can copy, and where it fits.

A number that goes from 41 minutes to 3.5 minutes

Oct 2: The author of LLMKube, a Kubernetes inference platform, wrote a blog post that opens like this: he fed his Strix Halo a 250k token prompt, and on llama.cpp, which he'd been running for weeks, the first token took about 41 minutes to show up. Switching to Gufo, a new engine he'd never used, cut it to three and a half minutes.

"My first reaction wasn't excitement. It was: are we sure? Numbers like this usually mean the benchmark is broken."

He then ran all four llama.cpp tuning configs, took the best result at each depth, and re-tested with the same quality checks: a 250k-token cold prefill, Gufo at 1,184 tokens per second, and llama.cpp's best at only 101, a 11.7x gap. On the LiveCodeBench coding test, it scored 83 correct at 82 and 83 points, with no quality loss. He also honestly noted Gufo's weakness: single-stream decoding is 7% slower (33.5 vs 36.2 tok/s). In the end he switched his own coding agent over to Gufo.

This post is how Gufo spread: not through the author's own hype, but through a stranger spending days retesting it and publishing the results.

What it is

A local inference engine with hand-written C++ / HIP kernels, MIT-licensed, optimized only for Strix Halo (gfx1151 iGPU, 128GB unified memory). One engine runs Qwen3.8 27B, Qwen3.8 Flash-Next, DeepSeek V4 Flash, speech recognition and synthesis, and image generation, exposed through an OpenAI-compatible API. The README has a section on how to hook it up to Codex and Claude Code.

In the launch post on the Framework community forum, author Federico Izzo said: "We both have a 128GB Framework Desktop and got tired of juggling a pile of llama.cpp forks, tweaking this model here, that model there. We wanted something vertically integrated for the machine we actually own."

A few rules were spelled out in the post: support only the best model of each size class on this machine; write a separate kernel for each model rather than reusing code, to "limit the blast radius of code changes"; attach a quality report to every model; and treat concurrency, request cancellation, and conversation caching as first-class features.

Two authors with roughly 30 followers each on GitHub, no org behind it, and no existing brand to lean on.

Why late September took off

Three things stacked together:

• A model that just fits. Aug 26 Qwen3.8-Flash-Next goes open source: a 125B MoE with only about 6B active per token. At 4-bit quantization it fits in 128GB of unified memory. People who bought a Framework Desktop or one of the AI Max+ 395 mini PCs can run a near-frontier model on their desk for the first time.
• Existing tools aren't fast enough. llama.cpp runs it, but long-context prefill is too slow for coding agents (41 minutes on the one above). Halogen, the fast option that showed up the same day, is a closed-source container. A post on r/StrixHalo with 48 upvotes was titled "Gufo looks like a good open-source option besides the closed-source Halogen" — that's the spot users are looking for.
• The author ships fast. 57 days, 440 PRs merged, 10-30 to 11-05 shipped 5 releases in a week, up to v0.31.0. The cache bug LLMKube hit during re-testing was reported by someone else, reproduced by the author, fixed and released before the blog post was even finished.

The star curve is very typical: the repo was created in Aug 11, and until Sep 23 it was 0 (written in private for six weeks); 9-24 +123, and it made #5 on the Trendshift C++ daily chart; 9-25 +69,9-26 +54; since then it has steadily added 10 to 35 a day. In the last 7 days +100, and in the 7 days before that +135, so it's slowly cooling off.

Where the stars came from

No Hacker News, no tech-press feature, no big-account reposts. The whole spread stayed within the Strix Halo player community (estimated at a few thousand people).

One thing worth noting: the GitHub bio says "73.17 tg", which is speculative decoding's ceiling of about 100% acceptance on repetitive text. In real coding use, the user's own estimate is 30 more tok/s. The number grabbed attention and drew skepticism too. The author's response was to explain it openly and keep publishing quality reports, and the skeptical threads ended up driving another wave of traffic.

Who it's competing with

In the Strix Halo pool, Gufo is already the open-source #1, with plenty of similar small projects. Its community forks are growing too: a Windows port (39 stars), a two-machine RDMA interconnect version (22 stars), and a version that offloads prefill to the NPU.

The one to watch is Strata: same model, cross-hardware, 41,000 stars, 40× Gufo. "Only one chip" bought it extreme speed, but it also caps the ceiling at that chip's user count.

What you can learn

1. Stake out the spot where new hardware meets a new hit model. Don't build a better llama.cpp; build "this machine plus these few models, tuned to the limit." The two to six weeks after a hit model goes open-source is when users are most anxious and existing tools lag the most.
2. Picking a closed-source rival already proves where the demand is. Halogen proved that "10x faster on Strix Halo" sells, and users want an open-source version.
3. Launch in the hardware's vertical communities, and make the numbers reproducible for strangers. Subforums + the vendor forum + one benchmark table + a ready-made container; let retest blogs handle the rest of the spread.
4. Make the quality report your selling point. This crowd is extremely wary of inflated benchmarks. "Every model ships with a quality report" and "no fast kernel that slows things down overall" are trust signals in themselves.

If it were you

B. The playbook is clear and copyable, but the Strix Halo slot is already taken, and the bar is handwritten GPU kernels. Few people can do it, so don't try if you can't.

Who it's for: people who have written CUDA / HIP / Metal kernels, already own a new piece of hardware, and run models locally every day. Two people is ideal: one writes kernels, one handles releases and community.

Who it's not for: people who just wrap an API with a UI; anyone who wants to build another engine of the same kind on Strix Halo (Gufo ships 7 PRs a day, you won't keep up); anyone hoping to make money from it.

How to build it: swap the hardware, not the approach. Good candidates are platforms that are shipping now, have large unified memory, and lack well-tuned tooling: NVIDIA DGX Spark-class GB10 mini PCs, 128GB Macs, next-gen AI PC chips. (Inference: pick whichever platform has the most active player community and where llama.cpp runs slowest.) Or take the reverse of Strata: a newly open-sourced hit model, made fastest across hardware. The audience is much bigger, but you're racing more people. Either way, support only two or three models, ship a reproducible quality and speed report for each, and offer an OpenAI-compatible endpoint plus setup for Codex / Claude Code from day one, since most people buying this kind of machine want to run coding agents locally.

How to get users: on launch day, post simultaneously to the subreddit for this hardware, a general r/LocalLLaMA-style forum, and the vendor's official forum, with the benchmark table and a one-line docker command as the body. Stay in the comments to answer questions. Message bloggers who have already reviewed this hardware and ask them to re-test. If someone questions the numbers, reply that day and add clarifications.

Deadline: within 4 weeks of the next hit open-source model that fits on this kind of machine. Gufo launched on day 29 after Qwen3.8-Flash-Next was open-sourced, and Halogen launched the same day. Launching more than 6 weeks later, the slot is likely already taken (inferred).

Signs you've misread the market: ① llama.cpp mainline closes the gap on Strix Halo's long-context prefill within a month or two, and Gufo's daily star gains drop to single digits, meaning the narrow engine was just a gust of wind while the tooling caught up; ② the hardware vendor's official stack (AMD's Lemonade and the like) reaches the same speed directly; ③ within two weeks of launch on the machine you picked, there is not a single re-test blog post. The crowd is too small to sustain a thousand stars.

---

Data and sources: stars, forks, and releases come from GitHub (11-06); daily chart rankings and PR counts come from Trendshift; the original launch post comes from the Framework community (9-25); re-test figures come from LLMKube (10-02), aifinitee (10-18), elumenotion (11-02), and Hack-Log (10-01); Reddit post upvotes and comment counts come from search snippets, as the original posts could not be opened directly; X post data comes from public posts by @TeksEdge, @LFrefman, and @julianharris. Anything marked "estimate" or "inferred" is exactly that.