One 5090, one Windows game console, and a benchmark table: how Infernix got 274 stars amid the inference engine brawl

On Oct 10, a Reddit user named zipzak spent a full day running the same model through both Strata and Infernix on his RTX 5090 Windows machine with 192GB of RAM. He wrote on r/LocalLLaMA: "I was blown away by Strata, then saw someone posting Infernix with bold claims, and it seems few people know about it," so he tested it himself. His verdict in one line: "Infernix NVFP4 is the daily driver," with Strata as the backup. Over those days Infernix's stars went from 54 to 98. It is a dedicated engine written by a hobbyist with AI help, serving one model on one GPU family. Below: what it is, where the stars came from, who it's competing with, and how much room is left for a project like this to get stars.

What it is: an engine written for one card and one model

Infernix is a C++/CUDA inference engine that does one thing: run Qwen3.8, and especially Qwen3.8-Flash-Next, on a single RTX 5090. It is an MoE model with 125B total parameters and 6B active per token. The expert weights don't fit in 32GB of VRAM, so they are offloaded to RAM and fetched on demand. In Oct 8 it was forked from another open-source engine, NInfer, adding Flash-Next expert offloading, prefix caching, and native Windows support.

The first paragraph of the README is an "AI disclaimer." The code added in the fork, along with most of the README, was written mainly by Claude Opus 5.5 and by Qwen3.8-27B running on Infernix itself. It "is probably neither complete nor fully accurate; this is hobby development." The author's GitHub account was registered 2019 years ago, has only this one public repo, and has 2 followers. At the bottom of the repo is a GitHub Sponsors link, with the note "help me pay for my Claude Code subscription."

As of 11-06:274 stars and 37 forks, with roughly 1,939 commits, and several more that same day.

Why now: one model spawned a whole category

The starting point wasn't Infernix. It was Qwen3.8-Flash-Next and Strata. That model made "a consumer GPU plus a big stick of RAM running a 100B-class model" possible for the first time. On Oct 4, a Hacker News post, "Running Qwen 3.8 Flash Next (125B) on an RTX 4090 at 100 tok/s," got 940 points and 420 comments. The link was to Strata. By 11-06, Strata had over 40k stars and was still adding 4,500 or more over the last 7 days.

After that, all of LocalLLaMA kept asking the same question: which engine is fastest on my 5090? A blog post from Sep 30 gave this kind of thing a name, "overfit inference engines": tools tuned for one or two models and one class of hardware, which can beat general tools like llama.cpp inside that small circle. With AI writing the code, building a dedicated engine went from months to weeks. In one month six or seven appeared: Strata forks, Basalt, PolyStrata, capy-flash, Maya, Colibri, and more. By Nov 4 someone on r/LocalLLaMA posted "Keeping up with new inference engines," meaning there were too many new engines to keep track of.

Infernix didn't invent the category. It found a corner nobody had claimed, within the demand Strata had created.

How the stars came in: each wave matched a Reddit post

(Star counts were captured daily. Which specific post drove the 11-03 wave can't be attributed star by star. The link is inferred from timing.)

Three things stand out. First, the most valuable piece wasn't the author's own post. It was third-party benchmarks. The zipzak thread got translated and reposted across German, Japanese and Chinese AI news aggregators, and that drove nearly all of the early growth. Second, rivals ended up vouching for it. The Basalt author put it in the top tier of their comparison, right below the 11-01 launch post. The Basalt maintainer questioned the claim that first-token speed is 5~8x faster, since it didn't match their own tests, but still publicly praised the launcher and the bit-exact mode. Third, the precision question got raised by the community itself on 11-02, and that's exactly what Infernix has been talking about since day one.

Two other entry points are less visible but substantial:

• Weights on HuggingFace. Starting in mid-September, the author published his own converted Qwen3.8 quantized weights on HF, 15 in total, including uncensored variants. The most downloaded, the 27B-Uncensored, had 9,758 downloads in nearly 30 days, far more than the GitHub stars. Every model card links back to Infernix.
• No-install Windows package. No CUDA Toolkit needed: download, unzip, run. Downloads across three Release versions total about 1,400. Basalt is Linux-only. Strata also runs on Windows, but Infernix treats Windows users as first-class. The most surprised line in zipzak's post was "both of these run on Windows, and setup isn't painful."

No ads, creators, or media coverage seen.

Why it got noticed among so many engines

Riding someone else's heat. The title says "faster than Strata," and every table in the README uses Strata and Basalt as the comparison baseline. Users searching for Strata naturally run into it when comparing. The README even thanks Strata's author at the end, saying he "set the benchmark for running Flash-Next on a single card."

Benchmarks written like a paper, with self-admitted losses. The first screen of the README is a table of first-token, decode, and agentic replay numbers. Test conditions are spelled out down to PCIe Gen5 x8 and a unified 84GiB memory budget. The losing cells are shown too: on the IQ3_S quant, Basalt is faster overall (4.4 minutes vs 5.5 minutes), and its decode is 1.4~1.7x faster. On LocalLLaMA, a table that admits losses is the most believable, and the most likely to get re-run by others.

Taking a position nobody else wanted. Other engines trade accuracy for speed: 8-bit activations, skipping some experts, using a 2-bit output head for coarse filtering. Infernix does none of this by default. Activations keep the model's original precision, and it adds a bit-exact --lossless mode made for people who want to check quality. If you want speed, it offers switches too (--a8-activations, --expert-tail-skip). It also has prefix caching for agent coding, so the first token comes back faster in multi-turn sessions. In the agentic replay the author measured, total time was 56% lower than Strata's and 36% lower than Basalt's. It is also compatible with request formats from tools like Claude Code and Codex.

People are actually tinkering. Issues and PRs have reached #59. Many are users posting their own benchmarks, along with externally contributed serve mode, podman setup, and Linux benchmarks. 37 forks against 274 stars is a high ratio, which suggests more than casual starring.

Where it falls short

The scale is small. Infernix has 274 stars, Strata has over 40k, two orders of magnitude apart. Basalt, which started around the same time, has 690 stars and gained 326 in nearly 7 days, while Infernix gained 107. It is known only within the small r/LocalLLaMA circle. It hasn't made the HN front page and hasn't been covered by mainstream media.

The differentiation is narrow and easy to copy. Basalt's maintainer already responded on the accuracy angle, saying a Q8 dense-layer build could be made. The author's own benchmarks are disputed and haven't been independently verified. The biggest unknown is the model itself. The author said in the launch post that it "supports Qwen 4 as soon as it's out," but how much work a model tuned for Qwen3.8 would need for the next generation is not visible from outside.

Who else is in this lane

The "Flash-Next on 5090" niche is already crowded.

If it were you

The short verdict: Building another Flash-Next engine on Infernix's model isn't worth it. What's worth learning is the playbook: when a new model drops, go after the dedicated tool for "this model × this card" first, and write benchmarks against the most popular engine. Even so, it only earned a few hundred stars.

Who it's for: Developers who know CUDA or Metal or ROCm kernels, actually own the matching hardware, and are willing to live in r/LocalLLaMA. The Infernix author says AI-written code lets a hobbyist ship an engine in a few weeks that rivals ones that rivals rate as top tier. But only if you can read, test, and judge whether the AI-written kernels are correct.

Who it's not for: Anyone without a high-end GPU (5090 plus 96~192GB of RAM is the baseline for this crowd, and without the machine you can't run benchmarks, and benchmarks are the whole argument in this space); anyone who wants thousands of stars (this niche tops out in the hundreds, reaching thousands means being first, like Strata did); anyone who won't keep up with model updates.

How to do it:

• Change the hardware. Right now these engines all revolve around NVIDIA, and 5090 is the main battlefield. Apple M-series Macs with big memory, AMD GPUs, and older dual-GPU setups like 3090 are underserved groups (inferred; 3090 dual-GPU, 5080, and 12GB laptop cards already have people posting patches, which shows there's demand).
• Go after the next model. The biggest window is Qwen 4. Whoever first makes it fast on a common consumer card becomes the next Strata. Infernix's position was carved out of Strata's buzz; you either be first, or be ready to be the one who gets a few hundred stars.
• Fill the gap nobody has filled. The question the community has been asking at 11-02, "how much quality do these engines actually lose?", still has no independent answer. A cross-engine quality and speed benchmark, or a launcher that manages all the engines, wouldn't need to compete with them on the kernel level (inference).

How to get users: Follow the Infernix playbook: title your Reddit launch post with the hottest rival's name; put a reproducible comparison table on the first screen of the README, including the cells where you lose; actively invite people with the hardware to re-run your benchmarks. A long third-party post is worth more than ten you write yourself. Publish converted weights on HuggingFace and link the model card back to the repo. Ship a no-install package and a GUI launcher for Windows users.

When to get in: The window follows the model. The Qwen3.8-Flash-Next wave is already crowded, so the next window is the first two to three weeks after Qwen 4 releases (my inference, based on Strata hitting the HN front page within 10 days and six or seven rivals appearing within a month).

What would mean you read it wrong: ① Once Qwen 4 ships, llama.cpp or Ollama match the speed of the dedicated engines within two weeks, so the "overfit engine" category is a one-off; ② Strata and Basalt add support for the new model right at the version change, leaving no gap for latecomers, so the position gets locked by whoever came first; ③ The reverse: Infernix is first to run Qwen 4 and its stars jump into the thousands, which would show that in this space "no precision loss" alone can be a selling point, and the category deserves a second look.

---

Data and sources: GitHub repo page, README, Releases and issues (11-06); daily star counts captured every day (since 10-10); r/LocalLLaMA and r/LocalLLM posts by u/zipzak (10-10), u/jesdga95 (10-23/24), and the author u/Suspicious-Mark-5450 (11-01), with the original posts read via reposts on equalexperts AI Wiki, Lumeric, KBlip, zgaiagent and others; the Hacker News Strata thread (10-04); carteakey.dev, "The Rise of Overfit Inference Engines" (9-30); the HuggingFace wallawalla47 profile (downloads over the last 30 days); @TeksEdge's post on X (11-06). All benchmarks were self-run by the authors or rival authors and have not been independently verified; the engines' actual user counts are not public.