ninfer
Fewer than 5 people recommend it so far, so there's no written verdict. Here is what commenters said.
3recommend
0warn against
Aug 14, 2026first mention
Sep 3, 2026latest mention
Why people recommend it
“on RTX 5090 I can now get ~200 tok/s and over 400 tok/s on concurrent requests which is plenty fast for a local model”
“Strongly recommend https://github.com/Neroued/ninfer, which can pull ~180 TPS on 5090 with 3.8”
runs on an RTX 5090on “Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)”
Show 1 more
“as inference engine gets me ~138 tokens/second, roughly double what I get with a naive llama.cpp setup.”
Caveats and dissent
No one in these comments warned against it.
Often compared with
Who recommends it in their own words
Threads where it came up
Data: ninfer.json. Quotes are verbatim; the verdict and theme labels are written by a model from these comments, and every clause links to its sources.
All comments 3
“on RTX 5090 I can now get ~200 tok/s and over 400 tok/s on concurrent requests which is plenty fast for a local model”
“Strongly recommend https://github.com/Neroued/ninfer, which can pull ~180 TPS on 5090 with 3.8”
“as inference engine gets me ~138 tokens/second, roughly double what I get with a naive llama.cpp setup.”