Bonsai
No written verdict yet. Here is what commenters said.
Why people recommend it
“with the Bonsai 2 quantisations it is quite achievable”
“Or try these with your trained embedder”
Show 3 more
“keep in mind we are talking about Bonsai 2 and not the original Bonsai.”
“You might be interested in https://prismml.com/news/bonsai-2-27b”
“For anyone who wants to try Bonsai 2 without setting up runtimes”
Caveats and dissent
“I had significant issues with the previous Bonsai 27”
“1-bit bonsai 27B is quite "dumb" when asked a question”
Show 2 more
“these models feel significantly worse than the base model when you start using them for real tasks”
Often compared with
Who recommends it in their own words
From the makers not counted in the ranking
“an open-source conversational framework for brand-safe assistants and agents”
“If you can share details on where it is failing, we'd love to help fix.”
Threads where it came up
- Show HN: 1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs 11 mentions
- Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint 4 mentions
- Running Gemma 4 26B at 5 tokens/SEC on a 13-year-old Xeon with no GPU 4 mentions
- Bonsai 27B (1-bit LLM): The First 27B-Class Model to Run on a Phone 4 mentions
- Best LLM for every budget, updated daily 2 mentions
- Memdoor: A team of 5 Qwen3.8-27B models beats Fable 5 on LiveCodeBench Hard 1 mention
Also written as Bonsai 27B, Bonsai, Bonsai 2, Bonsai 2 27B. Data: bonsai.json. Quotes are verbatim; the verdict and theme labels are written by a model from these comments, and every clause links to its sources.
All comments 31
“with the Bonsai 2 quantisations it is quite achievable”
“Or try these with your trained embedder”
“keep in mind we are talking about Bonsai 2 and not the original Bonsai.”
“You might be interested in https://prismml.com/news/bonsai-2-27b”
“but I was hoping for a better token generation rate”
“For anyone who wants to try Bonsai 2 without setting up runtimes”
“this new Bonsai is nice because the original was already exciting for making use of low VRAM devices”
“Pretty decent for a 5.5GB model file.”
“I had significant issues with the previous Bonsai 27”
“I use Bonsai 27B ternary on my 24GB MBP.”
“1-bit bonsai 27B is quite "dumb" when asked a question”
“it's just dumb as hell”
“i found this space to work better/at all”
“these models feel significantly worse than the base model when you start using them for real tasks”
“bonsai retains roughly 95% of the fp16 27b model's average performance”
There is also the 1bit version @ 3.9 GB that retains 90% of the intelligence - quite a feat! https://huggingface.co/prism-ml/Bonsai-27B-mlx-1bit
“runs at 44+ t/sec on an m4 max laptop”
“Bonsai vs Qwen (quick) Benchmark”
“Looks like they quant'd too hard at 4 bits, can't imagine the ternary being any good based on this.”
“I spent quite sometime trying to install their tools and nothing really worked.”
“All failed.”
“its an extremely fun model to play with!”
“I ran my custom agentic SQL debugging benchmark against it and I'm impressed.”
“I'm very skeptical of the advantage they're claiming here”
“1 bit with a FP16 scale factor every 128 bits. Fascinating that this works so well.”
“A bit of of benchmarks I did:”
To the author: why is this taking 4.56GB ? I was expecting this to be under 1GB for 4B model. https://ibb.co/CprTGZ1c And this is when Im serving zero prompts.. just loaded the model (using llama-server).
“the quality of outputs is on par with top models of similar size”
“Literally just downloaded the model into a folder”
“See page 10:”
“so it is a tradeoff that is pretty respectable”