Unsloth
Open-source toolkit for fast LLM fine-tuning that also publishes quantized GGUF model weights.
Recommended mainly for its quantized GGUF models, which commenters say hold up well at lower bit widths1234, and for running and fine-tuning your own local LLMs.567 Complaints are scattered rather than shared: one user got far less context room than with regular quants8, and others flag hacky llama.cpp support for one model and weak vLLM support for GGUF.910
Why people recommend it
Quant quality · 3
Commenters say Unsloth's dynamic quants degrade less and are their default choice when available.
“almost always, I pick the unsloth quantization if there is one”
“if using dynamic quantizations like unsloth provides”
Show 1 more
“I recommend unsloth quants, i recommend 6bit for MoEs and 5bit for dense”
Fixes issues · 2
Users report that switching to Unsloth quants resolved problems they had with other quants.
“I had similar issue until I used Unsloth’s.”
“I’m using Unsloth’s Q8_K_XL quant”
Consumer hardware · 3
Users run Unsloth quants with long context on Macs and single consumer GPUs.
“I use Unsloth's Q3 XXS with 128k context.”
“I use unsloth's UD-Q5_K_XL at 256k context”
Show 1 more
“almost always, I pick the unsloth quantization if there is one”
Recommended settings · 2
Unsloth's recommended sampler and temperature settings are cited as worth following.
“make sure you're following Unsloth's recommendations for temperature/etc.”
“This is Unsloth's UD-Q4_K_S quantization”
Worth funding · 3
Some see Unsloth as valuable open-source work deserving support.
“Unsloth (https://github.com/sponsors/unslothai)”
“I love the work unsloth is doing.”
Show 1 more
“Unsloth is the real thing. Highly recommended for those running their own AI engines”
Fine-tuning · 1
One commenter plans to fine-tune models through Unsloth for vertical SaaS sub-agents.
“Run through unsloth, and with some fine tuning”
Caveats and dissent
Context room · 1
One user found Unsloth models left much less room for context than regular quants.
“For some reason the unsloth models leave hardly any room for context.”
llama.cpp hacks · 1
One commenter says Unsloth's llama.cpp version of a model relied on non-optimal hacks.
“it hacks a bunch of things to make it work and isn't optimal”
vLLM support · 1
GGUF quants have limited vLLM support, making working quants hard to find.
“I love the work unsloth is doing.”
Low-bit speed · 1
A 2-bit quant on a 32GB MacBook ran too slowly to be usable.
“I use a 2-bit quant from Unsloth on my MBP M5 32GB of RAM.”
Community moderation · 1
One researcher reported a hostile, message-deleting exchange in Unsloth's community channel.
“They remained accusatory, made it personal, and then started deleting messages.”
Often compared with
Who recommends it in their own words
Threads where it came up
- Qwen 3.6 27B is the sweet spot for local development 2 mentions
- Omarchy Is a Power Grab 1 mention
- Qwen 3.8 follows GPT-5.5 Pro reasoning prefills 1 mention
- Benchmarking Qwen3.8 27B quantizations: 4-bit holds up, 1-bit collapses 1 mention
- Apple caught off guard by AI demand for Mac Mini and Mac Studio 1 mention
- GLM-5.3-Flash 1 mention
Data: unsloth.json. Quotes are verbatim; the verdict and theme labels are written by a model from these comments, and every clause links to its sources.
All comments 20
“Unsloth (https://github.com/sponsors/unslothai)”
“This is Unsloth's UD-Q4_K_S quantization”
“I use a 2-bit quant from Unsloth on my MBP M5 32GB of RAM.”
“I use Unsloth's Q3 XXS with 128k context.”
“I use the Unsloth UD_Q2_K_XL GGUF with default parameters”
“I used unsloth's UD-Q{3-6}_K_XL quants across 11 evals.”
“For some reason the unsloth models leave hardly any room for context.”
“almost always, I pick the unsloth quantization if there is one”
“if using dynamic quantizations like unsloth provides”
“I’m using Unsloth’s Q8_K_XL quant”
“I use unsloth's UD-Q5_K_XL at 256k context”
“it hacks a bunch of things to make it work and isn't optimal”
“They remained accusatory, made it personal, and then started deleting messages.”
“I recommend unsloth quants, i recommend 6bit for MoEs and 5bit for dense”
“make sure you're following Unsloth's recommendations for temperature/etc.”
“Run through unsloth, and with some fine tuning”
“I had similar issue until I used Unsloth’s.”
“Unsloth is the real thing. Highly recommended for those running their own AI engines”
“I love the work unsloth is doing.”
“the unsloth quants are great”