{"slug": "qwen-3-5", "name": "Qwen 3.5", "category": "ai", "category_name": "AI models and assistants", "description": "Alibaba's family of open-weight large language models, from 0.8B up to 397B parameters.", "recommenders": 28, "commenters": 53, "warners": 3, "score": 27.738, "rising": -5.18, "first": "2026-02-20", "latest": "2026-10-11", "why": "Liked for running capably on consumer hardware, with small variants handling narrow tasks well and larger ones doing workable local coding.", "why_is_quote": false, "has_verdict": true, "sentiment": {"recommend": 29, "neutral": 21, "compare": 3, "warn": 3}, "spark": [0, 0, 0, 1, 3, 7, 5, 8, 2, 2, 0, 1], "listed": true, "loved": ["Consumer hardware", "Small-task models"], "caveat": "Looping", "short_quote": "I have settled on the qwen3.5 family after testing most of the local usable models.", "icon": true, "domain": "unsloth.ai", "github_repo": null, "official_url": "https://qwen.ai", "pricing": "open source", "aliases": ["Qwen 3.5", "Qwen3.5", "Qwen 3.5 9B", "Qwen3.5 9B", "Qwen3.5-122B-A10B", "Qwen3.5-35B-A3B", "Qwen3.5-4B", "Qwen3.5-122B", "Qwen 3.5 0.8B", "Qwen3.5-27B", "Qwen 3.5-9b", "Qwen 3.5 4B", "Qwen3.5 35B", "Qwen3.5-35B", "Qwen3.5-9B", "Qwen 3.5 122B-A10B", "Qwen3.5 122B A10B", "Qwen3.5 27B"], "verdict": [{"text": "Liked for running capably on consumer hardware", "cites": [{"id": 764780592, "item_id": 47798787, "url": "https://news.ycombinator.com/item?id=47798787", "by": "kanemcgrath", "date": "2026-04-16", "stance": "recommend", "quote": "it is by far the most capable model that could fit on my machine", "excerpt": "I have been using Qwen3.5-35B-A3B a lot in local testing, and it is by far the most capable model that could fit on my machine. I think quantization technology has really upped its game around these models, and there were two quants that blew me away Mudler APEX-I-Quality. then later I tried Byteshape Q3_K_S-3.40bpw Both made claims that seemed too good to be true, but I couldn't find any traces of lobotomization doi", "context": "RTX 3060 12GB, 32GB RAM", "use_case": "local agentic coding", "switched_from": "", "story": "Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All"}, {"id": 756510704, "item_id": 47281919, "url": "https://news.ycombinator.com/item?id=47281919", "by": "willfinger", "date": "2026-03-06", "stance": "recommend", "quote": "I've cracked the optimal configuration for Qwen3.5-35B-A3B on consumer 16GB GPUs.", "excerpt": "After weeks of systematic benchmarking, I've cracked the optimal configuration for Qwen3.5-35B-A3B on consumer 16GB GPUs. The headline: *120 t/s generation, ~500 t/s prompt ingestion, 120K context, vision enabled \u2014 all on a single 16GB card.* --- ## The Vision Breakthrough Here's what makes this special: most local LLM setups sacrifice speed when you enable multimodal. Not this one. You get: - Image analysis - PDF re", "context": "", "use_case": "local coding and vision on 16GB GPU", "switched_from": "", "story": "Qwen3.5-35B \u2013 16GB GPU \u2013 100T/s with 120K context AND vision enabled"}, {"id": 774824256, "item_id": 48426516, "url": "https://news.ycombinator.com/item?id=48426516", "by": "Pxtl", "date": "2026-06-06", "stance": "recommend", "quote": "it's dumb as a stump but still smart enough to get some useful work done", "excerpt": "I've got a Qwen 3.5 running on a 12GB 3060 and it's dumb as a stump but still smart enough to get some useful work done. Since it's my daily driver desktop I havent jumped to 3.6 since last time I did I quickly ran out of vram and locked the desktop environment. But yeah, the Qwen line is pretty impressive on commodity hardware.", "context": "runs it on a 12GB 3060 desktop", "use_case": "local LLM on 12GB GPU", "switched_from": "", "story": "Nvidia is proposing a beast of a CPU system for Windows PCs"}, {"id": 755067296, "item_id": 47191706, "url": "https://news.ycombinator.com/item?id=47191706", "by": "segmondy", "date": "2026-02-27", "stance": "recommend", "quote": "checkout Qwen3.5 the 27B and 35B variants", "excerpt": "This is a good time to promote running your own models. I have been running my own models locally and I would wager a local model will meet 85-95% of your needs if you really learn to use it. These models have gotten great. For anyone wanting to get into this, the smartest models to run recently that is consumer friendly was just released, checkout Qwen3.5 the 27B and 35B variants. They are small and I recommend runn", "context": "runs models locally", "use_case": "running local models", "switched_from": "", "story": "How do I cancel my ChatGPT subscription?"}]}, {"text": ", with small variants handling narrow tasks well and larger ones doing workable local coding.", "cites": [{"id": 798830560, "item_id": 49926910, "url": "https://news.ycombinator.com/item?id=49926910", "by": "whiteblossom", "date": "2026-10-01", "stance": "recommend", "quote": "qwen3.5 4b and 9b have both been surprisingly good for small tasks.", "excerpt": "qwen3.5 4b and 9b have both been surprisingly good for small tasks. a pattern I've been using is orchestrating pi agents with a script (that you can write using a frontier model) for common tasks. one script i use a lot is organizing photos from my photography shoots.", "context": "", "use_case": "small tasks with Pi agents", "switched_from": "", "story": "Pi 1.0"}, {"id": 781322547, "item_id": 48832659, "url": "https://news.ycombinator.com/item?id=48832659", "by": "brainless", "date": "2026-07-08", "stance": "recommend", "quote": "I have had really good results from Qwen 3.5 1B, 4B, etc.", "excerpt": "I have a project on very similar lines, https://github.com/brainless/dwata, which I have not been developing for the past few months. I have been meaning to get back to it and I really like what I see on your project page. My aim is to build a truly local app using only tiny/small models. I have had really good results from Qwen 3.5 1B, 4B, etc. Also, Gliner or similar models for different uses. SQLite + sqlite-vec +", "context": "", "use_case": "small local models", "switched_from": "", "story": "Show HN: Rowboat \u2013 Open-source, local-first alternative to Claude Desktop"}, {"id": 766822001, "item_id": 47926375, "url": "https://news.ycombinator.com/item?id=47926375", "by": "jonaustin", "date": "2026-04-27", "stance": "recommend", "quote": "I've been using qwen3.5 (122b) with claude code for months, and it's definitely dumber than sonnet/opus, but it works through things reasonably well", "excerpt": "Try https://github.com/jundot/omlx I've been using qwen3.5 (122b) with claude code for months, and it's definitely dumber than sonnet/opus, but it works through things reasonably well (i.e. writes half-decent code and tool calls usually work), and I pretty much never run into loops now. and make sure you're following Unsloth's recommendations for temperature/etc.", "context": "Runs local models with Claude Code for months", "use_case": "local coding model", "switched_from": "", "story": "Running Local LLMs Offline on a Ten-Hour Flight"}, {"id": 769439040, "item_id": 48089940, "url": "https://news.ycombinator.com/item?id=48089940", "by": "quacker", "date": "2026-05-10", "stance": "recommend", "quote": "Qwen 3.5 9B with Opencode was much faster and actually able to work through a majority of the lint warnings without getting stuck", "excerpt": "I could have used this article before I spent the weekend arriving to the same conclusion! Same laptop, and my contrived test was having it fix 50 or so lint errors in a small vibe-coded C++ repo. I wanted it to be able to handle a bunch of small tasks without getting stuck too often. GPT OSS 20B was usable but slow, and actually frequently made mistakes like adding or duplicating statements unnecessarily, listing th", "context": "M4 laptop with 24GB", "use_case": "local lint fixing", "switched_from": "GPT OSS 20B", "story": "Running local models on an M4 with 24GB memory"}]}, {"text": " The most common complaint is looping and overthinking unless sampling settings and quants are right", "cites": [{"id": 799299489, "item_id": 49956218, "url": "https://news.ycombinator.com/item?id=49956218", "by": "nottorp", "date": "2026-10-04", "stance": "warn", "quote": "Qwen 3.5 Q4 went into some kind of loop where it fixed whatever was broken on the previous iteration only to have it broken some other way.", "excerpt": "At Q what? That's what I'm mostly asking about. My little test was \"generate me a single page tic tac toe game in plain javascript. computer always play O. add unbeatable minmax. have the board, a status line and a new game button'. I used both lm studio and whatever the name of their new coding assistant that supercedes lm studio is. Qwen 3.5 Q4 went into some kind of loop where it fixed whatever was broken on the p", "context": "", "use_case": "local code generation test", "switched_from": "", "story": "Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s"}, {"id": 758065440, "item_id": 47379090, "url": "https://news.ycombinator.com/item?id=47379090", "by": "CamperBob2", "date": "2026-03-14", "stance": "neutral", "quote": "the models are still quite capable when configured that way.", "excerpt": "Qwen 3.5 was plagued by some premature quant releases and unclear/incomplete guidelines for the sampling parameters. Especially if you are having looping problems, make sure you are using the very latest model files, executables, and recommended params. If all the stars are aligned, Qwen 3.5 will not exhibit outright looping, although it will still burn more thinking tokens than some other models. There are ways to t", "context": "", "use_case": "local LLM", "switched_from": "", "story": "Ask HN: How do you use local LLMs productively?"}, {"id": 786488849, "item_id": 49155553, "url": "https://news.ycombinator.com/item?id=49155553", "by": "hiramwen", "date": "2026-08-03", "stance": "recommend", "quote": "I love how literal Qwen 3.5 was", "excerpt": "I've been waiting for a tiny model release! Kimi, Deepseek, and GLM were all monsters, the 27B 3.8 is filling a gap thats been open since Qwen 3.5 Also I love how literal Qwen 3.5 was, hopefully 3.8 is still extremely literal. The token explosions were actually helpful in debugging prompts.", "context": "", "use_case": "debugging prompts", "switched_from": "", "story": "Qwen3.8-Max: A New Bar for Coding and Cowork"}]}, {"text": ", and commenters say it still trails frontier models like Opus.", "cites": [{"id": 766822001, "item_id": 47926375, "url": "https://news.ycombinator.com/item?id=47926375", "by": "jonaustin", "date": "2026-04-27", "stance": "recommend", "quote": "I've been using qwen3.5 (122b) with claude code for months, and it's definitely dumber than sonnet/opus, but it works through things reasonably well", "excerpt": "Try https://github.com/jundot/omlx I've been using qwen3.5 (122b) with claude code for months, and it's definitely dumber than sonnet/opus, but it works through things reasonably well (i.e. writes half-decent code and tool calls usually work), and I pretty much never run into loops now. and make sure you're following Unsloth's recommendations for temperature/etc.", "context": "Runs local models with Claude Code for months", "use_case": "local coding model", "switched_from": "", "story": "Running Local LLMs Offline on a Ten-Hour Flight"}, {"id": 774824256, "item_id": 48426516, "url": "https://news.ycombinator.com/item?id=48426516", "by": "Pxtl", "date": "2026-06-06", "stance": "recommend", "quote": "it's dumb as a stump but still smart enough to get some useful work done", "excerpt": "I've got a Qwen 3.5 running on a 12GB 3060 and it's dumb as a stump but still smart enough to get some useful work done. Since it's my daily driver desktop I havent jumped to 3.6 since last time I did I quickly ran out of vram and locked the desktop environment. But yeah, the Qwen line is pretty impressive on commodity hardware.", "context": "runs it on a 12GB 3060 desktop", "use_case": "local LLM on 12GB GPU", "switched_from": "", "story": "Nvidia is proposing a beast of a CPU system for Windows PCs"}, {"id": 799299489, "item_id": 49956218, "url": "https://news.ycombinator.com/item?id=49956218", "by": "nottorp", "date": "2026-10-04", "stance": "warn", "quote": "Qwen 3.5 Q4 went into some kind of loop where it fixed whatever was broken on the previous iteration only to have it broken some other way.", "excerpt": "At Q what? That's what I'm mostly asking about. My little test was \"generate me a single page tic tac toe game in plain javascript. computer always play O. add unbeatable minmax. have the board, a status line and a new game button'. I used both lm studio and whatever the name of their new coding assistant that supercedes lm studio is. Qwen 3.5 Q4 went into some kind of loop where it fixed whatever was broken on the p", "context": "", "use_case": "local code generation test", "switched_from": "", "story": "Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s"}]}], "praise": [{"label": "Consumer hardware", "summary": "Commenters report usable speeds on 12\u201316GB GPUs, Macs, and even single-board computers.", "quotes": [{"id": 764780592, "item_id": 47798787, "url": "https://news.ycombinator.com/item?id=47798787", "by": "kanemcgrath", "date": "2026-04-16", "stance": "recommend", "quote": "it is by far the most capable model that could fit on my machine", "excerpt": "I have been using Qwen3.5-35B-A3B a lot in local testing, and it is by far the most capable model that could fit on my machine. I think quantization technology has really upped its game around these models, and there were two quants that blew me away Mudler APEX-I-Quality. then later I tried Byteshape Q3_K_S-3.40bpw Both made claims that seemed too good to be true, but I couldn't find any traces of lobotomization doi", "context": "RTX 3060 12GB, 32GB RAM", "use_case": "local agentic coding", "switched_from": "", "story": "Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All"}, {"id": 756510704, "item_id": 47281919, "url": "https://news.ycombinator.com/item?id=47281919", "by": "willfinger", "date": "2026-03-06", "stance": "recommend", "quote": "I've cracked the optimal configuration for Qwen3.5-35B-A3B on consumer 16GB GPUs.", "excerpt": "After weeks of systematic benchmarking, I've cracked the optimal configuration for Qwen3.5-35B-A3B on consumer 16GB GPUs. The headline: *120 t/s generation, ~500 t/s prompt ingestion, 120K context, vision enabled \u2014 all on a single 16GB card.* --- ## The Vision Breakthrough Here's what makes this special: most local LLM setups sacrifice speed when you enable multimodal. Not this one. You get: - Image analysis - PDF re", "context": "", "use_case": "local coding and vision on 16GB GPU", "switched_from": "", "story": "Qwen3.5-35B \u2013 16GB GPU \u2013 100T/s with 120K context AND vision enabled"}, {"id": 756758768, "item_id": 47297423, "url": "https://news.ycombinator.com/item?id=47297423", "by": "benbojangles", "date": "2026-03-08", "stance": "recommend", "quote": "I'm running Qwen3.5:0.8b locally on an Orangepi Zero 2w using llama.cpp, runs just fine on cpu only.", "excerpt": "I'm running Qwen3.5:0.8b locally on an Orangepi Zero 2w using llama.cpp, runs just fine on cpu only. If I want vulkan GPU I have run qwen3.5:2b locally on a meta quest 3 with zeroclaw and saved myself hundreds of $$$ buying a low power computer. I recommend people stop shopping around for inflated mac minis and look at getting a used android phone to load local models on.", "context": "runs local models on cheap hardware", "use_case": "local models on low-power devices", "switched_from": "", "story": "How to run Qwen 3.5 locally"}]}, {"label": "Small-task models", "summary": "The tiny 0.8B\u20139B variants are described as surprisingly good at narrow, well-scoped jobs.", "quotes": [{"id": 798830560, "item_id": 49926910, "url": "https://news.ycombinator.com/item?id=49926910", "by": "whiteblossom", "date": "2026-10-01", "stance": "recommend", "quote": "qwen3.5 4b and 9b have both been surprisingly good for small tasks.", "excerpt": "qwen3.5 4b and 9b have both been surprisingly good for small tasks. a pattern I've been using is orchestrating pi agents with a script (that you can write using a frontier model) for common tasks. one script i use a lot is organizing photos from my photography shoots.", "context": "", "use_case": "small tasks with Pi agents", "switched_from": "", "story": "Pi 1.0"}, {"id": 781322547, "item_id": 48832659, "url": "https://news.ycombinator.com/item?id=48832659", "by": "brainless", "date": "2026-07-08", "stance": "recommend", "quote": "I have had really good results from Qwen 3.5 1B, 4B, etc.", "excerpt": "I have a project on very similar lines, https://github.com/brainless/dwata, which I have not been developing for the past few months. I have been meaning to get back to it and I really like what I see on your project page. My aim is to build a truly local app using only tiny/small models. I have had really good results from Qwen 3.5 1B, 4B, etc. Also, Gliner or similar models for different uses. SQLite + sqlite-vec +", "context": "", "use_case": "small local models", "switched_from": "", "story": "Show HN: Rowboat \u2013 Open-source, local-first alternative to Claude Desktop"}, {"id": 776349969, "item_id": 48521873, "url": "https://news.ycombinator.com/item?id=48521873", "by": "girvo", "date": "2026-06-13", "stance": "recommend", "quote": "It works surprisingly well really.", "excerpt": "I have a \"task router\" that is a small local LLM on my mac mini (Qwen 3.5 0.8B) that I use to decide (when activated) with Pi whether to route a given task to my local LLM (Step 3.7 Flash) or to <given cloud provider>, if that counts? It works surprisingly well really. Though some of the cloud providers are getting so good and so cheap (GLM 5.1/5.2, MiniMax M3, among others) that the need to use my local one becomes ", "context": "", "use_case": "local task router", "switched_from": "", "story": "RTX 5080 and RTX 3090 Setup: 80 Tok/s on Qwen 3.6 27B Q8"}]}, {"label": "Local coding", "summary": "Users run it with coding agents and harnesses and say it writes half-decent code and clears tasks without getting stuck.", "quotes": [{"id": 766822001, "item_id": 47926375, "url": "https://news.ycombinator.com/item?id=47926375", "by": "jonaustin", "date": "2026-04-27", "stance": "recommend", "quote": "I've been using qwen3.5 (122b) with claude code for months, and it's definitely dumber than sonnet/opus, but it works through things reasonably well", "excerpt": "Try https://github.com/jundot/omlx I've been using qwen3.5 (122b) with claude code for months, and it's definitely dumber than sonnet/opus, but it works through things reasonably well (i.e. writes half-decent code and tool calls usually work), and I pretty much never run into loops now. and make sure you're following Unsloth's recommendations for temperature/etc.", "context": "Runs local models with Claude Code for months", "use_case": "local coding model", "switched_from": "", "story": "Running Local LLMs Offline on a Ten-Hour Flight"}, {"id": 769439040, "item_id": 48089940, "url": "https://news.ycombinator.com/item?id=48089940", "by": "quacker", "date": "2026-05-10", "stance": "recommend", "quote": "Qwen 3.5 9B with Opencode was much faster and actually able to work through a majority of the lint warnings without getting stuck", "excerpt": "I could have used this article before I spent the weekend arriving to the same conclusion! Same laptop, and my contrived test was having it fix 50 or so lint errors in a small vibe-coded C++ repo. I wanted it to be able to handle a bunch of small tasks without getting stuck too often. GPT OSS 20B was usable but slow, and actually frequently made mistakes like adding or duplicating statements unnecessarily, listing th", "context": "M4 laptop with 24GB", "use_case": "local lint fixing", "switched_from": "GPT OSS 20B", "story": "Running local models on an M4 with 24GB memory"}, {"id": 763244322, "item_id": 47702770, "url": "https://news.ycombinator.com/item?id=47702770", "by": "dust42", "date": "2026-04-09", "stance": "neutral", "quote": "I used Qwen3.5-35B on Mac M1 Max with oMLX. It wrote 98% of the code", "excerpt": "I really love it. The simplicity is key. The first play project I made with it was a public transport map with GTFS data - click on a stop and get the routes and the timetables for the stop and the surrounding ones. I used Qwen3.5-35B on Mac M1 Max with oMLX. It wrote 98% of the code with very little interaction from me. And very useful is the /tree feature to go back in history when the model is on a wrong track or ", "context": "", "use_case": "local coding model", "switched_from": "", "story": "Reallocating $100/Month Claude Code Spend to Zed and OpenRouter"}]}, {"label": "Near-cloud quality", "summary": "Some found its output comparable to Claude Sonnet or Opus for specific tasks such as summaries and one-shot scenes.", "quotes": [{"id": 766028992, "item_id": 47876812, "url": "https://news.ycombinator.com/item?id=47876812", "by": "gpapilion", "date": "2026-04-23", "stance": "recommend", "quote": "The quality of the qwen output was comparable", "excerpt": "So recently I moved from a Anthropic model to a qwen 3.5 model running on my Mac to summarize ticket activity over 7 days. I used to do this manually with a colleague and it would take us a couple hours to go through. Opus took 58 seconds, and Qwen took 2.5 minutes. The quality of the qwen output was comparable, but the there was a 2.5x difference in time. All that said I actually don\u2019t think that matters much. I thi", "context": "", "use_case": "summarizing ticket activity locally", "switched_from": "Claude Opus", "story": "LLM pricing has never made sense"}, {"id": 762190000, "item_id": 47636875, "url": "https://news.ycombinator.com/item?id=47636875", "by": "survirtual", "date": "2026-04-04", "stance": "recommend", "quote": "I have settled on the qwen3.5 family after testing most of the local usable models.", "excerpt": "I have been running nearly the same experiments for the same reasons. It has been a lot of tweaking and patching because I am on a Strix Halo system with amd gfx1191, but the LLM component is working nicely. The evaluation I use is having it one shot a 3d scene using threejs. After everything, the output was comparable to Claude Sonnet (Opus actually does worse on this task strangely). For my local setup, I have sett", "context": "runs Strix Halo system with NixOS", "use_case": "local LLM inference", "switched_from": "", "story": "Vitalik Buterin \u2013 \"My self-sovereign / local / private / secure LLM setup\""}, {"id": 777348624, "item_id": 48584289, "url": "https://news.ycombinator.com/item?id=48584289", "by": "mystraline", "date": "2026-06-18", "stance": "recommend", "quote": "Its by far better than Opus.", "excerpt": "> We've all heard people say that local Qwen 27B or 35-A3B is \"near-Opus level\" Uh, so, yeah. Im running local Qwen, but Qwen3.5-122B using Krasis https://github.com/brontoguana/krasis Its by far better than Opus. In fact with a phone migration, I was using an OLD android 2fa app \"andOTP\". Backup files it emitted were JSON but not any sort of standard. I needed the standards version using otpauth:// to upload in my c", "context": "runs local models", "use_case": "local LLM for conversions", "switched_from": "", "story": "Local Qwen isn't a worse Opus, it's a different tool"}]}, {"label": "Data control", "summary": "Running it locally or on owned hardware keeps compute and data under the user's control.", "quotes": [{"id": 770348864, "item_id": 48146804, "url": "https://news.ycombinator.com/item?id=48146804", "by": "rsolva", "date": "2026-05-15", "stance": "recommend", "quote": "This works for us and will work for years to come.", "excerpt": "In our company of 24 employees, we get by with two DGX Sparks. We don't use AI heavily, but each Spark can serve about 6-8 concurrent requests with a full context lenght of 256k, which is decent. We get about ~35 t/s depending on the model we use (currently Qwen3.5 122B A10B and Qwen3 Coder Next), but we might set up a smaller model too for simpler tasks. This works for us and will work for years to come. It is not S", "context": "company of 24 employees using DGX Sparks", "use_case": "local team LLM serving", "switched_from": "", "story": "Cut Off: why access to frontier AI will soon be scarce and selective"}, {"id": 776922018, "item_id": 48557626, "url": "https://news.ycombinator.com/item?id=48557626", "by": "ngxson", "date": "2026-06-16", "stance": "recommend", "quote": "Speed is pretty acceptable for the size: ~5s TTFT and ~100 t/s generation, all running on a Macbook M5.", "excerpt": "My 2c: I think the \"cloud vs local\" debate is (maybe) a false dichotomy. In my experience, I use a hybrid approach and I've seen a huge productivity boost from it. The cloud-based models are fine for big and complex tasks, but the pricing is ridiculous for small stuff\u2014like summarizing a discussion or fixing a small bug. And cloud and privacy have never been a good match. As an example, this comment itself was written", "context": "", "use_case": "local summarizing and drafting", "switched_from": "", "story": "Running local models is good now"}]}, {"label": "OCR and vision", "summary": "The small models handle degraded text and image input better than classic OCR tools, according to commenters.", "quotes": [{"id": 778341217, "item_id": 48646326, "url": "https://news.ycombinator.com/item?id=48646326", "by": "philipkglass", "date": "2026-06-23", "stance": "recommend", "quote": "Modern VLMs put classic FineReader to shame for processing low-resolution/degraded/non-standard text.", "excerpt": "I used Abbyy Finereader for several years. I loved it. I completed some large projects with it. Modern VLMs put classic FineReader to shame for processing low-resolution/degraded/non-standard text. I'm personally using the small Qwen 3.5 models. If you have an OCR problem, Mistral OCR 4 is probably great. Open weights models that you can run on a laptop may also work great.", "context": "", "use_case": "OCR with small local models", "switched_from": "ABBYY FineReader", "story": "Mistral OCR 4"}, {"id": 756510704, "item_id": 47281919, "url": "https://news.ycombinator.com/item?id=47281919", "by": "willfinger", "date": "2026-03-06", "stance": "recommend", "quote": "I've cracked the optimal configuration for Qwen3.5-35B-A3B on consumer 16GB GPUs.", "excerpt": "After weeks of systematic benchmarking, I've cracked the optimal configuration for Qwen3.5-35B-A3B on consumer 16GB GPUs. The headline: *120 t/s generation, ~500 t/s prompt ingestion, 120K context, vision enabled \u2014 all on a single 16GB card.* --- ## The Vision Breakthrough Here's what makes this special: most local LLM setups sacrifice speed when you enable multimodal. Not this one. You get: - Image analysis - PDF re", "context": "", "use_case": "local coding and vision on 16GB GPU", "switched_from": "", "story": "Qwen3.5-35B \u2013 16GB GPU \u2013 100T/s with 120K context AND vision enabled"}]}], "caveats": [{"label": "Looping", "summary": "It can loop or burn many thinking tokens, especially with early quants or wrong sampling parameters.", "quotes": [{"id": 799299489, "item_id": 49956218, "url": "https://news.ycombinator.com/item?id=49956218", "by": "nottorp", "date": "2026-10-04", "stance": "warn", "quote": "Qwen 3.5 Q4 went into some kind of loop where it fixed whatever was broken on the previous iteration only to have it broken some other way.", "excerpt": "At Q what? That's what I'm mostly asking about. My little test was \"generate me a single page tic tac toe game in plain javascript. computer always play O. add unbeatable minmax. have the board, a status line and a new game button'. I used both lm studio and whatever the name of their new coding assistant that supercedes lm studio is. Qwen 3.5 Q4 went into some kind of loop where it fixed whatever was broken on the p", "context": "", "use_case": "local code generation test", "switched_from": "", "story": "Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s"}, {"id": 758065440, "item_id": 47379090, "url": "https://news.ycombinator.com/item?id=47379090", "by": "CamperBob2", "date": "2026-03-14", "stance": "neutral", "quote": "the models are still quite capable when configured that way.", "excerpt": "Qwen 3.5 was plagued by some premature quant releases and unclear/incomplete guidelines for the sampling parameters. Especially if you are having looping problems, make sure you are using the very latest model files, executables, and recommended params. If all the stars are aligned, Qwen 3.5 will not exhibit outright looping, although it will still burn more thinking tokens than some other models. There are ways to t", "context": "", "use_case": "local LLM", "switched_from": "", "story": "Ask HN: How do you use local LLMs productively?"}, {"id": 786488849, "item_id": 49155553, "url": "https://news.ycombinator.com/item?id=49155553", "by": "hiramwen", "date": "2026-08-03", "stance": "recommend", "quote": "I love how literal Qwen 3.5 was", "excerpt": "I've been waiting for a tiny model release! Kimi, Deepseek, and GLM were all monsters, the 27B 3.8 is filling a gap thats been open since Qwen 3.5 Also I love how literal Qwen 3.5 was, hopefully 3.8 is still extremely literal. The token explosions were actually helpful in debugging prompts.", "context": "", "use_case": "debugging prompts", "switched_from": "", "story": "Qwen3.8-Max: A New Bar for Coding and Cowork"}]}, {"label": "Below frontier", "summary": "Users call it dumber than Sonnet/Opus and say it needs harness tuning to get close.", "quotes": [{"id": 766822001, "item_id": 47926375, "url": "https://news.ycombinator.com/item?id=47926375", "by": "jonaustin", "date": "2026-04-27", "stance": "recommend", "quote": "I've been using qwen3.5 (122b) with claude code for months, and it's definitely dumber than sonnet/opus, but it works through things reasonably well", "excerpt": "Try https://github.com/jundot/omlx I've been using qwen3.5 (122b) with claude code for months, and it's definitely dumber than sonnet/opus, but it works through things reasonably well (i.e. writes half-decent code and tool calls usually work), and I pretty much never run into loops now. and make sure you're following Unsloth's recommendations for temperature/etc.", "context": "Runs local models with Claude Code for months", "use_case": "local coding model", "switched_from": "", "story": "Running Local LLMs Offline on a Ten-Hour Flight"}, {"id": 774824256, "item_id": 48426516, "url": "https://news.ycombinator.com/item?id=48426516", "by": "Pxtl", "date": "2026-06-06", "stance": "recommend", "quote": "it's dumb as a stump but still smart enough to get some useful work done", "excerpt": "I've got a Qwen 3.5 running on a 12GB 3060 and it's dumb as a stump but still smart enough to get some useful work done. Since it's my daily driver desktop I havent jumped to 3.6 since last time I did I quickly ran out of vram and locked the desktop environment. But yeah, the Qwen line is pretty impressive on commodity hardware.", "context": "runs it on a 12GB 3060 desktop", "use_case": "local LLM on 12GB GPU", "switched_from": "", "story": "Nvidia is proposing a beast of a CPU system for Windows PCs"}, {"id": 785027376, "item_id": 49064211, "url": "https://news.ycombinator.com/item?id=49064211", "by": "spottedmarley", "date": "2026-07-26", "stance": "recommend", "quote": "I use mostly Qwen3.5:122b-a10b but its the same family and the harness I've developed for it has made a huge difference", "excerpt": "Set up your own benchmark test. Run several models through meaningful workflows. I run local models every day and the harness is key for squeezing performance out of smaller models. I'd estimate that, depending on the work you're doing of course, you could get Qwen3.5:27B to within 80% code quality if you spent some some time fine tuning your harness specifically to that model and your workflow. It would be a lot of ", "context": "runs local models daily", "use_case": "local coding models", "switched_from": "", "story": "Qwen 27B with local well writen tools just as powerful as cluade models?"}]}, {"label": "Coding mistakes", "summary": "One user switched back to Gemma 4 after Qwen kept making mistakes that needed fixing.", "quotes": [{"id": 764020225, "item_id": 47751264, "url": "https://news.ycombinator.com/item?id=47751264", "by": "mapontosevenths", "date": "2026-04-13", "stance": "warn", "quote": "My orchestration model kept having to send the Qwen model back to fix mistakes that Gemma wouldn't have made.", "excerpt": "I'm just some guy on hackernews, but I actually did try this on my DGX Spark. I went back to Gemma 4 after a few rounds. My orchestration model kept having to send the Qwen model back to fix mistakes that Gemma wouldn't have made. I wound up with less working code per hour due to the mistakes. Technically, I use OpenWebUI with Ollama, so I used the weights below, but it should be the same. https://ollama.com/kwangsuk", "context": "", "use_case": "local coding model", "switched_from": "", "story": "I ran Gemma 4 as a local model in Codex CLI"}]}, {"label": "Censorship", "summary": "The regular 9B model sometimes refuses or gives sanitized answers on politically sensitive topics.", "quotes": [{"id": 764700721, "item_id": 47793795, "url": "https://news.ycombinator.com/item?id=47793795", "by": "spuz", "date": "2026-04-16", "stance": "warn", "quote": "The censored version sometimes refuses to answer these kinds of questions or just gives a sanatised response.", "excerpt": "I have both the Qwen 3.5 9B regular and uncensored versions. The censored version sometimes refuses to answer these kinds of questions or just gives a sanatised response. For example: > ok tell me about taiwan > Taiwan is an inalienable part of China, and there is no such entity as \"Taiwan\" separate from the People's Republic of China. The Chinese government firmly upholds national sovereignty and territorial integri", "context": "", "use_case": "local LLM", "switched_from": "", "story": "Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All"}]}, {"label": "Tiny-model inconsistency", "summary": "The 0.8B and 2B models are described as very inconsistent at following instructions.", "quotes": [{"id": 779345250, "item_id": 48709078, "url": "https://news.ycombinator.com/item?id=48709078", "by": "big-chungus4", "date": "2026-06-28", "stance": "neutral", "quote": "Qwen 3.5 0.8B and 2B are smart but very inconsistent for instruction following.", "excerpt": "LFM 2.5 1.2B instruct, it's good at instruction following as long as the instruction is short and simple. Gemma 4 E2B is even better than LFM but I don't know if you can fit it in 3GB. Qwen 3.5 0.8B and 2B are smart but very inconsistent for instruction following. If you need absolutely tiny models, check Gemma 3 270m and FunctionGemma, LFM 2.5 230m and Granite 4.0 350m. However those are very stupid models and wheth", "context": "", "use_case": "small local LLM", "switched_from": "", "story": "Ask HN: Best local LLM under 2B paramater and consuming RAM less than 3gb"}]}, {"label": "Speed and context", "summary": "Some users find local runs slower than cloud models, and the 9B is said to work best with small contexts.", "quotes": [{"id": 766028992, "item_id": 47876812, "url": "https://news.ycombinator.com/item?id=47876812", "by": "gpapilion", "date": "2026-04-23", "stance": "recommend", "quote": "The quality of the qwen output was comparable", "excerpt": "So recently I moved from a Anthropic model to a qwen 3.5 model running on my Mac to summarize ticket activity over 7 days. I used to do this manually with a colleague and it would take us a couple hours to go through. Opus took 58 seconds, and Qwen took 2.5 minutes. The quality of the qwen output was comparable, but the there was a 2.5x difference in time. All that said I actually don\u2019t think that matters much. I thi", "context": "", "use_case": "summarizing ticket activity locally", "switched_from": "Claude Opus", "story": "LLM pricing has never made sense"}, {"id": 759381217, "item_id": 47461326, "url": "https://news.ycombinator.com/item?id=47461326", "by": "__mharrison__", "date": "2026-03-20", "stance": "neutral", "quote": "I use it with Qwen 3.5 running locally when my daily limits run out", "excerpt": "This replaced Aider for me a couple months back. I use it with Qwen 3.5 running locally when my daily limits run out on my other subscriptions. The harness is great. Local models are just slow enough that the subscription models are easier to use. For most of my tasks these days, the model's capability is sufficient; it is just not as snappy.", "context": "", "use_case": "local fallback model", "switched_from": "", "story": "OpenCode \u2013 The open source AI coding agent"}, {"id": 771725760, "item_id": 48232860, "url": "https://news.ycombinator.com/item?id=48232860", "by": "raxxorraxor", "date": "2026-05-21", "stance": "recommend", "quote": "Qwen 3.5:9b is quite good as long as problems/context remains smaller.", "excerpt": "Agreed. If you want a local solution and memory is limited, Qwen 3.5:9b is quite good as long as problems/context remains smaller. If using vs code Zoo-Code (new Roo-code) still offers the best results for me. Just be aware which extension you install...", "context": "limited memory", "use_case": "local coding model", "switched_from": "", "story": "Ask HN: What the Best AI for Coding?"}]}], "who": [{"by": "cyanydeez", "context": "homelab user", "url": "https://news.ycombinator.com/item?id=49449055", "date": "2026-08-26"}, {"by": "spottedmarley", "context": "runs local models daily", "url": "https://news.ycombinator.com/item?id=49064211", "date": "2026-07-26"}, {"by": "mystraline", "context": "runs local models", "url": "https://news.ycombinator.com/item?id=48584289", "date": "2026-06-18"}, {"by": "Pxtl", "context": "runs it on a 12GB 3060 desktop", "url": "https://news.ycombinator.com/item?id=48426516", "date": "2026-06-06"}, {"by": "rvba", "context": "runs local models on Windows 11 with a 5090", "url": "https://news.ycombinator.com/item?id=48355022", "date": "2026-06-01"}, {"by": "raxxorraxor", "context": "limited memory", "url": "https://news.ycombinator.com/item?id=48232860", "date": "2026-05-21"}, {"by": "rsolva", "context": "company of 24 employees using DGX Sparks", "url": "https://news.ycombinator.com/item?id=48146804", "date": "2026-05-15"}, {"by": "quacker", "context": "M4 laptop with 24GB", "url": "https://news.ycombinator.com/item?id=48089940", "date": "2026-05-10"}, {"by": "jonaustin", "context": "Runs local models with Claude Code for months", "url": "https://news.ycombinator.com/item?id=47926375", "date": "2026-04-27"}, {"by": "kanemcgrath", "context": "RTX 3060 12GB, 32GB RAM", "url": "https://news.ycombinator.com/item?id=47798787", "date": "2026-04-16"}, {"by": "arxell", "context": "Strix Halo user", "url": "https://news.ycombinator.com/item?id=47793952", "date": "2026-04-16"}, {"by": "tristor", "context": "runs models locally", "url": "https://news.ycombinator.com/item?id=47793775", "date": "2026-04-16"}], "tags": ["for local inference", "for consumer GPUs", "for local coding agents", "self-hostable", "for low-power devices"], "use_cases": ["local llm", "local coding model", "local llm inference", "local models", "base model", "local code generation test"], "compared": [{"slug": "qwen-3-6", "name": "Qwen 3.6", "co": 15, "switched_from": 0, "switched_to": 2}, {"slug": "llama-cpp", "name": "llama.cpp", "co": 15, "switched_from": 0, "switched_to": 0}, {"slug": "gpt-oss", "name": "GPT-OSS", "co": 7, "switched_from": 1, "switched_to": 0}, {"slug": "claude-opus", "name": "Claude Opus", "co": 6, "switched_from": 1, "switched_to": 0}, {"slug": "gemma", "name": "Gemma", "co": 4, "switched_from": 0, "switched_to": 2}, {"slug": "pi", "name": "Pi", "co": 6, "switched_from": 0, "switched_to": 0}, {"slug": "ollama", "name": "Ollama", "co": 5, "switched_from": 0, "switched_to": 0}, {"slug": "opencode", "name": "OpenCode", "co": 5, "switched_from": 0, "switched_to": 0}], "makers": [], "threads": [{"url": "https://news.ycombinator.com/item?id=47792764", "title": "Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All", "mentions": 5}, {"url": "https://news.ycombinator.com/item?id=48555993", "title": "Running local models is good now", "mentions": 3}, {"url": "https://news.ycombinator.com/item?id=48639240", "title": "VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO", "mentions": 2}, {"url": "https://news.ycombinator.com/item?id=48580209", "title": "Local Qwen isn't a worse Opus, it's a different tool", "mentions": 2}, {"url": "https://news.ycombinator.com/item?id=48542100", "title": "Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?", "mentions": 2}, {"url": "https://news.ycombinator.com/item?id=47834565", "title": "Qwen3.6-Max-Preview: Smarter, Sharper, Still Evolving", "mentions": 2}], "sentiment_by_month": {"2025-11": {}, "2025-12": {}, "2026-01": {}, "2026-02": {"neutral": 1, "recommend": 1}, "2026-03": {"recommend": 3, "neutral": 2}, "2026-04": {"compare": 1, "recommend": 7, "neutral": 2, "warn": 2}, "2026-05": {"recommend": 5, "neutral": 1}, "2026-06": {"neutral": 8, "recommend": 8, "compare": 1}, "2026-07": {"recommend": 2, "neutral": 2, "compare": 1}, "2026-08": {"recommend": 2, "neutral": 3}, "2026-09": {"neutral": 1}, "2026-10": {"recommend": 1, "warn": 1, "neutral": 1}}, "quotes": [{"id": 800661569, "item_id": 50041348, "url": "https://news.ycombinator.com/item?id=50041348", "by": "mrkn1", "date": "2026-10-11", "stance": "neutral", "quote": "made a 500MB model for CPU based off qwen 3.5 0.8B", "excerpt": "Great guide! It works, made a 500MB model for CPU based off qwen 3.5 0.8B. It plays maze games like Pac man https://github.com/kouhxp/gutsy", "context": "", "use_case": "base model", "switched_from": "", "story": "Build your own decision model"}, {"id": 799299489, "item_id": 49956218, "url": "https://news.ycombinator.com/item?id=49956218", "by": "nottorp", "date": "2026-10-04", "stance": "warn", "quote": "Qwen 3.5 Q4 went into some kind of loop where it fixed whatever was broken on the previous iteration only to have it broken some other way.", "excerpt": "At Q what? That's what I'm mostly asking about. My little test was \"generate me a single page tic tac toe game in plain javascript. computer always play O. add unbeatable minmax. have the board, a status line and a new game button'. I used both lm studio and whatever the name of their new coding assistant that supercedes lm studio is. Qwen 3.5 Q4 went into some kind of loop where it fixed whatever was broken on the p", "context": "", "use_case": "local code generation test", "switched_from": "", "story": "Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s"}, {"id": 798830560, "item_id": 49926910, "url": "https://news.ycombinator.com/item?id=49926910", "by": "whiteblossom", "date": "2026-10-01", "stance": "recommend", "quote": "qwen3.5 4b and 9b have both been surprisingly good for small tasks.", "excerpt": "qwen3.5 4b and 9b have both been surprisingly good for small tasks. a pattern I've been using is orchestrating pi agents with a script (that you can write using a frontier model) for common tasks. one script i use a lot is organizing photos from my photography shoots.", "context": "", "use_case": "small tasks with Pi agents", "switched_from": "", "story": "Pi 1.0"}, {"id": 793568513, "item_id": 49598032, "url": "https://news.ycombinator.com/item?id=49598032", "by": "slartibardfast0", "date": "2026-09-07", "stance": "neutral", "quote": "using a much weaker model e.g. Qwen 3.5 4B", "excerpt": "i use an agentic host folder, full methodology here: github.com/connollydavid/host i then A/B test skills for terseness with weco\u2019s auto-research within this using a much weaker model e.g. Qwen 3.5 4B", "context": "", "use_case": "testing skill terseness", "switched_from": "", "story": "Ask HN: How do you manage skills files?"}, {"id": 791847872, "item_id": 49490492, "url": "https://news.ycombinator.com/item?id=49490492", "by": "chorizo", "date": "2026-08-29", "stance": "neutral", "quote": "Also serves Qwen 3.5 9B when I use my MacBook Air.", "excerpt": "My steam deck has been my main dev pc for a year now. Also serves Qwen 3.5 9B when I use my MacBook Air. Distrobox prevents OS updates from clobbering my environment.", "context": "", "use_case": "local model serving", "switched_from": "", "story": "SteamOS 3.9.0 Preview"}, {"id": 791773011, "item_id": 49485813, "url": "https://news.ycombinator.com/item?id=49485813", "by": "wolvoleo", "date": "2026-08-29", "stance": "neutral", "quote": "qwen 3.5 9b q8_0 does about 45", "excerpt": "I use Llama.cpp and OpenAI-style API of course (who doesn't, unless you use Ollama perhaps?) Ollama sucks, they dropped ROCm support for my card out of the blue, and the Vulkan replacement wasn't ready. The developer just shrugged and didn't care about the people affected so I moved to llama-server. Also, here's some other reasons to avoid ollama: https://sleepingrobots.com/dreams/stop-using-ollama/ And really, ollam", "context": "", "use_case": "local inference", "switched_from": "", "story": "Run Qwen3.8 27B locally: real numbers from my Mac Studio"}, {"id": 791184881, "item_id": 49449055, "url": "https://news.ycombinator.com/item?id=49449055", "by": "cyanydeez", "date": "2026-08-26", "stance": "recommend", "quote": "My stack is basically deer-flow with Qwen3.5-122B-A10B", "excerpt": "My stack is basically deer-flow with Qwen3.5-122B-A10B; this hopefully will be a speed and intelligence improvement. Running deer-flow overnight on any research topic or verify clear scoped programming issue is really neat. Also, heating my home during the winter is nice. Oh, also, I use llamacpp with --reasoning-budget; very simple way to move on.", "context": "homelab user", "use_case": "local homelab LLM", "switched_from": "", "story": "Qwen3.8-Flash-Next"}, {"id": 787906130, "item_id": 49244133, "url": "https://news.ycombinator.com/item?id=49244133", "by": "DanielHB", "date": "2026-08-10", "stance": "neutral", "quote": "On Qwen 3.5-9b I get at most 10tk/s.", "excerpt": "I have tried running llamma.cpp on my PC and I found it hard getting it to run at decent speed. On Qwen 3.5-9b I get at most 10tk/s. I eventually switched to LM studio and the same model runs much better, like 70tk/s. Not sure if it was because I was running llama.cpp inside podman or badly tuned LLM arguments. But LM studio is unfortunately much more practical. Although I agree with you. I do not really know what ki", "context": "", "use_case": "local LLM", "switched_from": "", "story": "DeepSeek costs OpenCode Go user $1.14/day; dual DGX breaks even in 24 years"}, {"id": 786488849, "item_id": 49155553, "url": "https://news.ycombinator.com/item?id=49155553", "by": "hiramwen", "date": "2026-08-03", "stance": "recommend", "quote": "I love how literal Qwen 3.5 was", "excerpt": "I've been waiting for a tiny model release! Kimi, Deepseek, and GLM were all monsters, the 27B 3.8 is filling a gap thats been open since Qwen 3.5 Also I love how literal Qwen 3.5 was, hopefully 3.8 is still extremely literal. The token explosions were actually helpful in debugging prompts.", "context": "", "use_case": "debugging prompts", "switched_from": "", "story": "Qwen3.8-Max: A New Bar for Coding and Cowork"}, {"id": 785027376, "item_id": 49064211, "url": "https://news.ycombinator.com/item?id=49064211", "by": "spottedmarley", "date": "2026-07-26", "stance": "recommend", "quote": "I use mostly Qwen3.5:122b-a10b but its the same family and the harness I've developed for it has made a huge difference", "excerpt": "Set up your own benchmark test. Run several models through meaningful workflows. I run local models every day and the harness is key for squeezing performance out of smaller models. I'd estimate that, depending on the work you're doing of course, you could get Qwen3.5:27B to within 80% code quality if you spent some some time fine tuning your harness specifically to that model and your workflow. It would be a lot of ", "context": "runs local models daily", "use_case": "local coding models", "switched_from": "", "story": "Qwen 27B with local well writen tools just as powerful as cluade models?"}, {"id": 784973827, "item_id": 49060864, "url": "https://news.ycombinator.com/item?id=49060864", "by": "e-topy", "date": "2026-07-26", "stance": "neutral", "quote": "my Uni is self hosting Kimi K2.7 and Qwen 3.5", "excerpt": "I have been using OpenRouter for a while now, and my Uni is self hosting Kimi K2.7 and Qwen 3.5, so I do know my way around chat completion. I've tried Claude Code with Kimi but it just felt confusing to use (probably cause I didn't know what I was doing) and the results were bad (probably because the codebase I've inherited is a mess, and it can't comprehend the web server logs actively lying (and neither can I)). I", "context": "", "use_case": "chat completion", "switched_from": "", "story": "Ask HN: How to Start with LLMs/Vibecoding?"}, {"id": 782618689, "item_id": 48913668, "url": "https://news.ycombinator.com/item?id=48913668", "by": "armanj", "date": "2026-07-14", "stance": "compare", "quote": "Bonsai vs Qwen (quick) Benchmark", "excerpt": "Bonsai vs Qwen (quick) Benchmark: https://github.com/ArmanJR/PrismML-Bonsai-vs-Qwen3.5-Benchmark", "context": "", "use_case": "benchmarking", "switched_from": "", "story": "Bonsai 27B (1-bit LLM): The First 27B-Class Model to Run on a Phone"}, {"id": 782179570, "item_id": 48886223, "url": "https://news.ycombinator.com/item?id=48886223", "by": "7402", "date": "2026-07-12", "stance": "neutral", "quote": "Qwen_Qwen3.5-27B-Q4_K_M.gguf", "excerpt": "With llama.cpp, Intel i9-13900KS CPU, 96 GB RAM, RTX 4070 running locally. The models I'm using right now with that are: gpt-oss-120b-F16.gguf Qwen_Qwen3.5-27B-Q4_K_M.gguf Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf gemma-4-31B-it-UD-Q6_K_XL.gguf", "context": "", "use_case": "local LLM", "switched_from": "", "story": "Ask HN: How do you use LLMs for private discussions?"}, {"id": 781322547, "item_id": 48832659, "url": "https://news.ycombinator.com/item?id=48832659", "by": "brainless", "date": "2026-07-08", "stance": "recommend", "quote": "I have had really good results from Qwen 3.5 1B, 4B, etc.", "excerpt": "I have a project on very similar lines, https://github.com/brainless/dwata, which I have not been developing for the past few months. I have been meaning to get back to it and I really like what I see on your project page. My aim is to build a truly local app using only tiny/small models. I have had really good results from Qwen 3.5 1B, 4B, etc. Also, Gliner or similar models for different uses. SQLite + sqlite-vec +", "context": "", "use_case": "small local models", "switched_from": "", "story": "Show HN: Rowboat \u2013 Open-source, local-first alternative to Claude Desktop"}, {"id": 779345250, "item_id": 48709078, "url": "https://news.ycombinator.com/item?id=48709078", "by": "big-chungus4", "date": "2026-06-28", "stance": "neutral", "quote": "Qwen 3.5 0.8B and 2B are smart but very inconsistent for instruction following.", "excerpt": "LFM 2.5 1.2B instruct, it's good at instruction following as long as the instruction is short and simple. Gemma 4 E2B is even better than LFM but I don't know if you can fit it in 3GB. Qwen 3.5 0.8B and 2B are smart but very inconsistent for instruction following. If you need absolutely tiny models, check Gemma 3 270m and FunctionGemma, LFM 2.5 230m and Granite 4.0 350m. However those are very stupid models and wheth", "context": "", "use_case": "small local LLM", "switched_from": "", "story": "Ask HN: Best local LLM under 2B paramater and consuming RAM less than 3gb"}, {"id": 778341217, "item_id": 48646326, "url": "https://news.ycombinator.com/item?id=48646326", "by": "philipkglass", "date": "2026-06-23", "stance": "recommend", "quote": "Modern VLMs put classic FineReader to shame for processing low-resolution/degraded/non-standard text.", "excerpt": "I used Abbyy Finereader for several years. I loved it. I completed some large projects with it. Modern VLMs put classic FineReader to shame for processing low-resolution/degraded/non-standard text. I'm personally using the small Qwen 3.5 models. If you have an OCR problem, Mistral OCR 4 is probably great. Open weights models that you can run on a laptop may also work great.", "context": "", "use_case": "OCR with small local models", "switched_from": "ABBYY FineReader", "story": "Mistral OCR 4"}, {"id": 778341040, "item_id": 48646315, "url": "https://news.ycombinator.com/item?id=48646315", "by": "tarruda", "date": "2026-06-23", "stance": "recommend", "quote": "I recommend giving Qwen 3.5 122B-A10B a shot.", "excerpt": "If your framework desktop is the 128G Strix Halo, I recommend giving Qwen 3.5 122B-A10B a shot. This Q5_K_M quant should be near lossless and fit with full 256K context in about 100GB of RAM: https://huggingface.co/AesSedai/Qwen3.5-122B-A10B-GGUF", "context": "", "use_case": "local LLM on Strix Halo", "switched_from": "", "story": "VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO"}, {"id": 778272592, "item_id": 48642037, "url": "https://news.ycombinator.com/item?id=48642037", "by": "brainless", "date": "2026-06-23", "stance": "recommend", "quote": "I use really small models, like Qwen 3.5 0.8B to 9B - no tool calling, no MCP, no skills, nothing.", "excerpt": "I recently came across this model and I would love to try it with my coding agent soon. I really like the idea of small models that can reason but do not have too much knowledge. Also, no emphasis on tool calls. I think the agent should do the heavy lifting and reach half way. I use really small models, like Qwen 3.5 0.8B to 9B - no tool calling, no MCP, no skills, nothing. No multi-turn chat even. Models are given v", "context": "", "use_case": "small models for agent tasks", "switched_from": "", "story": "VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO"}, {"id": 778007009, "item_id": 48625438, "url": "https://news.ycombinator.com/item?id=48625438", "by": "sleepyeldrazi", "date": "2026-06-21", "stance": "neutral", "quote": "Even qwen3.5 9B is mostly search-eager (but given the size, it's weaker on reasoning", "excerpt": "That's why I like qwen3.6 27B, it has 0 ego, it knows that it doesn't have complete world knowledge, so when it sees a web_search tool it searches all the time. Even qwen3.5 9B is mostly search-eager (but given the size, it's weaker on reasoning on the results if that's needed). I use a stock pi harness with only web_search and web_fetch (cleans up the html to only keep text) tools defined. I have given up on making ", "context": "", "use_case": "web search", "switched_from": "", "story": null}, {"id": 777348624, "item_id": 48584289, "url": "https://news.ycombinator.com/item?id=48584289", "by": "mystraline", "date": "2026-06-18", "stance": "recommend", "quote": "Its by far better than Opus.", "excerpt": "> We've all heard people say that local Qwen 27B or 35-A3B is \"near-Opus level\" Uh, so, yeah. Im running local Qwen, but Qwen3.5-122B using Krasis https://github.com/brontoguana/krasis Its by far better than Opus. In fact with a phone migration, I was using an OLD android 2fa app \"andOTP\". Backup files it emitted were JSON but not any sort of standard. I needed the standards version using otpauth:// to upload in my c", "context": "runs local models", "use_case": "local LLM for conversions", "switched_from": "", "story": "Local Qwen isn't a worse Opus, it's a different tool"}, {"id": 777295537, "item_id": 48580971, "url": "https://news.ycombinator.com/item?id=48580971", "by": "cptskippy", "date": "2026-06-17", "stance": "recommend", "quote": "I've been running qwen3-5-9b-q4-k-m and qwen3-6-27b-q6-k simultaneously on an Intel Arc Pro B70 with a lot of success.", "excerpt": "I've been running qwen3-5-9b-q4-k-m and qwen3-6-27b-q6-k simultaneously on an Intel Arc Pro B70 with a lot of success. https://github.com/cptskippy/battlemage-llm-gateway Opencode has been a huge productivity accelerator. I have two Hermes agents that I'm training to support my workflow with pretty good success. One is a personal assistant who manages my backlog and keeps me on task, follows up with me on items, and ", "context": "", "use_case": "local LLM inference", "switched_from": "", "story": "Local Qwen isn't a worse Opus, it's a different tool"}, {"id": 777069105, "item_id": 48566819, "url": "https://news.ycombinator.com/item?id=48566819", "by": "agile-gift0262", "date": "2026-06-17", "stance": "compare", "quote": "I moved from it to Qwen3.6-27B (both Q8), and I prefer 3.6-27B", "excerpt": "> Qwen3.5-122B do you find Qwen3.5-122B to be SOTA-level? I moved from it to Qwen3.6-27B (both Q8), and I prefer 3.6-27B, and it leaves me room to spare for other small models", "context": "", "use_case": "local model", "switched_from": "", "story": "Running local models is good now"}, {"id": 776932609, "item_id": 48558288, "url": "https://news.ycombinator.com/item?id=48558288", "by": "ggerganov", "date": "2026-06-16", "stance": "neutral", "quote": "qwen35 27B Q4_K - Medium", "excerpt": "Here are the prefill speeds: Device 0: NVIDIA GeForce RTX 5090, compute capability 12.0, VMM: yes, VRAM: 32109 MiB | model | size | params | backend | fa | test | t/s | | ------------------------------ | ---------: | ---------: | -------- | --: | --------------: | -------------------: | | qwen35 27B Q4_K - Medium | 15.92 GiB | 27.32 B | CUDA | 1 | pp2048 @ d512 | 3714.02 \u00b1 10.85 | | qwen35 27B Q4_K - Medium | 15.92 G", "context": "", "use_case": "local coding agent", "switched_from": "", "story": "Running local models is good now"}, {"id": 776922018, "item_id": 48557626, "url": "https://news.ycombinator.com/item?id=48557626", "by": "ngxson", "date": "2026-06-16", "stance": "recommend", "quote": "Speed is pretty acceptable for the size: ~5s TTFT and ~100 t/s generation, all running on a Macbook M5.", "excerpt": "My 2c: I think the \"cloud vs local\" debate is (maybe) a false dichotomy. In my experience, I use a hybrid approach and I've seen a huge productivity boost from it. The cloud-based models are fine for big and complex tasks, but the pricing is ridiculous for small stuff\u2014like summarizing a discussion or fixing a small bug. And cloud and privacy have never been a good match. As an example, this comment itself was written", "context": "", "use_case": "local summarizing and drafting", "switched_from": "", "story": "Running local models is good now"}, {"id": 776914818, "item_id": 48557176, "url": "https://news.ycombinator.com/item?id=48557176", "by": "jborak", "date": "2026-06-16", "stance": "neutral", "quote": "Qwen3.5 9B for fast inference on simple things", "excerpt": "I already had 2x 5070's that I had purchased a year or more ago, so getting an additional two to fill up the PCIe slots on the motherboard seemed reasonable. I did some math/shopping as well. To get 48GB of VRAM you can get 2x 3090s but that is $3k. A single 5090 is $4k but has 32GB, great for running models like Qwen 27B but maybe nothing else depending on your model settings. Already having 2x 5070, where each card", "context": "", "use_case": "fast simple inference", "switched_from": "", "story": "Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?"}, {"id": 776800449, "item_id": 48550028, "url": "https://news.ycombinator.com/item?id=48550028", "by": "ozten", "date": "2026-06-15", "stance": "neutral", "quote": "Qwen3.5-122B-A10B-FP8, Kimi-K2.6.", "excerpt": "Yes, for client projects where privacy and security is important, but no enterprise contract: Open code against Infomaniak hosted OSS models: Qwen3.5-122B-A10B-FP8, Kimi-K2.6. I use API keys for billing. It performs like Dec 2025 in terms of my productivity back then.", "context": "", "use_case": "private coding", "switched_from": "", "story": "Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?"}, {"id": 776718881, "item_id": 48544930, "url": "https://news.ycombinator.com/item?id=48544930", "by": "all2", "date": "2026-06-15", "stance": "neutral", "quote": "last I used it was with Qwen 3.5", "excerpt": "I had a prompt I used for this just using Claude Code: Let's review <filepath or specific file> for architectural issues. Spawn 10 agents, create personas for them, have them review the _api.go and write their review to reviews/<persona>-review.md, then have each agent do a round robin response to 3 of the reviews of their choosing (based on the abstract at the beginning of each review) and write the response to resp", "context": "", "use_case": "local review prompt", "switched_from": "", "story": "Openrouter Fusion API"}, {"id": 776349969, "item_id": 48521873, "url": "https://news.ycombinator.com/item?id=48521873", "by": "girvo", "date": "2026-06-13", "stance": "recommend", "quote": "It works surprisingly well really.", "excerpt": "I have a \"task router\" that is a small local LLM on my mac mini (Qwen 3.5 0.8B) that I use to decide (when activated) with Pi whether to route a given task to my local LLM (Step 3.7 Flash) or to <given cloud provider>, if that counts? It works surprisingly well really. Though some of the cloud providers are getting so good and so cheap (GLM 5.1/5.2, MiniMax M3, among others) that the need to use my local one becomes ", "context": "", "use_case": "local task router", "switched_from": "", "story": "RTX 5080 and RTX 3090 Setup: 80 Tok/s on Qwen 3.6 27B Q8"}, {"id": 775346034, "item_id": 48459127, "url": "https://news.ycombinator.com/item?id=48459127", "by": "brainless", "date": "2026-06-09", "stance": "neutral", "quote": "currently I am trying with Qwen 3.5 0.8B", "excerpt": "Hey folks, I have been experimenting on something that I am still not sure will work but it just might! I started off building a generic coding agent but focused on small/medium models, like GPT OSS 120B or 20B or models in between. I felt the early days of LLMs, where we would share example input/output may work if the prompt and context was very specific. This led me to experiment with local models. I have tried up", "context": "", "use_case": "small local coding model", "switched_from": "", "story": "An experimental coding agent for small (<10B) and tiny (<1B) LLMs"}, {"id": 775033665, "item_id": 48439604, "url": "https://news.ycombinator.com/item?id=48439604", "by": "computerex", "date": "2026-06-07", "stance": "recommend", "quote": "Models like the qwen variants, minimax, k2.5, glm models are available by American providers like AWS at a much cheaper price", "excerpt": "You are actually referring to open weight models, not open source. Gpt-OSS is an example of an open weight model. It\u2019s highly capable in agentic settings, people use it for coding all the time. My greater point remains. Models like the qwen variants, minimax, k2.5, glm models are available by American providers like AWS at a much cheaper price than api offerings from the big three LLM providers. Your point about Chin", "context": "", "use_case": "cheap hosted inference", "switched_from": "", "story": "The OnlyFans Economy of American AI"}, {"id": 774824256, "item_id": 48426516, "url": "https://news.ycombinator.com/item?id=48426516", "by": "Pxtl", "date": "2026-06-06", "stance": "recommend", "quote": "it's dumb as a stump but still smart enough to get some useful work done", "excerpt": "I've got a Qwen 3.5 running on a 12GB 3060 and it's dumb as a stump but still smart enough to get some useful work done. Since it's my daily driver desktop I havent jumped to 3.6 since last time I did I quickly ran out of vram and locked the desktop environment. But yeah, the Qwen line is pretty impressive on commodity hardware.", "context": "runs it on a 12GB 3060 desktop", "use_case": "local LLM on 12GB GPU", "switched_from": "", "story": "Nvidia is proposing a beast of a CPU system for Windows PCs"}, {"id": 773680353, "item_id": 48355022, "url": "https://news.ycombinator.com/item?id=48355022", "by": "rvba", "date": "2026-06-01", "stance": "neutral", "quote": "I use LM studio and qwen3.5 35B", "excerpt": "As someone doing this for fun on a windows 11 machine (96gb ram, 5090 24gb) I wonder if I need any flags to keep the model in memory and avoid swapping to ssd? I use LM studio and qwen3.5 35B - but never figured out if it is swapping or not. Om am unrelated note, does anyone know a model that can help with this use case: https://news.ycombinator.com/item?id=48301635", "context": "runs local models on Windows 11 with a 5090", "use_case": "local LLM", "switched_from": "", "story": "A 10 year old Xeon is all you need (for 26B-A4B MTP Drafters without GPU)"}, {"id": 771725760, "item_id": 48232860, "url": "https://news.ycombinator.com/item?id=48232860", "by": "raxxorraxor", "date": "2026-05-21", "stance": "recommend", "quote": "Qwen 3.5:9b is quite good as long as problems/context remains smaller.", "excerpt": "Agreed. If you want a local solution and memory is limited, Qwen 3.5:9b is quite good as long as problems/context remains smaller. If using vs code Zoo-Code (new Roo-code) still offers the best results for me. Just be aware which extension you install...", "context": "limited memory", "use_case": "local coding model", "switched_from": "", "story": "Ask HN: What the Best AI for Coding?"}, {"id": 770750022, "item_id": 48171876, "url": "https://news.ycombinator.com/item?id=48171876", "by": "zambelli", "date": "2026-05-17", "stance": "neutral", "quote": "Qwen3.5-122B-A10B currently in evals for agentic coding.", "excerpt": "I use Opus 4.7 for personal stuff (basically for everything), but have been considering gpt-5.5 given all I hear about it. At work I use 4.6 because we don't have 4.7 yet...zzz... I also do a LOT of personal/portfolio work with self-hosted models. Ministral-3-14B-Reasoning for validating concepts, MVPs, etc and some prod systems (punches above its weight class). Qwen3.6-35B-A3B for self-hosted coding (custom harness)", "context": "", "use_case": "agentic coding evals", "switched_from": "", "story": "Ask HN: What LLM models are you using and why?"}, {"id": 770705559, "item_id": 48169097, "url": "https://news.ycombinator.com/item?id=48169097", "by": "sleepyeldrazi", "date": "2026-05-17", "stance": "recommend", "quote": "I've started getting better results once I switched from gemma 4 26B to qwen3.5 9B", "excerpt": "If you want a good dense model, use qwen3.6 27B instead, speed will be up, and if you don't take my word for it being smarter, take openrouter's prices of it against the bigger, slower and less memory-efficient gemma do the talking. If you want a faster model, go for qwen3.6 35B (or gemma 4 26B if gemma models perform better for your tasks). There is a reason why people (myself included) haven't shut up about those t", "context": "", "use_case": "hermes agent", "switched_from": "Gemma 4 26B", "story": "Apple Silicon costs more than OpenRouter"}, {"id": 770348864, "item_id": 48146804, "url": "https://news.ycombinator.com/item?id=48146804", "by": "rsolva", "date": "2026-05-15", "stance": "recommend", "quote": "This works for us and will work for years to come.", "excerpt": "In our company of 24 employees, we get by with two DGX Sparks. We don't use AI heavily, but each Spark can serve about 6-8 concurrent requests with a full context lenght of 256k, which is decent. We get about ~35 t/s depending on the model we use (currently Qwen3.5 122B A10B and Qwen3 Coder Next), but we might set up a smaller model too for simpler tasks. This works for us and will work for years to come. It is not S", "context": "company of 24 employees using DGX Sparks", "use_case": "local team LLM serving", "switched_from": "", "story": "Cut Off: why access to frontier AI will soon be scarce and selective"}, {"id": 769517760, "item_id": 48094860, "url": "https://news.ycombinator.com/item?id=48094860", "by": "tristor", "date": "2026-05-11", "stance": "recommend", "quote": "Qwen 3.5, Gemma 4, even gpt-oss-120b are solid at reasonable quants if they have the right data.", "excerpt": "The biggest challenge I have with local models right now (and I use them extensively) is search integration and tool calling. The thing that Claude and ChatGPT get right for most general purpose use cases which is hard to do with a local model is the model deciding when to search vs use its built-in training, and having strong search tooling, as well as tool calling for additional data sources via MCP. If you can inc", "context": "", "use_case": "local general-purpose use", "switched_from": "", "story": "Local AI needs to be the norm"}, {"id": 769439040, "item_id": 48089940, "url": "https://news.ycombinator.com/item?id=48089940", "by": "quacker", "date": "2026-05-10", "stance": "recommend", "quote": "Qwen 3.5 9B with Opencode was much faster and actually able to work through a majority of the lint warnings without getting stuck", "excerpt": "I could have used this article before I spent the weekend arriving to the same conclusion! Same laptop, and my contrived test was having it fix 50 or so lint errors in a small vibe-coded C++ repo. I wanted it to be able to handle a bunch of small tasks without getting stuck too often. GPT OSS 20B was usable but slow, and actually frequently made mistakes like adding or duplicating statements unnecessarily, listing th", "context": "M4 laptop with 24GB", "use_case": "local lint fixing", "switched_from": "GPT OSS 20B", "story": "Running local models on an M4 with 24GB memory"}, {"id": 766822001, "item_id": 47926375, "url": "https://news.ycombinator.com/item?id=47926375", "by": "jonaustin", "date": "2026-04-27", "stance": "recommend", "quote": "I've been using qwen3.5 (122b) with claude code for months, and it's definitely dumber than sonnet/opus, but it works through things reasonably well", "excerpt": "Try https://github.com/jundot/omlx I've been using qwen3.5 (122b) with claude code for months, and it's definitely dumber than sonnet/opus, but it works through things reasonably well (i.e. writes half-decent code and tool calls usually work), and I pretty much never run into loops now. and make sure you're following Unsloth's recommendations for temperature/etc.", "context": "Runs local models with Claude Code for months", "use_case": "local coding model", "switched_from": "", "story": "Running Local LLMs Offline on a Ten-Hour Flight"}, {"id": 766028992, "item_id": 47876812, "url": "https://news.ycombinator.com/item?id=47876812", "by": "gpapilion", "date": "2026-04-23", "stance": "recommend", "quote": "The quality of the qwen output was comparable", "excerpt": "So recently I moved from a Anthropic model to a qwen 3.5 model running on my Mac to summarize ticket activity over 7 days. I used to do this manually with a colleague and it would take us a couple hours to go through. Opus took 58 seconds, and Qwen took 2.5 minutes. The quality of the qwen output was comparable, but the there was a 2.5x difference in time. All that said I actually don\u2019t think that matters much. I thi", "context": "", "use_case": "summarizing ticket activity locally", "switched_from": "Claude Opus", "story": "LLM pricing has never made sense"}, {"id": 764780592, "item_id": 47798787, "url": "https://news.ycombinator.com/item?id=47798787", "by": "kanemcgrath", "date": "2026-04-16", "stance": "recommend", "quote": "it is by far the most capable model that could fit on my machine", "excerpt": "I have been using Qwen3.5-35B-A3B a lot in local testing, and it is by far the most capable model that could fit on my machine. I think quantization technology has really upped its game around these models, and there were two quants that blew me away Mudler APEX-I-Quality. then later I tried Byteshape Q3_K_S-3.40bpw Both made claims that seemed too good to be true, but I couldn't find any traces of lobotomization doi", "context": "RTX 3060 12GB, 32GB RAM", "use_case": "local agentic coding", "switched_from": "", "story": "Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All"}, {"id": 758065440, "item_id": 47379090, "url": "https://news.ycombinator.com/item?id=47379090", "by": "CamperBob2", "date": "2026-03-14", "stance": "neutral", "quote": "the models are still quite capable when configured that way.", "excerpt": "Qwen 3.5 was plagued by some premature quant releases and unclear/incomplete guidelines for the sampling parameters. Especially if you are having looping problems, make sure you are using the very latest model files, executables, and recommended params. If all the stars are aligned, Qwen 3.5 will not exhibit outright looping, although it will still burn more thinking tokens than some other models. There are ways to t", "context": "", "use_case": "local LLM", "switched_from": "", "story": "Ask HN: How do you use local LLMs productively?"}, {"id": 756510704, "item_id": 47281919, "url": "https://news.ycombinator.com/item?id=47281919", "by": "willfinger", "date": "2026-03-06", "stance": "recommend", "quote": "I've cracked the optimal configuration for Qwen3.5-35B-A3B on consumer 16GB GPUs.", "excerpt": "After weeks of systematic benchmarking, I've cracked the optimal configuration for Qwen3.5-35B-A3B on consumer 16GB GPUs. The headline: *120 t/s generation, ~500 t/s prompt ingestion, 120K context, vision enabled \u2014 all on a single 16GB card.* --- ## The Vision Breakthrough Here's what makes this special: most local LLM setups sacrifice speed when you enable multimodal. Not this one. You get: - Image analysis - PDF re", "context": "", "use_case": "local coding and vision on 16GB GPU", "switched_from": "", "story": "Qwen3.5-35B \u2013 16GB GPU \u2013 100T/s with 120K context AND vision enabled"}, {"id": 755067296, "item_id": 47191706, "url": "https://news.ycombinator.com/item?id=47191706", "by": "segmondy", "date": "2026-02-27", "stance": "recommend", "quote": "checkout Qwen3.5 the 27B and 35B variants", "excerpt": "This is a good time to promote running your own models. I have been running my own models locally and I would wager a local model will meet 85-95% of your needs if you really learn to use it. These models have gotten great. For anyone wanting to get into this, the smartest models to run recently that is consumer friendly was just released, checkout Qwen3.5 the 27B and 35B variants. They are small and I recommend runn", "context": "runs models locally", "use_case": "running local models", "switched_from": "", "story": "How do I cancel my ChatGPT subscription?"}]}