{"slug": "gemma", "name": "Gemma", "category": "ai", "category_name": "AI models and assistants", "description": "Google DeepMind's family of open-weight language models that can run locally.", "recommenders": 58, "commenters": 102, "warners": 4, "score": 59.362, "rising": 2.6, "first": "2025-11-15", "latest": "2026-10-22", "why": "Liked as a capable open-weight model for its size that runs well on consumer hardware, and for writing and small utility tasks.", "why_is_quote": false, "has_verdict": true, "sentiment": {"recommend": 65, "neutral": 37, "compare": 3, "warn": 5}, "spark": [0, 0, 0, 1, 0, 11, 3, 19, 13, 11, 5, 2], "listed": true, "loved": ["Local hardware", "Writing quality"], "caveat": "Compaction failures", "short_quote": "Gemma 4 is performing really well for me, and seems to hallucinate much less than other models I tried in this size range.", "icon": true, "domain": null, "github_repo": "github.com/google-deepmind/gemma", "official_url": "https://ai.google.dev/gemma", "pricing": "free", "aliases": ["Gemma 4", "Gemma", "Gemma4", "Gemma 3", "Gemma-4", "Gemma3"], "verdict": [{"text": "Liked as a capable open-weight model for its size that runs well on consumer hardware", "cites": [{"id": 782202705, "item_id": 48887669, "url": "https://news.ycombinator.com/item?id=48887669", "by": "MyMemoryfails", "date": "2026-07-12", "stance": "recommend", "quote": "Its shockingly capable for its size.", "excerpt": "Running Local LLM isn't really a lot effort anymore, since its one command away to get the familar interface of chat AI. This setup requires 24GB ish VRAM and 32GB ram, so if you have capable gaming PC which can do AAA games, you can run it. - llama-server -hf ggml-org/gemma-4-26b-a4b-it-GGUF:Q4_K_M After that simply open browser and enter: http://localhost:8080 What this does: This will download Gemma4 AI with 26B p", "context": "", "use_case": "private local chat", "switched_from": "", "story": "Ask HN: How do you use LLMs for private discussions?"}, {"id": 781616241, "item_id": 48851015, "url": "https://news.ycombinator.com/item?id=48851015", "by": "bevekspldnw", "date": "2026-07-09", "stance": "recommend", "quote": "Gemma4 models are unbeatable at their size.", "excerpt": "Gemini is more than fine as a mobile application, and will be the \u201cbrains\u201d of the currently braindead Siri, between than and Android it\u2019s hard to come up with an argument they are behind. Likewise, Gemma4 models are unbeatable at their size. So I use Gemini for coding? Hell no, but that\u2019s not the same as Google failing writ large. Googles only real goal is to retain the ownership and dominance of the online world the", "context": "", "use_case": "", "switched_from": "", "story": "Introducing Muse Spark 1.1"}, {"id": 786514896, "item_id": 49157181, "url": "https://news.ycombinator.com/item?id=49157181", "by": "nextblock", "date": "2026-08-03", "stance": "recommend", "quote": "I use Gemma 4 right now on a Debian Linux with no GUI that I access remotely. Its works pretty well", "excerpt": "I have a 5080 Super RTX but its still not enough to run these big models. I use Gemma 4 right now on a Debian Linux with no GUI that I access remotely. Its works pretty well, but still not as good as any of the big models like Opus 5... Can't wait to be able to buy a personal home server that can run much bigger models. I heard some companies have started building AI PCs only used to have a personal AI model at home,", "context": "has a 5080 Super RTX GPU", "use_case": "local LLM on home server", "switched_from": "", "story": "Qwen3.8-Max: A New Bar for Coding and Cowork"}, {"id": 782647040, "item_id": 48915440, "url": "https://news.ycombinator.com/item?id=48915440", "by": "SwellJoe", "date": "2026-07-14", "stance": "recommend", "quote": "Gemma 4 is the best all-rounder among the models I can self-host", "excerpt": "I want 31B. The 12B 4-bit QAT is already small enough to run well enough on every device I use regularly, including phone and tablet, I don't need a 1-bit or ternary version of the 12B. But, what I really want is for Google to release bigger Gemma 4 models, particularly a bigger MoE, like a ~70B or ~120B. Gemma 4 is the best all-rounder among the models I can self-host even though I've got a 128GB Strix Halo. A 4-bit", "context": "has a 128GB Strix Halo", "use_case": "self-hosted local model", "switched_from": "", "story": "Bonsai 27B (1-bit LLM): The First 27B-Class Model to Run on a Phone"}]}, {"text": ", and for writing and small utility tasks.", "cites": [{"id": 791115043, "item_id": 49444690, "url": "https://news.ycombinator.com/item?id=49444690", "by": "SturgeonsLaw", "date": "2026-08-26", "stance": "recommend", "quote": "I use Qwen/Claude for coding and Gemini/Gemma to draft text intended for a human.", "excerpt": "I use Qwen/Claude for coding and Gemini/Gemma to draft text intended for a human. To me that feels like each model playing to their strengths.", "context": "", "use_case": "drafting human-facing text", "switched_from": "", "story": "Analyzing student votes across AI models for college essay help"}, {"id": 793822033, "item_id": 49613877, "url": "https://news.ycombinator.com/item?id=49613877", "by": "dofm", "date": "2026-09-08", "stance": "recommend", "quote": "I absolutely don't hate the way Gemma 4 writes", "excerpt": "I don't think so, no, because it extends to LLM-generated text I want to recall. I use LLMs to generate starter/tutorial material. I may hate the way Claude writes but I absolutely don't hate the way Gemma 4 writes. But I have to continuously consult it in a way that I do not with human-written text, which gets its message across in a more persistent way I find less troubling. (This non-memorability extends to AI ima", "context": "", "use_case": "generating tutorial material", "switched_from": "", "story": "Benchmarking Qwen3.8 27B quantizations: 4-bit holds up, 1-bit collapses"}, {"id": 783889120, "item_id": 48993070, "url": "https://news.ycombinator.com/item?id=48993070", "by": "efficax", "date": "2026-07-21", "stance": "recommend", "quote": "The local model is too slow in my setup for a chat agent, but for memory extraction it works pretty well", "excerpt": "I use a gemma4 model locally to extract content from messages to a personal agent I'm building for its memory graph (to break the message up into the topic, source (assistant or owner), facts, entities, etc. in the message content (all getting thrown into a magma-esque graph using NLEmbeddings for memory search). This is for a custom personal agent that targets deepseek-v4 flash. The local model is too slow in my set", "context": "building a custom personal agent", "use_case": "local memory extraction for agent", "switched_from": "", "story": "Nativ: Run frontier open models locally on your Mac"}, {"id": 774205920, "item_id": 48387870, "url": "https://news.ycombinator.com/item?id=48387870", "by": "quickthoughts", "date": "2026-06-03", "stance": "recommend", "quote": "I use small models like Gemma to improve transcriptions from ASR models amongst other micro-tasks.", "excerpt": "I use small models like Gemma to improve transcriptions from ASR models amongst other micro-tasks. I actually built out a fine-tuning whisper pipeline with all local (smaller) models meaning no cloud/big-tech co is able to train/sell my (private) data. Repo is https://github.com/Rebreda/listenr - mainly geared toward Whisper fine-tuning, AMD hardware and local inference", "context": "", "use_case": "improving ASR transcriptions", "switched_from": "", "story": "Gemma 4 12B: A unified, encoder-free multimodal model"}]}, {"text": " The most common complaint is that it falls apart after context compaction or at long contexts", "cites": [{"id": 799009537, "item_id": 49938096, "url": "https://news.ycombinator.com/item?id=49938096", "by": "roosterIllusi0n", "date": "2026-10-02", "stance": "warn", "quote": "The gemma models were terrible and fell apart during compaction.", "excerpt": "I am using qwen3.8-27B-UD-Q3_K_XL on a 5080 16gb card with 16gb of system ram at around 43-45tok/s with 65k context. I am using that model to set up containers on a proxmox server with two b70s and 96gb ram. The Q3 model implemented 10 different chat models, 2 image models, and 7 web based harnesses so I can compare. It can rewrite any part to improve it. qwen3.8 is more than capable for software dev. The gemma model", "context": "", "use_case": "local coding", "switched_from": "", "story": "Several vulnerabilities have been discovered in the Linux kernel"}, {"id": 789487265, "item_id": 49342954, "url": "https://news.ycombinator.com/item?id=49342954", "by": "roosterIllusi0n", "date": "2026-08-18", "stance": "warn", "quote": "My issue with Gemma 4 is that any task fails to complete after any compaction event.", "excerpt": "My issue with Gemma 4 is that any task fails to complete after any compaction event. It often ends up in a loop that keeps compacting and showing the same compaction output. Qwen3.8-27B-IQ4_XS was a massive improvement. It's tasks survive compaction and actually get completed. I switched to Qwen3.8-27B-UD-Q3_K_XL for better performance and its working just as well. Gemma4 screwed up a proxmox install I had. I booted ", "context": "", "use_case": "local agentic coding", "switched_from": "", "story": "Qwen 3.8 27B is excellent, but it defaults to overthinking things"}, {"id": 768523264, "item_id": 48032704, "url": "https://news.ycombinator.com/item?id=48032704", "by": "gertlabs", "date": "2026-05-05", "stance": "recommend", "quote": "Gemma 4 26B writes some really efficient code, especially considering the size.", "excerpt": "This is true, we have the numbers to back it up on https://gertlabs.com/rankings?mode=oneshot_coding (check out the efficiency chart too) GPT 5.5/5.4 are the smartest models, but at great token / code bloat cost. Qwen 3.6 Max strikes a good balance. But Gemma 4 26B writes some really efficient code, especially considering the size. Things do start falling apart under higher contexts.", "context": "", "use_case": "efficient code generation", "switched_from": "", "story": "Accelerating Gemma 4: faster inference with multi-token prediction drafters"}]}, {"text": ", with others reporting weak tool calling for coding.", "cites": [{"id": 785714240, "item_id": 49107140, "url": "https://news.ycombinator.com/item?id=49107140", "by": "sznio", "date": "2026-07-30", "stance": "warn", "quote": "Gemma feels nice to write with but every time I use it for coding it struggles with tool calling significantly.", "excerpt": "Nice. Gemma feels nice to write with but every time I use it for coding it struggles with tool calling significantly.", "context": "", "use_case": "writing and coding", "switched_from": "", "story": "Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac"}, {"id": 767455153, "item_id": 47965947, "url": "https://news.ycombinator.com/item?id=47965947", "by": "basscodes", "date": "2026-04-30", "stance": "warn", "quote": "I've been struggling to get Gemma 4 to run with it reliably", "excerpt": "Hey everyone! At eno we've been using local models in our evals to benchmark our harness and with the recent releases of some of these open-weight models like Qwen3.6 we've found that they are only a couple points off in accuracy compared to the cloud SOTA models we use. We thought that was pretty incredible, so we decided to expose the provider configuration to users so they can bring their local models to use with ", "context": "", "use_case": "local agentic search", "switched_from": "", "story": "Running Local Agentic PDF Search with Eno"}]}], "praise": [{"label": "Local hardware", "summary": "Commenters run it on consumer GPUs, Macs and budget PCs with acceptable results.", "quotes": [{"id": 786514896, "item_id": 49157181, "url": "https://news.ycombinator.com/item?id=49157181", "by": "nextblock", "date": "2026-08-03", "stance": "recommend", "quote": "I use Gemma 4 right now on a Debian Linux with no GUI that I access remotely. Its works pretty well", "excerpt": "I have a 5080 Super RTX but its still not enough to run these big models. I use Gemma 4 right now on a Debian Linux with no GUI that I access remotely. Its works pretty well, but still not as good as any of the big models like Opus 5... Can't wait to be able to buy a personal home server that can run much bigger models. I heard some companies have started building AI PCs only used to have a personal AI model at home,", "context": "has a 5080 Super RTX GPU", "use_case": "local LLM on home server", "switched_from": "", "story": "Qwen3.8-Max: A New Bar for Coding and Cowork"}, {"id": 783409154, "item_id": 48963072, "url": "https://news.ycombinator.com/item?id=48963072", "by": "jonhohle", "date": "2026-07-18", "stance": "recommend", "quote": "smaller Qwens and Gemmas work well", "excerpt": "Plug it in, get the cuda drivers, run ollama or equivalent, and go from there. I use an 4GB Quadro card from the same generation and depending on the model can get faster than I can read output. It\u2019s not as good as commercial models, but smaller Qwens and Gemmas work well. The Quadro came with a Dell refurb that cost about as much as the card would have by itself at the time. (I\u2019m doing this with passthrough on a Fre", "context": "", "use_case": "local models on old GPU", "switched_from": "", "story": "GTX 1080s: Testing a Legend"}, {"id": 776320384, "item_id": 48520024, "url": "https://news.ycombinator.com/item?id=48520024", "by": "bachmeier", "date": "2026-06-13", "stance": "recommend", "quote": "I run Gemma 4 26b (mixture of experts) and Qwen 3 coder with Ollama.", "excerpt": "> The upfront cost is steep and the models you can actually run at home are weaker than what the frontier labs ship, so this only pays off if you can keep the rig busy with long running tasks where a slower, cheaper model grinds away overnight. Most people can\u2019t keep a home machine that loaded, and the hardware you buy today may look like a bad bet in a year. Oh, so this is not a post about AI coding at home. It's ab", "context": "64 GB RAM, no GPU", "use_case": "local AI coding", "switched_from": "", "story": "AI Coding at Home Without Going Broke"}]}, {"label": "Writing quality", "summary": "Several prefer it for drafting human-facing text, stories and creative role-play.", "quotes": [{"id": 791115043, "item_id": 49444690, "url": "https://news.ycombinator.com/item?id=49444690", "by": "SturgeonsLaw", "date": "2026-08-26", "stance": "recommend", "quote": "I use Qwen/Claude for coding and Gemini/Gemma to draft text intended for a human.", "excerpt": "I use Qwen/Claude for coding and Gemini/Gemma to draft text intended for a human. To me that feels like each model playing to their strengths.", "context": "", "use_case": "drafting human-facing text", "switched_from": "", "story": "Analyzing student votes across AI models for college essay help"}, {"id": 793822033, "item_id": 49613877, "url": "https://news.ycombinator.com/item?id=49613877", "by": "dofm", "date": "2026-09-08", "stance": "recommend", "quote": "I absolutely don't hate the way Gemma 4 writes", "excerpt": "I don't think so, no, because it extends to LLM-generated text I want to recall. I use LLMs to generate starter/tutorial material. I may hate the way Claude writes but I absolutely don't hate the way Gemma 4 writes. But I have to continuously consult it in a way that I do not with human-written text, which gets its message across in a more persistent way I find less troubling. (This non-memorability extends to AI ima", "context": "", "use_case": "generating tutorial material", "switched_from": "", "story": "Benchmarking Qwen3.8 27B quantizations: 4-bit holds up, 1-bit collapses"}, {"id": 787913168, "item_id": 49244573, "url": "https://news.ycombinator.com/item?id=49244573", "by": "jakswa", "date": "2026-08-10", "stance": "recommend", "quote": "you should check out Gemma 4 for creative DMing use case", "excerpt": "I like the tabletop RPG use case, and wanted to say: If your hardware likes it you should check out Gemma 4 for creative DMing use case. I found it to be much better at holding the plotlines and being creative on gaming turns. My experimental case was an audio-only Zork and Gemma 12B and even E4B were pretty good!", "context": "", "use_case": "creative tabletop RPG DMing", "switched_from": "", "story": "Meta Muse Glimmer \u2013 open weights 30B local coding model"}]}, {"label": "Strong for size", "summary": "Described as unusually capable relative to its parameter count.", "quotes": [{"id": 782202705, "item_id": 48887669, "url": "https://news.ycombinator.com/item?id=48887669", "by": "MyMemoryfails", "date": "2026-07-12", "stance": "recommend", "quote": "Its shockingly capable for its size.", "excerpt": "Running Local LLM isn't really a lot effort anymore, since its one command away to get the familar interface of chat AI. This setup requires 24GB ish VRAM and 32GB ram, so if you have capable gaming PC which can do AAA games, you can run it. - llama-server -hf ggml-org/gemma-4-26b-a4b-it-GGUF:Q4_K_M After that simply open browser and enter: http://localhost:8080 What this does: This will download Gemma4 AI with 26B p", "context": "", "use_case": "private local chat", "switched_from": "", "story": "Ask HN: How do you use LLMs for private discussions?"}, {"id": 781616241, "item_id": 48851015, "url": "https://news.ycombinator.com/item?id=48851015", "by": "bevekspldnw", "date": "2026-07-09", "stance": "recommend", "quote": "Gemma4 models are unbeatable at their size.", "excerpt": "Gemini is more than fine as a mobile application, and will be the \u201cbrains\u201d of the currently braindead Siri, between than and Android it\u2019s hard to come up with an argument they are behind. Likewise, Gemma4 models are unbeatable at their size. So I use Gemini for coding? Hell no, but that\u2019s not the same as Google failing writ large. Googles only real goal is to retain the ownership and dominance of the online world the", "context": "", "use_case": "", "switched_from": "", "story": "Introducing Muse Spark 1.1"}, {"id": 768523264, "item_id": 48032704, "url": "https://news.ycombinator.com/item?id=48032704", "by": "gertlabs", "date": "2026-05-05", "stance": "recommend", "quote": "Gemma 4 26B writes some really efficient code, especially considering the size.", "excerpt": "This is true, we have the numbers to back it up on https://gertlabs.com/rankings?mode=oneshot_coding (check out the efficiency chart too) GPT 5.5/5.4 are the smartest models, but at great token / code bloat cost. Qwen 3.6 Max strikes a good balance. But Gemma 4 26B writes some really efficient code, especially considering the size. Things do start falling apart under higher contexts.", "context": "", "use_case": "efficient code generation", "switched_from": "", "story": "Accelerating Gemma 4: faster inference with multi-token prediction drafters"}]}, {"label": "Utility tasks", "summary": "Used for classification, memory extraction, transcription cleanup and similar micro-tasks.", "quotes": [{"id": 800421665, "item_id": 50026354, "url": "https://news.ycombinator.com/item?id=50026354", "by": "denverllc", "date": "2026-10-09", "stance": "recommend", "quote": "found a small classifier (gemma 4) with traditional embeddings worked 2x as well.", "excerpt": "> but I question how much folks are actually experimenting with what's out there I did. Originally I had a project that I had been wanting to do and thought to use a decision model for it. Jev, OpenAI, etc. are all within percentage points of each other. Then I used traditional ML and found a small classifier (gemma 4) with traditional embeddings worked 2x as well. Jev is the general purpose ML pipeline for when you ", "context": "", "use_case": "classification", "switched_from": "Jev", "story": "Typesafe AI raises $870M at $7.5B"}, {"id": 783889120, "item_id": 48993070, "url": "https://news.ycombinator.com/item?id=48993070", "by": "efficax", "date": "2026-07-21", "stance": "recommend", "quote": "The local model is too slow in my setup for a chat agent, but for memory extraction it works pretty well", "excerpt": "I use a gemma4 model locally to extract content from messages to a personal agent I'm building for its memory graph (to break the message up into the topic, source (assistant or owner), facts, entities, etc. in the message content (all getting thrown into a magma-esque graph using NLEmbeddings for memory search). This is for a custom personal agent that targets deepseek-v4 flash. The local model is too slow in my set", "context": "building a custom personal agent", "use_case": "local memory extraction for agent", "switched_from": "", "story": "Nativ: Run frontier open models locally on your Mac"}, {"id": 774205920, "item_id": 48387870, "url": "https://news.ycombinator.com/item?id=48387870", "by": "quickthoughts", "date": "2026-06-03", "stance": "recommend", "quote": "I use small models like Gemma to improve transcriptions from ASR models amongst other micro-tasks.", "excerpt": "I use small models like Gemma to improve transcriptions from ASR models amongst other micro-tasks. I actually built out a fine-tuning whisper pipeline with all local (smaller) models meaning no cloud/big-tech co is able to train/sell my (private) data. Repo is https://github.com/Rebreda/listenr - mainly geared toward Whisper fine-tuning, AMD hardware and local inference", "context": "", "use_case": "improving ASR transcriptions", "switched_from": "", "story": "Gemma 4 12B: A unified, encoder-free multimodal model"}]}, {"label": "Reliable all-rounder", "summary": "Some call it the most reliable general-purpose model they can self-host.", "quotes": [{"id": 782647040, "item_id": 48915440, "url": "https://news.ycombinator.com/item?id=48915440", "by": "SwellJoe", "date": "2026-07-14", "stance": "recommend", "quote": "Gemma 4 is the best all-rounder among the models I can self-host", "excerpt": "I want 31B. The 12B 4-bit QAT is already small enough to run well enough on every device I use regularly, including phone and tablet, I don't need a 1-bit or ternary version of the 12B. But, what I really want is for Google to release bigger Gemma 4 models, particularly a bigger MoE, like a ~70B or ~120B. Gemma 4 is the best all-rounder among the models I can self-host even though I've got a 128GB Strix Halo. A 4-bit", "context": "has a 128GB Strix Halo", "use_case": "self-hosted local model", "switched_from": "", "story": "Bonsai 27B (1-bit LLM): The First 27B-Class Model to Run on a Phone"}, {"id": 788508609, "item_id": 49281788, "url": "https://news.ycombinator.com/item?id=49281788", "by": "SwellJoe", "date": "2026-08-12", "stance": "recommend", "quote": "The reason I use Gemma 4 so much often comes down to how reliable it is", "excerpt": "Counterpoint: I've been using the Unsloth Gemma 4 quants extensively since very soon after release (on ROCm and Apple Silicon, I don't have any Nvidia hardware big enough), pretty much every quantization down to 4 bits (the QAT is the business, indistinguishable from the full-fat version, runs great on a slightly chonky desktop or laptop), and I haven't had any issues. The reason I use Gemma 4 so much often comes dow", "context": "Runs local models on ROCm and Apple Silicon", "use_case": "local model experiments", "switched_from": "", "story": "Qwen/Qwen3.8-2.4T-A95B"}, {"id": 785598288, "item_id": 49099893, "url": "https://news.ycombinator.com/item?id=49099893", "by": "jwr", "date": "2026-07-29", "stance": "recommend", "quote": "Gemma is actually a very good general-purpose model that can work with multiple languages.", "excerpt": "In defense of this model, Gemma is actually a very good general-purpose model that can work with multiple languages. I use it for spam classification and for processing dictation, which means that I hold the entire model in memory all of the time, which is somewhat problematic (64GB RAM total, but heavy usage by docker, databases, etc)", "context": "", "use_case": "spam classification, dictation processing", "switched_from": "", "story": "Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac"}]}, {"label": "Speed", "summary": "Users report high local token rates on capable hardware.", "quotes": [{"id": 790939040, "item_id": 49433690, "url": "https://news.ycombinator.com/item?id=49433690", "by": "super_mario", "date": "2026-08-25", "stance": "recommend", "quote": "I get 100 tokens/s with Gemma 4 27b model", "excerpt": "I was in the same situation, I used maxed out 15'' M3 Max MacBook Pro docked to Studio Display closed on vertical stand behind the screen. It was fine for office work, but running local LLMs would definitely overheat it. The battery started degrading purely due to heat issues. And it was audible as well. I decided to get Mac Studio M4 Max, also all maxed out config and the cooling is so much better that I can run loc", "context": "", "use_case": "local LLM inference", "switched_from": "", "story": "New Mac Studio with M5 Max and M5 Ultra"}, {"id": 777075520, "item_id": 48567220, "url": "https://news.ycombinator.com/item?id=48567220", "by": "xlii", "date": "2026-06-17", "stance": "recommend", "quote": "I squeeze ~130 t/s on Gemma or ~70t/s on Qwen.", "excerpt": "> I use a lot of local models and they're still pretty painful to run locally. This really depends on how and what you're using. e.g. I can't suffer through slowness of inference on Macbook but I have gaming rig with quite powerful GPU and I squeeze ~130 t/s on Gemma or ~70t/s on Qwen. Tuning is not optional as well. Qwen on temperatures > 0.5 is unusable for coding and I found sweet spot around 0.32 for coding. Spec", "context": "has gaming rig with powerful GPU", "use_case": "local inference", "switched_from": "", "story": "Running local models is good now"}]}], "caveats": [{"label": "Compaction failures", "summary": "Tasks reportedly break or loop after context compaction or at higher contexts.", "quotes": [{"id": 799009537, "item_id": 49938096, "url": "https://news.ycombinator.com/item?id=49938096", "by": "roosterIllusi0n", "date": "2026-10-02", "stance": "warn", "quote": "The gemma models were terrible and fell apart during compaction.", "excerpt": "I am using qwen3.8-27B-UD-Q3_K_XL on a 5080 16gb card with 16gb of system ram at around 43-45tok/s with 65k context. I am using that model to set up containers on a proxmox server with two b70s and 96gb ram. The Q3 model implemented 10 different chat models, 2 image models, and 7 web based harnesses so I can compare. It can rewrite any part to improve it. qwen3.8 is more than capable for software dev. The gemma model", "context": "", "use_case": "local coding", "switched_from": "", "story": "Several vulnerabilities have been discovered in the Linux kernel"}, {"id": 789487265, "item_id": 49342954, "url": "https://news.ycombinator.com/item?id=49342954", "by": "roosterIllusi0n", "date": "2026-08-18", "stance": "warn", "quote": "My issue with Gemma 4 is that any task fails to complete after any compaction event.", "excerpt": "My issue with Gemma 4 is that any task fails to complete after any compaction event. It often ends up in a loop that keeps compacting and showing the same compaction output. Qwen3.8-27B-IQ4_XS was a massive improvement. It's tasks survive compaction and actually get completed. I switched to Qwen3.8-27B-UD-Q3_K_XL for better performance and its working just as well. Gemma4 screwed up a proxmox install I had. I booted ", "context": "", "use_case": "local agentic coding", "switched_from": "", "story": "Qwen 3.8 27B is excellent, but it defaults to overthinking things"}, {"id": 768523264, "item_id": 48032704, "url": "https://news.ycombinator.com/item?id=48032704", "by": "gertlabs", "date": "2026-05-05", "stance": "recommend", "quote": "Gemma 4 26B writes some really efficient code, especially considering the size.", "excerpt": "This is true, we have the numbers to back it up on https://gertlabs.com/rankings?mode=oneshot_coding (check out the efficiency chart too) GPT 5.5/5.4 are the smartest models, but at great token / code bloat cost. Qwen 3.6 Max strikes a good balance. But Gemma 4 26B writes some really efficient code, especially considering the size. Things do start falling apart under higher contexts.", "context": "", "use_case": "efficient code generation", "switched_from": "", "story": "Accelerating Gemma 4: faster inference with multi-token prediction drafters"}]}, {"label": "Tool calling", "summary": "Some found it struggles with tool calling and agentic coding setups.", "quotes": [{"id": 785714240, "item_id": 49107140, "url": "https://news.ycombinator.com/item?id=49107140", "by": "sznio", "date": "2026-07-30", "stance": "warn", "quote": "Gemma feels nice to write with but every time I use it for coding it struggles with tool calling significantly.", "excerpt": "Nice. Gemma feels nice to write with but every time I use it for coding it struggles with tool calling significantly.", "context": "", "use_case": "writing and coding", "switched_from": "", "story": "Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac"}, {"id": 767455153, "item_id": 47965947, "url": "https://news.ycombinator.com/item?id=47965947", "by": "basscodes", "date": "2026-04-30", "stance": "warn", "quote": "I've been struggling to get Gemma 4 to run with it reliably", "excerpt": "Hey everyone! At eno we've been using local models in our evals to benchmark our harness and with the recent releases of some of these open-weight models like Qwen3.6 we've found that they are only a couple points off in accuracy compared to the cloud SOTA models we use. We thought that was pretty incredible, so we decided to expose the provider configuration to users so they can bring their local models to use with ", "context": "", "use_case": "local agentic search", "switched_from": "", "story": "Running Local Agentic PDF Search with Eno"}, {"id": 774704753, "item_id": 48419047, "url": "https://news.ycombinator.com/item?id=48419047", "by": "satvikpendem", "date": "2026-06-05", "stance": "recommend", "quote": "I use it for recommendations rather than knowledge, like recipes or basic stuff", "excerpt": "I use it for recommendations rather than knowledge, like recipes or basic stuff like that rather than knowledge, I mean it's likely due to its knowledge cutoff so it's not necessarily accurate. But the agent skills section does have a query Wikipedia tool call. Try this on Unsloth Studio, they seem to have fixed Gemma tool calling.", "context": "", "use_case": "recipes and recommendations", "switched_from": "", "story": "Gemma 4 QAT models: Optimizing compression for mobile and laptop efficiency"}]}, {"label": "Below frontier", "summary": "Users say it still trails large frontier models like Opus or DeepSeek V4 Pro.", "quotes": [{"id": 777220464, "item_id": 48576279, "url": "https://news.ycombinator.com/item?id=48576279", "by": "dryarzeg", "date": "2026-06-17", "stance": "recommend", "quote": "I love Gemma, especially the latest Gemma 4 release", "excerpt": "I love Gemma, especially the latest Gemma 4 release - it's really great to see at least somewhat capable, not completely useless model that one can easily host on their own hardware without significant CAPEX investments first - but, to be honest, it doesn't quite compare to GLM 5.2, Kimi K2.6 or DeepSeek V4 Pro. Again, Gemma is amazing, especially for pet projects, but it's not something at least relatively near to \"", "context": "", "use_case": "self-hosted model for pet projects", "switched_from": "", "story": "US holds off blacklisting China's DeepSeek, +100 firms deemed security risks"}, {"id": 786514896, "item_id": 49157181, "url": "https://news.ycombinator.com/item?id=49157181", "by": "nextblock", "date": "2026-08-03", "stance": "recommend", "quote": "I use Gemma 4 right now on a Debian Linux with no GUI that I access remotely. Its works pretty well", "excerpt": "I have a 5080 Super RTX but its still not enough to run these big models. I use Gemma 4 right now on a Debian Linux with no GUI that I access remotely. Its works pretty well, but still not as good as any of the big models like Opus 5... Can't wait to be able to buy a personal home server that can run much bigger models. I heard some companies have started building AI PCs only used to have a personal AI model at home,", "context": "has a 5080 Super RTX GPU", "use_case": "local LLM on home server", "switched_from": "", "story": "Qwen3.8-Max: A New Bar for Coding and Cowork"}, {"id": 765908560, "item_id": 47869285, "url": "https://news.ycombinator.com/item?id=47869285", "by": "finnjohnsen2", "date": "2026-04-22", "stance": "recommend", "quote": "Since Gemma 4 came this easter the gap from self hosting models to Claude has decreased sigificantly", "excerpt": "Since Gemma 4 came this easter the gap from self hosting models to Claude has decreased sigificantly I think. The gap is still huge it just that local models were extremely non-competitive before easter. So now it seems Qwen 3.6 is another bump up from Gemma 4 which is exciting if it is so. I keep an Opus close ofcourse, because these local models still wander off in the wrong direction and fails. Something Opus almo", "context": "self-hosts models on RTX 5090", "use_case": "self-hosted coding model", "switched_from": "", "story": "Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model"}]}, {"label": "Template sensitivity", "summary": "The 26B A4B variant can get stuck in loops; one user advises using the original chat templates.", "quotes": [{"id": 788930480, "item_id": 49308155, "url": "https://news.ycombinator.com/item?id=49308155", "by": "CMay", "date": "2026-08-14", "stance": "recommend", "quote": "Gemma 4 12B, Gemma 4 12B QAT, Gemma 4 31B, Gemma 4 31B QAT", "excerpt": "Gemma 4 12B, Gemma 4 12B QAT, Gemma 4 31B, Gemma 4 31B QAT Gemma 4 26BA4B would get close, but not quite and sometimes even get stuck in loops despite a repeat penalty. Do not use any newer updated templates or Unsloth fixes. Use older official templates that released with the models on the huggingface repo. The template here worked: https://huggingface.co/google/gemma-4-12B-it/tree/657684fef0b5ac5d6bff39284ceb6ec371", "context": "", "use_case": "local model for private benchmarks", "switched_from": "", "story": "Qwen 3.8 27B is out: open weights, best local dense model yet"}]}, {"label": "Prompt injection", "summary": "A commenter flags a special-token prompt injection issue affecting Gemma 4 tooling.", "quotes": [{"id": 787261122, "item_id": 49203820, "url": "https://news.ycombinator.com/item?id=49203820", "by": "pavelai", "date": "2026-08-06", "stance": "warn", "quote": "For Gemma family it is <|turn>", "excerpt": "Prompt injection allows third-party to inject a system prompt with simple message, by inserting special HTML-like sequence (details below). Some of the issues are well-known and pretty old (almost 2 years for Transformers library) Issues: \u2022 Ollama: https://github.com/ollama/ollama/issues/15931 \u2022 HuggingFace: https://github.com/huggingface/transformers/issues/29279 (labeled as feature request) https://github.com/huggi", "context": "", "use_case": "", "switched_from": "", "story": "Prompt Injection Vulnerability in Ollama, Gemma4 and HuggingFace's Transformers"}]}], "who": [{"by": "SwellJoe", "context": "Runs local models on ROCm and Apple Silicon", "url": "https://news.ycombinator.com/item?id=49281788", "date": "2026-08-12"}, {"by": "BoredomIsFun", "context": "writing hobbyist", "url": "https://news.ycombinator.com/item?id=49255113", "date": "2026-08-11"}, {"by": "dofm", "context": "photographer", "url": "https://news.ycombinator.com/item?id=49254713", "date": "2026-08-11"}, {"by": "nextblock", "context": "has a 5080 Super RTX GPU", "url": "https://news.ycombinator.com/item?id=49157181", "date": "2026-08-03"}, {"by": "efficax", "context": "building a custom personal agent", "url": "https://news.ycombinator.com/item?id=48993070", "date": "2026-07-21"}, {"by": "esalman", "context": "two decades in the industry", "url": "https://news.ycombinator.com/item?id=48954980", "date": "2026-07-17"}, {"by": "kamranjon", "context": "runs local models on Apple Silicon", "url": "https://news.ycombinator.com/item?id=48677006", "date": "2026-06-25"}, {"by": "xlii", "context": "has gaming rig with powerful GPU", "url": "https://news.ycombinator.com/item?id=48567220", "date": "2026-06-17"}, {"by": "nikagrawal121", "context": "building a legal AI application", "url": "https://news.ycombinator.com/item?id=48560369", "date": "2026-06-16"}, {"by": "bachmeier", "context": "64 GB RAM, no GPU", "url": "https://news.ycombinator.com/item?id=48520024", "date": "2026-06-13"}, {"by": "c0rruptbytes", "context": "M4 Pro 48GB user", "url": "https://news.ycombinator.com/item?id=48514435", "date": "2026-06-13"}, {"by": "jongjong", "context": "software engineer", "url": "https://news.ycombinator.com/item?id=48513017", "date": "2026-06-12"}], "tags": ["self-hostable", "for Mac users", "for creative writing", "for consumer GPUs", "for private data"], "use_cases": ["local llm", "local coding model", "local coding", "local inference", "local models", "base model for decision model"], "compared": [{"slug": "qwen", "name": "Qwen", "co": 23, "switched_from": 0, "switched_to": 0}, {"slug": "qwen-3-6", "name": "Qwen 3.6", "co": 19, "switched_from": 0, "switched_to": 0}, {"slug": "claude", "name": "Claude", "co": 15, "switched_from": 1, "switched_to": 0}, {"slug": "ollama", "name": "Ollama", "co": 15, "switched_from": 0, "switched_to": 0}, {"slug": "llama-cpp", "name": "llama.cpp", "co": 11, "switched_from": 0, "switched_to": 0}, {"slug": "qwen-3-8", "name": "Qwen 3.8", "co": 7, "switched_from": 0, "switched_to": 1}, {"slug": "deepseek", "name": "DeepSeek", "co": 8, "switched_from": 0, "switched_to": 0}, {"slug": "qwen-3-5", "name": "Qwen 3.5", "co": 4, "switched_from": 2, "switched_to": 0}], "makers": [{"id": 769874448, "item_id": 48117153, "url": "https://news.ycombinator.com/item?id=48117153", "by": "fomoz", "date": "2026-05-12", "stance": "recommend", "quote": "We've been running bots exclusively on Gemma 4 31B and 26B for the last 7-10 days and they've been either breaking even or trading very well", "excerpt": "You can check in detail on the leaderboard, but I suggest you look at the last 24h to 7 days. We've been playing with different settings and models since January, older wallets have had some bleed with Gemini 3 Flash and bot settings that didn't perform well. We've been running bots exclusively on Gemma 4 31B and 26B for the last 7-10 days and they've been either breaking even or trading very well, it really depends ", "context": "runs a crypto trading bot platform", "use_case": "crypto trading bots", "switched_from": "Gemini 3 Flash", "story": "Ask HN: What Are You Working On? (May 2026)"}], "threads": [{"url": "https://news.ycombinator.com/item?id=48555993", "title": "Running local models is good now", "mentions": 6}, {"url": "https://news.ycombinator.com/item?id=47616361", "title": "Google releases Gemma 4 open models", "mentions": 5}, {"url": "https://news.ycombinator.com/item?id=49241679", "title": "Meta Muse Glimmer \u2013 open weights 30B local coding model", "mentions": 4}, {"url": "https://news.ycombinator.com/item?id=49098510", "title": "Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac", "mentions": 3}, {"url": "https://news.ycombinator.com/item?id=48542100", "title": "Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?", "mentions": 3}, {"url": "https://news.ycombinator.com/item?id=48536531", "title": "Ask HN: What does your local LLM setup looks like?", "mentions": 3}], "sentiment_by_month": {"2025-11": {"neutral": 1}, "2025-12": {}, "2026-01": {}, "2026-02": {"recommend": 1}, "2026-03": {"neutral": 1}, "2026-04": {"recommend": 11, "neutral": 4, "warn": 1}, "2026-05": {"neutral": 3, "recommend": 3}, "2026-06": {"recommend": 19, "neutral": 13, "compare": 2}, "2026-07": {"recommend": 13, "neutral": 6, "warn": 1}, "2026-08": {"neutral": 3, "recommend": 11, "warn": 2, "compare": 1}, "2026-09": {"recommend": 5, "neutral": 4}, "2026-10": {"neutral": 2, "warn": 1, "recommend": 2}}, "quotes": [{"id": 803159040, "item_id": 50197440, "url": "https://news.ycombinator.com/item?id=50197440", "by": "cj00", "date": "2026-10-22", "stance": "neutral", "quote": "runs them through a local Gemma model to write a paragraph description and OCR any text in the thumbnail", "excerpt": "I used the same concept to build a searchable database of YouTube thumbnails in my watch history. I always end up trying to remember a video I didn't finish from a week or two ago and YouTube's history search plain sucks. So I built an app that tracks all my watch history across all devices, downloads the thumbnails and runs them through a local Gemma model to write a paragraph description and OCR any text in the thu", "context": "", "use_case": "describing thumbnails, OCR", "switched_from": "", "story": "Semantic search across major museums"}, {"id": 801925937, "item_id": 50120371, "url": "https://news.ycombinator.com/item?id=50120371", "by": "cj00", "date": "2026-10-16", "stance": "recommend", "quote": "I switched to Gemma 4 instead.", "excerpt": "I've found some strange guardrails with the Foundation models. Like blocking describing of any photo with a single woman in it. It does this relatively consistently but with a different subset every time. Like out of 10 photos in the set, it's skips 6 then the next run, it skips 3. Have you seen guardrail issues like this? I switched to Gemma 4 instead.", "context": "", "use_case": "photo description", "switched_from": "Apple Foundation Models", "story": "Apple ML Research: Muse Spark Open with MLX on Mac Studio"}, {"id": 800661297, "item_id": 50041331, "url": "https://news.ycombinator.com/item?id=50041331", "by": "onatm", "date": "2026-10-11", "stance": "neutral", "quote": "Gemma4 based one is on a branch", "excerpt": "Nice read! I also gave a shot building one on Gemma3 and Gemma4. It was a fun exercise. I\u2019m sharing it if anyone interested, Gemma4 based one is on a branch: https://github.com/onatm/gev", "context": "", "use_case": "base model for decision model", "switched_from": "", "story": "Build your own decision model"}, {"id": 800661298, "item_id": 50041331, "url": "https://news.ycombinator.com/item?id=50041331", "by": "onatm", "date": "2026-10-11", "stance": "neutral", "quote": "building one on Gemma3 and Gemma4", "excerpt": "Nice read! I also gave a shot building one on Gemma3 and Gemma4. It was a fun exercise. I\u2019m sharing it if anyone interested, Gemma4 based one is on a branch: https://github.com/onatm/gev", "context": "", "use_case": "base model for decision model", "switched_from": "", "story": "Build your own decision model"}, {"id": 800421665, "item_id": 50026354, "url": "https://news.ycombinator.com/item?id=50026354", "by": "denverllc", "date": "2026-10-09", "stance": "recommend", "quote": "found a small classifier (gemma 4) with traditional embeddings worked 2x as well.", "excerpt": "> but I question how much folks are actually experimenting with what's out there I did. Originally I had a project that I had been wanting to do and thought to use a decision model for it. Jev, OpenAI, etc. are all within percentage points of each other. Then I used traditional ML and found a small classifier (gemma 4) with traditional embeddings worked 2x as well. Jev is the general purpose ML pipeline for when you ", "context": "", "use_case": "classification", "switched_from": "Jev", "story": "Typesafe AI raises $870M at $7.5B"}, {"id": 799009537, "item_id": 49938096, "url": "https://news.ycombinator.com/item?id=49938096", "by": "roosterIllusi0n", "date": "2026-10-02", "stance": "warn", "quote": "The gemma models were terrible and fell apart during compaction.", "excerpt": "I am using qwen3.8-27B-UD-Q3_K_XL on a 5080 16gb card with 16gb of system ram at around 43-45tok/s with 65k context. I am using that model to set up containers on a proxmox server with two b70s and 96gb ram. The Q3 model implemented 10 different chat models, 2 image models, and 7 web based harnesses so I can compare. It can rewrite any part to improve it. qwen3.8 is more than capable for software dev. The gemma model", "context": "", "use_case": "local coding", "switched_from": "", "story": "Several vulnerabilities have been discovered in the Linux kernel"}, {"id": 799009249, "item_id": 49938078, "url": "https://news.ycombinator.com/item?id=49938078", "by": "simoiacos", "date": "2026-10-02", "stance": "neutral", "quote": "The only model supported right now is a quantized Gemma-4", "excerpt": "Nothing comparable but inspired from DwarfStar I wrote a little inference engine for Intel Xe-LP (no XMX) 32GB laptops. The only model supported right now is a quantized Gemma-4, but I don't exclude in the future to support other MoE of similar size. Too bad we have no Qwen 3.8 35B-A3B yet. I'm also looking into expanding the protocol and the engine to support various steering techniques. https://github.com/simoneiac", "context": "", "use_case": "local inference", "switched_from": "", "story": "From the creator of Redis; run LLM locally with ds4"}, {"id": 798591969, "item_id": 49911998, "url": "https://news.ycombinator.com/item?id=49911998", "by": "jeff_ciesielski", "date": "2026-09-30", "stance": "recommend", "quote": "I settled on gemma-4-26B-A4B as my daily driver.", "excerpt": "Howdy folks. I was inspired by the really neat \"Jev in 25 lines of Python\" post a few days ago (and System One models in general), and after playing with the idea I tried whipping up something a little more complex, and Lichen is what popped out: https://github.com/Mushroom-Systems/lichen My question after reading that post (and I know I wasn't the only one; antirez came right out and said it in the comments) was whe", "context": "", "use_case": "daily driver local model", "switched_from": "", "story": "Lichen \u2013 A local, BOY model system1 (Jev) server with image support"}, {"id": 797158384, "item_id": 49822399, "url": "https://news.ycombinator.com/item?id=49822399", "by": "rcarmo", "date": "2026-09-23", "stance": "recommend", "quote": "Seems less prone to getting distracted with ordering.", "excerpt": "Yah, that's what I use: https://rcarmo.github.io/projects/go-system-one uses Gemma, and that's partly why. Seems less prone to getting distracted with ordering.", "context": "", "use_case": "email classification", "switched_from": "", "story": "Jev in 25 Lines of Python"}, {"id": 795919520, "item_id": 49744970, "url": "https://news.ycombinator.com/item?id=49744970", "by": "gentile", "date": "2026-09-17", "stance": "neutral", "quote": "I used to run gemma 4 e4b-it-qat fully in GPU (~150 t/s)", "excerpt": "It's still typically for code, like python data stuff, bash scripts, general web search, small javascript stuff for my website. By saying not SWE, I mean I don't really see much benefit from \"agentic\" stuff, although I've tried. 50 t/s means 40~60 seconds for a typical thinking response. I used to run gemma 4 e4b-it-qat fully in GPU (~150 t/s), but the quality improvement moving to a much larger MoE model was 100% wo", "context": "", "use_case": "local chat and coding help", "switched_from": "", "story": "Show HN: Share your AI Setup, Learn from others"}, {"id": 795911667, "item_id": 49744479, "url": "https://news.ycombinator.com/item?id=49744479", "by": "selicos", "date": "2026-09-17", "stance": "neutral", "quote": "Running Gemma e4b, Gemma 4 26b a4b IT", "excerpt": "So I have to connect over MCP or log into github through the page to share anything? How about being able to post a markdown summary of my setup instead of plugging my credentials and tools into a third party site? Strix HALO 128gb - Framework mainboard in a custom SFF PC. Just moved from Ubuntu to Fedora 44. Using: LM Studio (primary), Lemonade, not yet got into vLLM and llama.cpp directyly after moving to Fedora. R", "context": "", "use_case": "local inference", "switched_from": "", "story": "Show HN: Share your AI Setup, Learn from others"}, {"id": 794782035, "item_id": 49673877, "url": "https://news.ycombinator.com/item?id=49673877", "by": "VariousPrograms", "date": "2026-09-12", "stance": "neutral", "quote": "Locally I use Deepseek Flash, Qwen3.8-Next-Flash, or Gemma 4", "excerpt": "GLM 5.3 Flash for personal hobby stuff. It's the first local-ish model that feels capable enough to be a default model to me (and it's very cheap). I'd rather get addicted to an open weight LLM in a class I can theoretically run if consumers ever get access to RAM again and can't get taken away from me. Locally I use Deepseek Flash, Qwen3.8-Next-Flash, or Gemma 4 depending on how slowly I want my slop to generate.", "context": "", "use_case": "local models", "switched_from": "", "story": "Ask HN: What default model do you use and why?"}, {"id": 794058497, "item_id": 49628656, "url": "https://news.ycombinator.com/item?id=49628656", "by": "tmzt", "date": "2026-09-09", "stance": "neutral", "quote": "using Qwen and Gemma models", "excerpt": "I've been experimenting with using J-Space (and final hidden layers) to extract the semantic meaning of words to improve TTS output accuracy using Qwen and Gemma models. My goal is to either map to an alternative token set where heteronyms are preserved or to output parenthesized IPA annotations for ambiguous words (with standard tokens). It's interesting to me that LLMs preserve this data throughout their processing", "context": "", "use_case": "improving TTS output accuracy", "switched_from": "", "story": "Extracting Steering Vectors from J space"}, {"id": 793899297, "item_id": 49618706, "url": "https://news.ycombinator.com/item?id=49618706", "by": "nowittyusername", "date": "2026-09-08", "stance": "recommend", "quote": "Gemma models still outperform it in that department.", "excerpt": "I used it for testing my voice agent. It was basically what I expected. Good fast model but \"generic\" or \"vanilla\" is how i would describe its personality emulation capability as. Gemma models still outperform it in that department. As far as technicals, one thing i found annoying is cash use was not that good, it missed more then i liked, i contacted support and they were fast and responsive and said they were worki", "context": "", "use_case": "voice agent personality emulation", "switched_from": "", "story": "Mercury 2.5"}, {"id": 793822033, "item_id": 49613877, "url": "https://news.ycombinator.com/item?id=49613877", "by": "dofm", "date": "2026-09-08", "stance": "recommend", "quote": "I absolutely don't hate the way Gemma 4 writes", "excerpt": "I don't think so, no, because it extends to LLM-generated text I want to recall. I use LLMs to generate starter/tutorial material. I may hate the way Claude writes but I absolutely don't hate the way Gemma 4 writes. But I have to continuously consult it in a way that I do not with human-written text, which gets its message across in a more persistent way I find less troubling. (This non-memorability extends to AI ima", "context": "", "use_case": "generating tutorial material", "switched_from": "", "story": "Benchmarking Qwen3.8 27B quantizations: 4-bit holds up, 1-bit collapses"}, {"id": 792546000, "item_id": 49534125, "url": "https://news.ycombinator.com/item?id=49534125", "by": "ch_sm", "date": "2026-09-02", "stance": "recommend", "quote": "the smaller Gemma 4 models or qwen3.6:9b would probably run OK on that", "excerpt": "It depends on your use case, but the smaller Gemma 4 models or qwen3.6:9b would probably run OK on that. I recommend trying it, even just for fun. It\u2018s easy with omlx.", "context": "", "use_case": "local model on 24GB Mac", "switched_from": "", "story": "My local model setup on an M4 Pro Mac Mini"}, {"id": 791889698, "item_id": 49493106, "url": "https://news.ycombinator.com/item?id=49493106", "by": "kouteiheika", "date": "2026-08-29", "stance": "neutral", "quote": "I run Gemma-4 on an RTX 6000", "excerpt": "I love vLLM, but damn if it isn't frustratingly buggy. I was recently running DeepSeek-V4-Flash on a B300. On v0.26 it was totally broken, and I had to add three out-of-tree patches to fix it. I updated to v0.27 -- no patches necessary now, but the output is now broken as it randomly starts responding with garbage (repeated token loops). On my workstation where I run Gemma-4 on an RTX 6000 the whole process tends to ", "context": "", "use_case": "self-hosted inference", "switched_from": "", "story": "vLLM v0.28.0"}, {"id": 791287264, "item_id": 49455454, "url": "https://news.ycombinator.com/item?id=49455454", "by": "halyconWays", "date": "2026-08-26", "stance": "recommend", "quote": "I love all these open weight models", "excerpt": "Between Gemma 31/26/12/4/2, Deepseek-v4-flash-0731, Qwen 3.8 27B, Qwen 3.8 Flash Next (which I haven't even gotten to run yet!), and now GLM 5.3 Flash, I can't keep up. I love all these open weight models and am continually stunned that it's largely the West fighting for closed, restrictive, anti-user bullshit and China absolutely mogging the likes of OpenAI and Anthropic, with some notable exceptions like Gemma. Sti", "context": "", "use_case": "local open-weight models", "switched_from": "", "story": "GLM-5.3-Flash"}, {"id": 791115043, "item_id": 49444690, "url": "https://news.ycombinator.com/item?id=49444690", "by": "SturgeonsLaw", "date": "2026-08-26", "stance": "recommend", "quote": "I use Qwen/Claude for coding and Gemini/Gemma to draft text intended for a human.", "excerpt": "I use Qwen/Claude for coding and Gemini/Gemma to draft text intended for a human. To me that feels like each model playing to their strengths.", "context": "", "use_case": "drafting human-facing text", "switched_from": "", "story": "Analyzing student votes across AI models for college essay help"}, {"id": 790939040, "item_id": 49433690, "url": "https://news.ycombinator.com/item?id=49433690", "by": "super_mario", "date": "2026-08-25", "stance": "recommend", "quote": "I get 100 tokens/s with Gemma 4 27b model", "excerpt": "I was in the same situation, I used maxed out 15'' M3 Max MacBook Pro docked to Studio Display closed on vertical stand behind the screen. It was fine for office work, but running local LLMs would definitely overheat it. The battery started degrading purely due to heat issues. And it was audible as well. I decided to get Mac Studio M4 Max, also all maxed out config and the cooling is so much better that I can run loc", "context": "", "use_case": "local LLM inference", "switched_from": "", "story": "New Mac Studio with M5 Max and M5 Ultra"}, {"id": 789487265, "item_id": 49342954, "url": "https://news.ycombinator.com/item?id=49342954", "by": "roosterIllusi0n", "date": "2026-08-18", "stance": "warn", "quote": "My issue with Gemma 4 is that any task fails to complete after any compaction event.", "excerpt": "My issue with Gemma 4 is that any task fails to complete after any compaction event. It often ends up in a loop that keeps compacting and showing the same compaction output. Qwen3.8-27B-IQ4_XS was a massive improvement. It's tasks survive compaction and actually get completed. I switched to Qwen3.8-27B-UD-Q3_K_XL for better performance and its working just as well. Gemma4 screwed up a proxmox install I had. I booted ", "context": "", "use_case": "local agentic coding", "switched_from": "", "story": "Qwen 3.8 27B is excellent, but it defaults to overthinking things"}, {"id": 789304928, "item_id": 49331558, "url": "https://news.ycombinator.com/item?id=49331558", "by": "refsab", "date": "2026-08-17", "stance": "neutral", "quote": "I used gemma4 I think", "excerpt": "Context window depends on the model used. I used gemma4 I think and set it to 32k so startup is faster. I think even gemma supports up to 200k", "context": "", "use_case": "local model for fiction writing", "switched_from": "", "story": "Show HN: 1667, a terminal UI for writing fiction with language models"}, {"id": 788961666, "item_id": 49310104, "url": "https://news.ycombinator.com/item?id=49310104", "by": "scirob", "date": "2026-08-15", "stance": "compare", "quote": "German language has never been a big focus for asian models but they still outperform Gemma models", "excerpt": "We maintain German Langauge index as no one publishes or reruns these sepeartly. Qwen 3.8 27B is a small improvement with some regressions in our benchmarks not a huge jump like benchmarks listed. https://dach.peerbench.ai/compare?models=qwen%2Fqwen3.8-27b,qwen%2Fqwen3.6-27b German language has never been a big focus for asian models but they still outperform Gemma models https://dach.peerbench.ai/compare?models=open", "context": "", "use_case": "German language", "switched_from": "", "story": "Qwen 3.8 27B is out: open weights, best local dense model yet"}, {"id": 788930480, "item_id": 49308155, "url": "https://news.ycombinator.com/item?id=49308155", "by": "CMay", "date": "2026-08-14", "stance": "recommend", "quote": "Gemma 4 12B, Gemma 4 12B QAT, Gemma 4 31B, Gemma 4 31B QAT", "excerpt": "Gemma 4 12B, Gemma 4 12B QAT, Gemma 4 31B, Gemma 4 31B QAT Gemma 4 26BA4B would get close, but not quite and sometimes even get stuck in loops despite a repeat penalty. Do not use any newer updated templates or Unsloth fixes. Use older official templates that released with the models on the huggingface repo. The template here worked: https://huggingface.co/google/gemma-4-12B-it/tree/657684fef0b5ac5d6bff39284ceb6ec371", "context": "", "use_case": "local model for private benchmarks", "switched_from": "", "story": "Qwen 3.8 27B is out: open weights, best local dense model yet"}, {"id": 788508609, "item_id": 49281788, "url": "https://news.ycombinator.com/item?id=49281788", "by": "SwellJoe", "date": "2026-08-12", "stance": "recommend", "quote": "The reason I use Gemma 4 so much often comes down to how reliable it is", "excerpt": "Counterpoint: I've been using the Unsloth Gemma 4 quants extensively since very soon after release (on ROCm and Apple Silicon, I don't have any Nvidia hardware big enough), pretty much every quantization down to 4 bits (the QAT is the business, indistinguishable from the full-fat version, runs great on a slightly chonky desktop or laptop), and I haven't had any issues. The reason I use Gemma 4 so much often comes dow", "context": "Runs local models on ROCm and Apple Silicon", "use_case": "local model experiments", "switched_from": "", "story": "Qwen/Qwen3.8-2.4T-A95B"}, {"id": 788272241, "item_id": 49267015, "url": "https://news.ycombinator.com/item?id=49267015", "by": "esalman", "date": "2026-08-11", "stance": "recommend", "quote": "increasingly using opencode+Gemma 4, and developing some system prompts to iteratively improve the results", "excerpt": "I was skeptical at first but these days I use Gemini on Android phone and Android Auto, search AI summary, and lately Gemma 4 a lot. Skepticism is still needed though. The other day I asked Gemini while driving if there's a HoV lane going from 105 E to 110 S. It said yes. Actually there one to 110 N but not S. I did not act upon it because my co-passenger verified that Gemini was wrong. Search summary is useful but n", "context": "", "use_case": "local coding assistance", "switched_from": "Claude", "story": "Gemini becomes Google's fastest-growing product ever as it hits 1B users"}, {"id": 788081808, "item_id": 49255113, "url": "https://news.ycombinator.com/item?id=49255113", "by": "BoredomIsFun", "date": "2026-08-11", "stance": "recommend", "quote": "Exactly why Gemma 4 are so popular.", "excerpt": "> I suppose for dialogue generation in games? I use it to write short sci-fi stories. Life is not only about being an SDE. > Why on Earth would you ever want to write code with a model that is supposedly \"jailbroken\"? I need a model which decent enough at coding to write boilerplate for me, yet I do not need to unload in case I want to write an article or enguage in my writing hobby.Exactly why Gemma 4 are so popular", "context": "writing hobbyist", "use_case": "boilerplate coding and writing", "switched_from": "", "story": "Meta Muse Glimmer \u2013 open weights 30B local coding model"}, {"id": 788075408, "item_id": 49254713, "url": "https://news.ycombinator.com/item?id=49254713", "by": "dofm", "date": "2026-08-11", "stance": "recommend", "quote": "LLMs (including Gemma-4 12B with its built-in image encoder) understands a lot of my references and influences", "excerpt": "It might be an indication that analysis of photography is something of an ideal discipline for an LLM since so much content online involves discussion of pictures. A lot of what I am trying to do with my photography is sort of meta-photography. I am really interested in early photographic history, pictorialism and its opponents etc., but I try to avoid reproduction, so I try not to simulate processes too closely or t", "context": "photographer", "use_case": "photography analysis", "switched_from": "", "story": "Meta Muse Glimmer \u2013 open weights 30B local coding model"}, {"id": 787977650, "item_id": 49248603, "url": "https://news.ycombinator.com/item?id=49248603", "by": "dofm", "date": "2026-08-10", "stance": "recommend", "quote": "Again, I really like both the Qwen and Gemma MoEs", "excerpt": "One thing I would caution is staying out of the prediction market like this. Tech tends to get boring when you judge current products against the hypothetical capabilities of unannounced products that may never ship. It's like comparing Nikon cameras against Canon camera rumours, or comparing iPhones against unannounced and therefore largely imaginary Samsungs. - If they do a Qwen 3.8 35B A3B (and I hope they do beca", "context": "", "use_case": "local models", "switched_from": "", "story": "Meta Muse Glimmer \u2013 open weights 30B local coding model"}, {"id": 787913168, "item_id": 49244573, "url": "https://news.ycombinator.com/item?id=49244573", "by": "jakswa", "date": "2026-08-10", "stance": "recommend", "quote": "you should check out Gemma 4 for creative DMing use case", "excerpt": "I like the tabletop RPG use case, and wanted to say: If your hardware likes it you should check out Gemma 4 for creative DMing use case. I found it to be much better at holding the plotlines and being creative on gaming turns. My experimental case was an audio-only Zork and Gemma 12B and even E4B were pretty good!", "context": "", "use_case": "creative tabletop RPG DMing", "switched_from": "", "story": "Meta Muse Glimmer \u2013 open weights 30B local coding model"}, {"id": 787288388, "item_id": 49205524, "url": "https://news.ycombinator.com/item?id=49205524", "by": "alasr", "date": "2026-08-06", "stance": "recommend", "quote": "start with Qwen3.6, Google's Gemma4, OpenAI's gpt-oss-20b or Nvidia's Nemotron", "excerpt": "- See \"Llama.cpp vs Ollama\"[0] and \"Llama.cpp vs LM Studio\"[1] for a high-level features comparison and find out which one suite you best (check out the \"Target Users\" category[0][1], at least). - If you're still not sure, \"LM Studio\"[2] is not bad to start with as you'll be able to download, start/stop/manage and chat with your LLM model all in one place: a single desktop app. Also, once installed, enable \"Developer", "context": "", "use_case": "local LLM", "switched_from": "", "story": "Qwen3.8-Max: A New Bar for Coding and Cowork"}, {"id": 787261122, "item_id": 49203820, "url": "https://news.ycombinator.com/item?id=49203820", "by": "pavelai", "date": "2026-08-06", "stance": "warn", "quote": "For Gemma family it is <|turn>", "excerpt": "Prompt injection allows third-party to inject a system prompt with simple message, by inserting special HTML-like sequence (details below). Some of the issues are well-known and pretty old (almost 2 years for Transformers library) Issues: \u2022 Ollama: https://github.com/ollama/ollama/issues/15931 \u2022 HuggingFace: https://github.com/huggingface/transformers/issues/29279 (labeled as feature request) https://github.com/huggi", "context": "", "use_case": "", "switched_from": "", "story": "Prompt Injection Vulnerability in Ollama, Gemma4 and HuggingFace's Transformers"}, {"id": 786514896, "item_id": 49157181, "url": "https://news.ycombinator.com/item?id=49157181", "by": "nextblock", "date": "2026-08-03", "stance": "recommend", "quote": "I use Gemma 4 right now on a Debian Linux with no GUI that I access remotely. Its works pretty well", "excerpt": "I have a 5080 Super RTX but its still not enough to run these big models. I use Gemma 4 right now on a Debian Linux with no GUI that I access remotely. Its works pretty well, but still not as good as any of the big models like Opus 5... Can't wait to be able to buy a personal home server that can run much bigger models. I heard some companies have started building AI PCs only used to have a personal AI model at home,", "context": "has a 5080 Super RTX GPU", "use_case": "local LLM on home server", "switched_from": "", "story": "Qwen3.8-Max: A New Bar for Coding and Cowork"}, {"id": 786309200, "item_id": 49144325, "url": "https://news.ycombinator.com/item?id=49144325", "by": "sieve", "date": "2026-08-02", "stance": "neutral", "quote": "Let's me switch between any model including Gemma 4 running locally.", "excerpt": "I tried the programmer's method. Automating stuff. Then I realized I prefer the wood carver's method. So I take it slow. I think and write the outline for a chapter. Then I have the LLM produce it. I read the draft. Tweak things. Reroll. Take the best from multiple rolls and merge them to produce a organic whole. Manually patch the minor differences in pacing and tone. Rinse and repeat. It takes 100-250 hours across ", "context": "", "use_case": "drafting fiction", "switched_from": "", "story": "Generative AI floods and dilutes the market for books"}, {"id": 785714240, "item_id": 49107140, "url": "https://news.ycombinator.com/item?id=49107140", "by": "sznio", "date": "2026-07-30", "stance": "warn", "quote": "Gemma feels nice to write with but every time I use it for coding it struggles with tool calling significantly.", "excerpt": "Nice. Gemma feels nice to write with but every time I use it for coding it struggles with tool calling significantly.", "context": "", "use_case": "writing and coding", "switched_from": "", "story": "Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac"}, {"id": 785610769, "item_id": 49100673, "url": "https://news.ycombinator.com/item?id=49100673", "by": "y42", "date": "2026-07-29", "stance": "neutral", "quote": "I fine tuned Gemma with domain specific knowledge.", "excerpt": "Dont want to crash the party here, but I am still sceptic about all those on-premise-llm-approaches. I think we strongly need something like that (shameless plug, I tried to build something around bitNet for the same reason: https://github.com/nickyreinert/bitNetRTR). But at the end, all aproaches I saw, however genius they are: the actual results are always a mess. It's a better chat buddy, nothing else. It's e.g. f", "context": "", "use_case": "fine-tuned local model", "switched_from": "", "story": "Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac"}, {"id": 785598288, "item_id": 49099893, "url": "https://news.ycombinator.com/item?id=49099893", "by": "jwr", "date": "2026-07-29", "stance": "recommend", "quote": "Gemma is actually a very good general-purpose model that can work with multiple languages.", "excerpt": "In defense of this model, Gemma is actually a very good general-purpose model that can work with multiple languages. I use it for spam classification and for processing dictation, which means that I hold the entire model in memory all of the time, which is somewhat problematic (64GB RAM total, but heavy usage by docker, databases, etc)", "context": "", "use_case": "spam classification, dictation processing", "switched_from": "", "story": "Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac"}, {"id": 784736672, "item_id": 49046042, "url": "https://news.ycombinator.com/item?id=49046042", "by": "janantos", "date": "2026-07-25", "stance": "neutral", "quote": "near realtime spoken word translation/transcription project using selfhosted Gemma4 and Nvidia Parakeet LLM models", "excerpt": "Tbh, I don\u2019t want to be part of codeberg anymore, their community moved away from my believes. And as I have another projects being according them not welcomed there because they are tied to LLM ecosystem. I have prof of concept of AI coding agent entirely written in bash and also near realtime spoken word translation/transcription project using selfhosted Gemma4 and Nvidia Parakeet LLM models. In their latest blog t", "context": "", "use_case": "self-hosted transcription/translation", "switched_from": "", "story": "Brow6el: A full-featured web browser for the terminal using Chromium"}, {"id": 784341459, "item_id": 49021341, "url": "https://news.ycombinator.com/item?id=49021341", "by": "data-ottawa", "date": "2026-07-23", "stance": "neutral", "quote": "I'm able to use ROCm with MTP just fine (Qwen and Gemma).", "excerpt": "I'm able to use ROCm with MTP just fine (Qwen and Gemma). I use Lemonade and have only the ROCm installed through it (ETA: their llama.cpp version is a fork, not sure what is different vs the main trunk).", "context": "", "use_case": "local LLM inference", "switched_from": "", "story": "New Framework Desktop Option with AMD Ryzen AI Max+ Pro 495 and 192GB Memory"}, {"id": 784234755, "item_id": 49014672, "url": "https://news.ycombinator.com/item?id=49014672", "by": "InTheArena", "date": "2026-07-22", "stance": "neutral", "quote": "gemma is workable", "excerpt": "I have a DGX and a Ryzen AI Max 395 - while I love both of them, there are a few critical things that leave the DGX in use, while the Ryzen \"just\" is my primary homelab server. The biggest thing is prefil numbers, and the performance impact of higher context sizes. Qwen 27b is a great model, nemotron is decent, gemma is workable. But all of them need reasonable context for reasonable outputs. Unfortunitly, as others ", "context": "", "use_case": "local LLM", "switched_from": "", "story": "Nvidia DGX Spark as a daily driver"}, {"id": 783889120, "item_id": 48993070, "url": "https://news.ycombinator.com/item?id=48993070", "by": "efficax", "date": "2026-07-21", "stance": "recommend", "quote": "The local model is too slow in my setup for a chat agent, but for memory extraction it works pretty well", "excerpt": "I use a gemma4 model locally to extract content from messages to a personal agent I'm building for its memory graph (to break the message up into the topic, source (assistant or owner), facts, entities, etc. in the message content (all getting thrown into a magma-esque graph using NLEmbeddings for memory search). This is for a custom personal agent that targets deepseek-v4 flash. The local model is too slow in my set", "context": "building a custom personal agent", "use_case": "local memory extraction for agent", "switched_from": "", "story": "Nativ: Run frontier open models locally on your Mac"}, {"id": 782647040, "item_id": 48915440, "url": "https://news.ycombinator.com/item?id=48915440", "by": "SwellJoe", "date": "2026-07-14", "stance": "recommend", "quote": "Gemma 4 is the best all-rounder among the models I can self-host", "excerpt": "I want 31B. The 12B 4-bit QAT is already small enough to run well enough on every device I use regularly, including phone and tablet, I don't need a 1-bit or ternary version of the 12B. But, what I really want is for Google to release bigger Gemma 4 models, particularly a bigger MoE, like a ~70B or ~120B. Gemma 4 is the best all-rounder among the models I can self-host even though I've got a 128GB Strix Halo. A 4-bit", "context": "has a 128GB Strix Halo", "use_case": "self-hosted local model", "switched_from": "", "story": "Bonsai 27B (1-bit LLM): The First 27B-Class Model to Run on a Phone"}, {"id": 782202705, "item_id": 48887669, "url": "https://news.ycombinator.com/item?id=48887669", "by": "MyMemoryfails", "date": "2026-07-12", "stance": "recommend", "quote": "Its shockingly capable for its size.", "excerpt": "Running Local LLM isn't really a lot effort anymore, since its one command away to get the familar interface of chat AI. This setup requires 24GB ish VRAM and 32GB ram, so if you have capable gaming PC which can do AAA games, you can run it. - llama-server -hf ggml-org/gemma-4-26b-a4b-it-GGUF:Q4_K_M After that simply open browser and enter: http://localhost:8080 What this does: This will download Gemma4 AI with 26B p", "context": "", "use_case": "private local chat", "switched_from": "", "story": "Ask HN: How do you use LLMs for private discussions?"}, {"id": 781616241, "item_id": 48851015, "url": "https://news.ycombinator.com/item?id=48851015", "by": "bevekspldnw", "date": "2026-07-09", "stance": "recommend", "quote": "Gemma4 models are unbeatable at their size.", "excerpt": "Gemini is more than fine as a mobile application, and will be the \u201cbrains\u201d of the currently braindead Siri, between than and Android it\u2019s hard to come up with an argument they are behind. Likewise, Gemma4 models are unbeatable at their size. So I use Gemini for coding? Hell no, but that\u2019s not the same as Google failing writ large. Googles only real goal is to retain the ownership and dominance of the online world the", "context": "", "use_case": "", "switched_from": "", "story": "Introducing Muse Spark 1.1"}, {"id": 774205920, "item_id": 48387870, "url": "https://news.ycombinator.com/item?id=48387870", "by": "quickthoughts", "date": "2026-06-03", "stance": "recommend", "quote": "I use small models like Gemma to improve transcriptions from ASR models amongst other micro-tasks.", "excerpt": "I use small models like Gemma to improve transcriptions from ASR models amongst other micro-tasks. I actually built out a fine-tuning whisper pipeline with all local (smaller) models meaning no cloud/big-tech co is able to train/sell my (private) data. Repo is https://github.com/Rebreda/listenr - mainly geared toward Whisper fine-tuning, AMD hardware and local inference", "context": "", "use_case": "improving ASR transcriptions", "switched_from": "", "story": "Gemma 4 12B: A unified, encoder-free multimodal model"}, {"id": 768523264, "item_id": 48032704, "url": "https://news.ycombinator.com/item?id=48032704", "by": "gertlabs", "date": "2026-05-05", "stance": "recommend", "quote": "Gemma 4 26B writes some really efficient code, especially considering the size.", "excerpt": "This is true, we have the numbers to back it up on https://gertlabs.com/rankings?mode=oneshot_coding (check out the efficiency chart too) GPT 5.5/5.4 are the smartest models, but at great token / code bloat cost. Qwen 3.6 Max strikes a good balance. But Gemma 4 26B writes some really efficient code, especially considering the size. Things do start falling apart under higher contexts.", "context": "", "use_case": "efficient code generation", "switched_from": "", "story": "Accelerating Gemma 4: faster inference with multi-token prediction drafters"}, {"id": 767455153, "item_id": 47965947, "url": "https://news.ycombinator.com/item?id=47965947", "by": "basscodes", "date": "2026-04-30", "stance": "warn", "quote": "I've been struggling to get Gemma 4 to run with it reliably", "excerpt": "Hey everyone! At eno we've been using local models in our evals to benchmark our harness and with the recent releases of some of these open-weight models like Qwen3.6 we've found that they are only a couple points off in accuracy compared to the cloud SOTA models we use. We thought that was pretty incredible, so we decided to expose the provider configuration to users so they can bring their local models to use with ", "context": "", "use_case": "local agentic search", "switched_from": "", "story": "Running Local Agentic PDF Search with Eno"}]}