Gemma
Google DeepMind's family of open-weight language models that can run locally.
Liked as a capable open-weight model for its size that runs well on consumer hardware1234, and for writing and small utility tasks.5678 The most common complaint is that it falls apart after context compaction or at long contexts91011, with others reporting weak tool calling for coding.1213
Why people recommend it
Local hardware · 3
Commenters run it on consumer GPUs, Macs and budget PCs with acceptable results.
“I use Gemma 4 right now on a Debian Linux with no GUI that I access remotely. Its works pretty well”
Show 1 more
“I run Gemma 4 26b (mixture of experts) and Qwen 3 coder with Ollama.”
Writing quality · 3
Several prefer it for drafting human-facing text, stories and creative role-play.
“I use Qwen/Claude for coding and Gemini/Gemma to draft text intended for a human.”
“I absolutely don't hate the way Gemma 4 writes”
Show 1 more
“you should check out Gemma 4 for creative DMing use case”
Strong for size · 3
Described as unusually capable relative to its parameter count.
“Its shockingly capable for its size.”
“Gemma4 models are unbeatable at their size.”
Show 1 more
“Gemma 4 26B writes some really efficient code, especially considering the size.”
Utility tasks · 3
Used for classification, memory extraction, transcription cleanup and similar micro-tasks.
“found a small classifier (gemma 4) with traditional embeddings worked 2x as well.”
“The local model is too slow in my setup for a chat agent, but for memory extraction it works pretty well”
Show 1 more
“I use small models like Gemma to improve transcriptions from ASR models amongst other micro-tasks.”
Reliable all-rounder · 3
Some call it the most reliable general-purpose model they can self-host.
“Gemma 4 is the best all-rounder among the models I can self-host”
“The reason I use Gemma 4 so much often comes down to how reliable it is”
Show 1 more
“Gemma is actually a very good general-purpose model that can work with multiple languages.”
Speed · 2
Users report high local token rates on capable hardware.
“I get 100 tokens/s with Gemma 4 27b model”
“I squeeze ~130 t/s on Gemma or ~70t/s on Qwen.”
Caveats and dissent
Compaction failures · 3
Tasks reportedly break or loop after context compaction or at higher contexts.
“The gemma models were terrible and fell apart during compaction.”
“My issue with Gemma 4 is that any task fails to complete after any compaction event.”
Show 1 more
“Gemma 4 26B writes some really efficient code, especially considering the size.”
Tool calling · 3
Some found it struggles with tool calling and agentic coding setups.
“Gemma feels nice to write with but every time I use it for coding it struggles with tool calling significantly.”
“I've been struggling to get Gemma 4 to run with it reliably”
Show 1 more
“I use it for recommendations rather than knowledge, like recipes or basic stuff”
Below frontier · 3
Users say it still trails large frontier models like Opus or DeepSeek V4 Pro.
“I love Gemma, especially the latest Gemma 4 release”
“I use Gemma 4 right now on a Debian Linux with no GUI that I access remotely. Its works pretty well”
Show 1 more
“Since Gemma 4 came this easter the gap from self hosting models to Claude has decreased sigificantly”
Template sensitivity · 1
The 26B A4B variant can get stuck in loops; one user advises using the original chat templates.
“Gemma 4 12B, Gemma 4 12B QAT, Gemma 4 31B, Gemma 4 31B QAT”
Prompt injection · 1
A commenter flags a special-token prompt injection issue affecting Gemma 4 tooling.
“For Gemma family it is <|turn>”
Often compared with
1 moved from Claude to Gemma
1 moved from Gemma to Qwen 3.8
2 moved from Qwen 3.5 to Gemma
Who recommends it in their own words
From the makers not counted in the ranking
“We've been running bots exclusively on Gemma 4 31B and 26B for the last 7-10 days and they've been either breaking even or trading very well”
Threads where it came up
- Running local models is good now 6 mentions
- Google releases Gemma 4 open models 5 mentions
- Meta Muse Glimmer – open weights 30B local coding model 4 mentions
- Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac 3 mentions
- Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding? 3 mentions
- Ask HN: What does your local LLM setup looks like? 3 mentions
Also written as Gemma, Gemma4, Gemma 3, Gemma-4. Data: gemma.json. Quotes are verbatim; the verdict and theme labels are written by a model from these comments, and every clause links to its sources.
All comments 47
“runs them through a local Gemma model to write a paragraph description and OCR any text in the thumbnail”
“I switched to Gemma 4 instead.”
“Gemma4 based one is on a branch”
“building one on Gemma3 and Gemma4”
“found a small classifier (gemma 4) with traditional embeddings worked 2x as well.”
“The gemma models were terrible and fell apart during compaction.”
“The only model supported right now is a quantized Gemma-4”
“I settled on gemma-4-26B-A4B as my daily driver.”
“Seems less prone to getting distracted with ordering.”
“I used to run gemma 4 e4b-it-qat fully in GPU (~150 t/s)”
“Running Gemma e4b, Gemma 4 26b a4b IT”
“Locally I use Deepseek Flash, Qwen3.8-Next-Flash, or Gemma 4”
“using Qwen and Gemma models”
“Gemma models still outperform it in that department.”
“I absolutely don't hate the way Gemma 4 writes”
“the smaller Gemma 4 models or qwen3.6:9b would probably run OK on that”
“I run Gemma-4 on an RTX 6000”
“I love all these open weight models”
“I use Qwen/Claude for coding and Gemini/Gemma to draft text intended for a human.”
“I get 100 tokens/s with Gemma 4 27b model”
“My issue with Gemma 4 is that any task fails to complete after any compaction event.”
“I used gemma4 I think”
“German language has never been a big focus for asian models but they still outperform Gemma models”
“Gemma 4 12B, Gemma 4 12B QAT, Gemma 4 31B, Gemma 4 31B QAT”
“The reason I use Gemma 4 so much often comes down to how reliable it is”
“increasingly using opencode+Gemma 4, and developing some system prompts to iteratively improve the results”
“Exactly why Gemma 4 are so popular.”
“LLMs (including Gemma-4 12B with its built-in image encoder) understands a lot of my references and influences”
“Again, I really like both the Qwen and Gemma MoEs”
“you should check out Gemma 4 for creative DMing use case”
“start with Qwen3.6, Google's Gemma4, OpenAI's gpt-oss-20b or Nvidia's Nemotron”
“For Gemma family it is <|turn>”
“I use Gemma 4 right now on a Debian Linux with no GUI that I access remotely. Its works pretty well”
“Let's me switch between any model including Gemma 4 running locally.”
“Gemma feels nice to write with but every time I use it for coding it struggles with tool calling significantly.”
“I fine tuned Gemma with domain specific knowledge.”
“Gemma is actually a very good general-purpose model that can work with multiple languages.”
“near realtime spoken word translation/transcription project using selfhosted Gemma4 and Nvidia Parakeet LLM models”
“I'm able to use ROCm with MTP just fine (Qwen and Gemma).”
“gemma is workable”
“The local model is too slow in my setup for a chat agent, but for memory extraction it works pretty well”
“Gemma 4 is the best all-rounder among the models I can self-host”
“Its shockingly capable for its size.”
“Gemma4 models are unbeatable at their size.”
“I use small models like Gemma to improve transcriptions from ASR models amongst other micro-tasks.”
“Gemma 4 26B writes some really efficient code, especially considering the size.”
“I've been struggling to get Gemma 4 to run with it reliably”