{"slug": "strata", "name": "Strata", "category": "ai", "category_name": "AI models and assistants", "description": "Local LLM inference runtime for running large models on consumer GPUs.", "recommenders": 8, "commenters": 11, "warners": 1, "score": 7.322, "rising": 8.0, "first": "2026-10-02", "latest": "2026-10-27", "why": "Recommended for running bigger, less-quantized local models on consumer GPUs like AMD's 9070XT and R9700, with users reporting roughly 2-3x faster generation and better outputs than llama.cpp or Qwen-3.8-27B.", "why_is_quote": false, "has_verdict": true, "sentiment": {"recommend": 8, "neutral": 0, "compare": 2, "warn": 1}, "spark": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8], "listed": true, "loved": ["Consumer hardware", "Output quality"], "caveat": "Vision accuracy", "short_quote": "You can basically ignore everything in that table and use Strata instead, you can use bigger/less-quantized models and have way better results.", "icon": true, "domain": null, "github_repo": "github.com/niko1221/strata", "official_url": "https://github.com/Niko1221/Strata", "pricing": "unknown", "aliases": ["Strata"], "verdict": [{"text": "Recommended for running bigger, less-quantized local models on consumer GPUs like AMD's 9070XT and R9700,", "cites": [{"id": 804192352, "item_id": 50262022, "url": "https://news.ycombinator.com/item?id=50262022", "by": "plqbfbv", "date": "2026-10-27", "stance": "recommend", "quote": "You can basically ignore everything in that table and use Strata instead, you can use bigger/less-quantized models and have way better results.", "excerpt": "I've benchmarked quite a few models on my 9070XT, and I basically gave up hope of using models locally: https://claude.ai/artifact/77pHgucqQNsmic8ZEnqdBA I asked Claude to come up with 7 problems to solve that were not part of any benchmarks. Besides speed, I asked it to put together a scoring that would reflect models that perform well on the 7 problems and in a reasonable time-frame (RWS, the definition is in the a", "context": "runs models on a 9070XT", "use_case": "running local LLMs on 16GB VRAM", "switched_from": "", "story": "New iMac with M6"}, {"id": 799364528, "item_id": 49960283, "url": "https://news.ycombinator.com/item?id=49960283", "by": "cjdell", "date": "2026-10-05", "stance": "recommend", "quote": "This is game changing. My R9700 32GB is now smarter and about 2x faster than using Qwen-3.8-27B.", "excerpt": "This is game changing. My R9700 32GB is now smarter and about 2x faster than using Qwen-3.8-27B. About 60 t/s when combined with my 96GB of DDR4. My motherboard limits me to PCIe Gen3 so that is likely a bottleneck. For the Nix inclined: https://github.com/cjdell/nixos-config/blob/main/hosts/zen3-nixos/ai/strata.nix Even got it running on the iGPU of a GMKTec M6 Ryzen 6600H at reasonable speed (10 t/s). Fast enough t", "context": "", "use_case": "running large local models", "switched_from": "", "story": "Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s"}]}, {"text": " with users reporting roughly 2-3x faster generation and better outputs than llama.cpp or Qwen-3.8-27B.", "cites": [{"id": 799284000, "item_id": 49955250, "url": "https://news.ycombinator.com/item?id=49955250", "by": "latentsea", "date": "2026-10-04", "stance": "recommend", "quote": "I've switched to IQ3_XXS and am running at 60 t/s on Strata vs the 21 t/s I was getting in llama.cpp. Better outputs too.", "excerpt": "You can run IQ3_XXS, IQ3_S, and IQ4_XS too. I've switched to IQ3_XXS and am running at 60 t/s on Strata vs the 21 t/s I was getting in llama.cpp. Better outputs too.", "context": "", "use_case": "local LLM inference", "switched_from": "llama.cpp", "story": "Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s"}, {"id": 799364528, "item_id": 49960283, "url": "https://news.ycombinator.com/item?id=49960283", "by": "cjdell", "date": "2026-10-05", "stance": "recommend", "quote": "This is game changing. My R9700 32GB is now smarter and about 2x faster than using Qwen-3.8-27B.", "excerpt": "This is game changing. My R9700 32GB is now smarter and about 2x faster than using Qwen-3.8-27B. About 60 t/s when combined with my 96GB of DDR4. My motherboard limits me to PCIe Gen3 so that is likely a bottleneck. For the Nix inclined: https://github.com/cjdell/nixos-config/blob/main/hosts/zen3-nixos/ai/strata.nix Even got it running on the iGPU of a GMKTec M6 Ryzen 6600H at reasonable speed (10 t/s). Fast enough t", "context": "", "use_case": "running large local models", "switched_from": "", "story": "Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s"}]}, {"text": " Complaints are few and scattered: one user found it performed much worse on a vision benchmark with the same weights,", "cites": [{"id": 799387696, "item_id": 49961731, "url": "https://news.ycombinator.com/item?id=49961731", "by": "Jackson__", "date": "2026-10-05", "stance": "warn", "quote": "Strata performs so much worse with the same weights", "excerpt": ">To be done right, such vision benchmark should consider the peculiarities of the vision tower As per my previous comment, I used the _exact same weights_ for the comparison. The vision tower is always kept as an unquantized BF16 file for GGUF, as I believe is the default. >Second of all, the \"find coordinates\" of something is super difficult task of any model It is in fact not (anymore), and most recent VLMs I teste", "context": "", "use_case": "running vision models locally", "switched_from": "", "story": "Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s"}]}, {"text": " and it is a single-chat runtime built around a specific Qwen model.", "cites": [{"id": 799032354, "item_id": 49939522, "url": "https://news.ycombinator.com/item?id=49939522", "by": "sleight42", "date": "2026-10-02", "stance": "compare", "quote": "I'll use a single-chat/slot runtime like Strata or Freetoken.", "excerpt": "I don't do that. I use my 2020 top end gaming PC with its 3090. I just ordered 128GB RAM for it. I'll use a single-chat/slot runtime like Strata or Freetoken. And then I'll be able to run either a blazing fast 8-bit Qwen 3.8 27b or a 4 or 5 bit Qwen 3.8 Flash Next. That's good enough for me for development. And then I'll have my current gaming rig with less RAM but better CPU and GPU run a reasonable smart tool agent", "context": "has a 2020 gaming PC with a 3090", "use_case": "local model runtime", "switched_from": "", "story": "Dots: Always-on agents"}, {"id": 801050688, "item_id": 50065668, "url": "https://news.ycombinator.com/item?id=50065668", "by": "suprjami", "date": "2026-10-13", "stance": "recommend", "quote": "Strata already does this with Qwen3.8-Flash-Next", "excerpt": "Qwen 3? (Jumanji) What year is it? What a waste of time. Strata already does this with Qwen3.8-Flash-Next: https://github.com/Niko1221/Strata", "context": "", "use_case": "running LLMs larger than RAM", "switched_from": "", "story": "Janas: Run LLMs larger than RAM by streaming experts from SSD"}]}], "praise": [{"label": "Consumer hardware", "summary": "Runs large models on modest or older single GPUs, even an iGPU.", "quotes": [{"id": 799364528, "item_id": 49960283, "url": "https://news.ycombinator.com/item?id=49960283", "by": "cjdell", "date": "2026-10-05", "stance": "recommend", "quote": "This is game changing. My R9700 32GB is now smarter and about 2x faster than using Qwen-3.8-27B.", "excerpt": "This is game changing. My R9700 32GB is now smarter and about 2x faster than using Qwen-3.8-27B. About 60 t/s when combined with my 96GB of DDR4. My motherboard limits me to PCIe Gen3 so that is likely a bottleneck. For the Nix inclined: https://github.com/cjdell/nixos-config/blob/main/hosts/zen3-nixos/ai/strata.nix Even got it running on the iGPU of a GMKTec M6 Ryzen 6600H at reasonable speed (10 t/s). Fast enough t", "context": "", "use_case": "running large local models", "switched_from": "", "story": "Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s"}, {"id": 801082544, "item_id": 50067659, "url": "https://news.ycombinator.com/item?id=50067659", "by": "sosuke", "date": "2026-10-13", "stance": "recommend", "quote": "Strata has been such a boon to my own local development", "excerpt": "In a spirit of trying to do what my imagination wants to be possible I set off to use or recreate Strata but to load Poolside Laguna S 2.1 Q3 on my system. My machine has an RTX 4060 Ti 16 GB on PCI 3.0, Threadripper 1950X with AVX2, and 128 GB quad-channel DDR4-2133 I stopped after hitting a successful 84-minute autonomous run in OMP. Just 4 bugs in the result found by a frontier-ish model. Hit 770.83 tokens/sec on ", "context": "runs local models on RTX 4060 Ti, Threadripper", "use_case": "local development with open-weight models", "switched_from": "", "story": "Tried making Strata-like work for Poolside Laguna S 2.1 Q3 moderate success"}, {"id": 802045376, "item_id": 50127836, "url": "https://news.ycombinator.com/item?id=50127836", "by": "peter_d_sherman", "date": "2026-10-17", "stance": "compare", "quote": "Strata may have a worthy competitor here!", "excerpt": "A Strata (https://github.com/Niko1221/Strata) challenger! The first one I've seen (although I'm sure there are and will be more as more time progresses...) It seems to require a bit more system memory (70GiB free RAM as opposed to 32GB) and two newer nVidia RTX 50 series graphics cards with 16GB (as opposed to a larger choice of multi-vendor older single graphics cards with a minimum of 12GB), but on the flip side, i", "context": "", "use_case": "running large MoE models locally", "switched_from": "", "story": "Show HN: Capy-flash \u2013 a 125B MoE model on two 16 GB consumer GPUs"}]}, {"label": "Output quality", "summary": "Lets users run bigger, less-quantized models with better results.", "quotes": [{"id": 804192352, "item_id": 50262022, "url": "https://news.ycombinator.com/item?id=50262022", "by": "plqbfbv", "date": "2026-10-27", "stance": "recommend", "quote": "You can basically ignore everything in that table and use Strata instead, you can use bigger/less-quantized models and have way better results.", "excerpt": "I've benchmarked quite a few models on my 9070XT, and I basically gave up hope of using models locally: https://claude.ai/artifact/77pHgucqQNsmic8ZEnqdBA I asked Claude to come up with 7 problems to solve that were not part of any benchmarks. Besides speed, I asked it to put together a scoring that would reflect models that perform well on the 7 problems and in a reasonable time-frame (RWS, the definition is in the a", "context": "runs models on a 9070XT", "use_case": "running local LLMs on 16GB VRAM", "switched_from": "", "story": "New iMac with M6"}, {"id": 799284000, "item_id": 49955250, "url": "https://news.ycombinator.com/item?id=49955250", "by": "latentsea", "date": "2026-10-04", "stance": "recommend", "quote": "I've switched to IQ3_XXS and am running at 60 t/s on Strata vs the 21 t/s I was getting in llama.cpp. Better outputs too.", "excerpt": "You can run IQ3_XXS, IQ3_S, and IQ4_XS too. I've switched to IQ3_XXS and am running at 60 t/s on Strata vs the 21 t/s I was getting in llama.cpp. Better outputs too.", "context": "", "use_case": "local LLM inference", "switched_from": "llama.cpp", "story": "Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s"}, {"id": 799364528, "item_id": 49960283, "url": "https://news.ycombinator.com/item?id=49960283", "by": "cjdell", "date": "2026-10-05", "stance": "recommend", "quote": "This is game changing. My R9700 32GB is now smarter and about 2x faster than using Qwen-3.8-27B.", "excerpt": "This is game changing. My R9700 32GB is now smarter and about 2x faster than using Qwen-3.8-27B. About 60 t/s when combined with my 96GB of DDR4. My motherboard limits me to PCIe Gen3 so that is likely a bottleneck. For the Nix inclined: https://github.com/cjdell/nixos-config/blob/main/hosts/zen3-nixos/ai/strata.nix Even got it running on the iGPU of a GMKTec M6 Ryzen 6600H at reasonable speed (10 t/s). Fast enough t", "context": "", "use_case": "running large local models", "switched_from": "", "story": "Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s"}]}, {"label": "Speed", "summary": "Users report about 2-3x faster generation than their previous setups.", "quotes": [{"id": 799284000, "item_id": 49955250, "url": "https://news.ycombinator.com/item?id=49955250", "by": "latentsea", "date": "2026-10-04", "stance": "recommend", "quote": "I've switched to IQ3_XXS and am running at 60 t/s on Strata vs the 21 t/s I was getting in llama.cpp. Better outputs too.", "excerpt": "You can run IQ3_XXS, IQ3_S, and IQ4_XS too. I've switched to IQ3_XXS and am running at 60 t/s on Strata vs the 21 t/s I was getting in llama.cpp. Better outputs too.", "context": "", "use_case": "local LLM inference", "switched_from": "llama.cpp", "story": "Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s"}, {"id": 799364528, "item_id": 49960283, "url": "https://news.ycombinator.com/item?id=49960283", "by": "cjdell", "date": "2026-10-05", "stance": "recommend", "quote": "This is game changing. My R9700 32GB is now smarter and about 2x faster than using Qwen-3.8-27B.", "excerpt": "This is game changing. My R9700 32GB is now smarter and about 2x faster than using Qwen-3.8-27B. About 60 t/s when combined with my 96GB of DDR4. My motherboard limits me to PCIe Gen3 so that is likely a bottleneck. For the Nix inclined: https://github.com/cjdell/nixos-config/blob/main/hosts/zen3-nixos/ai/strata.nix Even got it running on the iGPU of a GMKTec M6 Ryzen 6600H at reasonable speed (10 t/s). Fast enough t", "context": "", "use_case": "running large local models", "switched_from": "", "story": "Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s"}]}, {"label": "User satisfaction", "summary": "Users describe it as very satisfying and a boon for local development.", "quotes": [{"id": 801086897, "item_id": 50067931, "url": "https://news.ycombinator.com/item?id=50067931", "by": "MadameMinty", "date": "2026-10-13", "stance": "recommend", "quote": "I use this (tuned) and I cannot understate how satisfied I am.", "excerpt": "Then check out those: https://github.com/prabanta-dev/janas <- this, focuses on Qwen3-Next-80B-A3B. https://github.com/Niko1221/Strata <- Qwen3.8-Flash-Next. I use this (tuned) and I cannot understate how satisfied I am. https://github.com/antirez/ds4 <- DeepSeek 4 family + more.", "context": "", "use_case": "local LLM inference", "switched_from": "", "story": "Janas: Run LLMs larger than RAM by streaming experts from SSD"}, {"id": 801082544, "item_id": 50067659, "url": "https://news.ycombinator.com/item?id=50067659", "by": "sosuke", "date": "2026-10-13", "stance": "recommend", "quote": "Strata has been such a boon to my own local development", "excerpt": "In a spirit of trying to do what my imagination wants to be possible I set off to use or recreate Strata but to load Poolside Laguna S 2.1 Q3 on my system. My machine has an RTX 4060 Ti 16 GB on PCI 3.0, Threadripper 1950X with AVX2, and 128 GB quad-channel DDR4-2133 I stopped after hitting a successful 84-minute autonomous run in OMP. Just 4 bugs in the result found by a frontier-ish model. Hit 770.83 tokens/sec on ", "context": "runs local models on RTX 4060 Ti, Threadripper", "use_case": "local development with open-weight models", "switched_from": "", "story": "Tried making Strata-like work for Poolside Laguna S 2.1 Q3 moderate success"}]}, {"label": "Docs", "summary": "Its docs include guidance on picking models by available memory.", "quotes": [{"id": 803830384, "item_id": 50239399, "url": "https://news.ycombinator.com/item?id=50239399", "by": "CamperBob2", "date": "2026-10-26", "stance": "recommend", "quote": "Take a look at the docs for https://github.com/niko1221/strata for some ideas", "excerpt": "Take a look at the docs for https://github.com/niko1221/strata for some ideas, particularly https://github.com/Niko1221/Strata/blob/main/docs/MODELS.md#pick-by-memory .", "context": "", "use_case": "running LLMs at home", "switched_from": "", "story": "OpenAI and Anthropic cut their LLM prices by up to 80% within a few months. Why?"}]}, {"label": "TUI to-do list", "summary": "A live to-do list in its TUI drew envy from users of another tool.", "quotes": [{"id": 799381185, "item_id": 49961324, "url": "https://news.ycombinator.com/item?id=49961324", "by": "Schlagbohrer", "date": "2026-10-05", "stance": "recommend", "quote": "I would love to have that To Do list interface in Pi.", "excerpt": "I would love to have that To Do list interface in Pi. Pi is great and my daily driver but it only has such nice pop-up TUI for the sub-agent management. I use it on a monitor which is tipped 90\u00b0, so I have a lot of room for TUI stuff, and a live To Do list would be awesome to glance at. Edit: I am referring to the To Do list shown in the TUI in this screenshot from the repo: https://raw.githubusercontent.com/Niko1221", "context": "", "use_case": "agent TUI with To Do list", "switched_from": "", "story": "Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s"}]}], "caveats": [{"label": "Vision accuracy", "summary": "One user found it performed much worse on a vision benchmark using the same weights.", "quotes": [{"id": 799387696, "item_id": 49961731, "url": "https://news.ycombinator.com/item?id=49961731", "by": "Jackson__", "date": "2026-10-05", "stance": "warn", "quote": "Strata performs so much worse with the same weights", "excerpt": ">To be done right, such vision benchmark should consider the peculiarities of the vision tower As per my previous comment, I used the _exact same weights_ for the comparison. The vision tower is always kept as an unquantized BF16 file for GGUF, as I believe is the default. >Second of all, the \"find coordinates\" of something is super difficult task of any model It is in fact not (anymore), and most recent VLMs I teste", "context": "", "use_case": "running vision models locally", "switched_from": "", "story": "Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s"}]}, {"label": "Narrow model focus", "summary": "It is built around Qwen3.8-Flash-Next; one user had to recreate it to load another model.", "quotes": [{"id": 801050688, "item_id": 50065668, "url": "https://news.ycombinator.com/item?id=50065668", "by": "suprjami", "date": "2026-10-13", "stance": "recommend", "quote": "Strata already does this with Qwen3.8-Flash-Next", "excerpt": "Qwen 3? (Jumanji) What year is it? What a waste of time. Strata already does this with Qwen3.8-Flash-Next: https://github.com/Niko1221/Strata", "context": "", "use_case": "running LLMs larger than RAM", "switched_from": "", "story": "Janas: Run LLMs larger than RAM by streaming experts from SSD"}, {"id": 801086897, "item_id": 50067931, "url": "https://news.ycombinator.com/item?id=50067931", "by": "MadameMinty", "date": "2026-10-13", "stance": "recommend", "quote": "I use this (tuned) and I cannot understate how satisfied I am.", "excerpt": "Then check out those: https://github.com/prabanta-dev/janas <- this, focuses on Qwen3-Next-80B-A3B. https://github.com/Niko1221/Strata <- Qwen3.8-Flash-Next. I use this (tuned) and I cannot understate how satisfied I am. https://github.com/antirez/ds4 <- DeepSeek 4 family + more.", "context": "", "use_case": "local LLM inference", "switched_from": "", "story": "Janas: Run LLMs larger than RAM by streaming experts from SSD"}, {"id": 801082544, "item_id": 50067659, "url": "https://news.ycombinator.com/item?id=50067659", "by": "sosuke", "date": "2026-10-13", "stance": "recommend", "quote": "Strata has been such a boon to my own local development", "excerpt": "In a spirit of trying to do what my imagination wants to be possible I set off to use or recreate Strata but to load Poolside Laguna S 2.1 Q3 on my system. My machine has an RTX 4060 Ti 16 GB on PCI 3.0, Threadripper 1950X with AVX2, and 128 GB quad-channel DDR4-2133 I stopped after hitting a successful 84-minute autonomous run in OMP. Just 4 bugs in the result found by a frontier-ish model. Hit 770.83 tokens/sec on ", "context": "runs local models on RTX 4060 Ti, Threadripper", "use_case": "local development with open-weight models", "switched_from": "", "story": "Tried making Strata-like work for Poolside Laguna S 2.1 Q3 moderate success"}]}, {"label": "Single session", "summary": "It is a single-chat/slot runtime.", "quotes": [{"id": 799032354, "item_id": 49939522, "url": "https://news.ycombinator.com/item?id=49939522", "by": "sleight42", "date": "2026-10-02", "stance": "compare", "quote": "I'll use a single-chat/slot runtime like Strata or Freetoken.", "excerpt": "I don't do that. I use my 2020 top end gaming PC with its 3090. I just ordered 128GB RAM for it. I'll use a single-chat/slot runtime like Strata or Freetoken. And then I'll be able to run either a blazing fast 8-bit Qwen 3.8 27b or a 4 or 5 bit Qwen 3.8 Flash Next. That's good enough for me for development. And then I'll have my current gaming rig with less RAM but better CPU and GPU run a reasonable smart tool agent", "context": "has a 2020 gaming PC with a 3090", "use_case": "local model runtime", "switched_from": "", "story": "Dots: Always-on agents"}]}, {"label": "Competitor speed", "summary": "A challenger offers faster token generation, though it needs more RAM and newer GPUs.", "quotes": [{"id": 802045376, "item_id": 50127836, "url": "https://news.ycombinator.com/item?id=50127836", "by": "peter_d_sherman", "date": "2026-10-17", "stance": "compare", "quote": "Strata may have a worthy competitor here!", "excerpt": "A Strata (https://github.com/Niko1221/Strata) challenger! The first one I've seen (although I'm sure there are and will be more as more time progresses...) It seems to require a bit more system memory (70GiB free RAM as opposed to 32GB) and two newer nVidia RTX 50 series graphics cards with 16GB (as opposed to a larger choice of multi-vendor older single graphics cards with a minimum of 12GB), but on the flip side, i", "context": "", "use_case": "running large MoE models locally", "switched_from": "", "story": "Show HN: Capy-flash \u2013 a 125B MoE model on two 16 GB consumer GPUs"}]}], "who": [{"by": "plqbfbv", "context": "runs models on a 9070XT", "url": "https://news.ycombinator.com/item?id=50262022", "date": "2026-10-27"}, {"by": "sosuke", "context": "runs local models on RTX 4060 Ti, Threadripper", "url": "https://news.ycombinator.com/item?id=50067659", "date": "2026-10-13"}], "tags": ["for local LLMs", "for AMD GPUs", "for consumer GPUs", "for local coding agents", "for Nix users"], "use_cases": ["local llm inference", "running local llms on 16gb vram", "running llms at home", "running large moe models locally", "local development with open-weight models", "running llms larger than ram"], "compared": [{"slug": "qwen-3-8", "name": "Qwen 3.8", "co": 4, "switched_from": 0, "switched_to": 0}, {"slug": "llama-cpp", "name": "llama.cpp", "co": 1, "switched_from": 1, "switched_to": 0}, {"slug": "openrouter", "name": "OpenRouter", "co": 2, "switched_from": 0, "switched_to": 0}], "makers": [{"id": 803825185, "item_id": 50239074, "url": "https://news.ycombinator.com/item?id=50239074", "by": "bitexploder", "date": "2026-10-26", "stance": "recommend", "quote": "I built my own inference engine from scratch for fun. You can run Strata on 16-32GB cards at very respectable token rates", "excerpt": "I mean, I just have 4xV100S GPU and a Macbook Pro w/48GB of RAM and I find it kind of amazing. My server can run Qwen Flash Next at 500 t/s aggregate (about 200 t/s single user). It's pretty great. I use it for local coding and all my worker agents. It is about as reliable as Sonnet 5 for agentic coding. I make 3D models, keep the home network and server updated, fully built ansible files. You do NOT need 20K in hard", "context": "", "use_case": "local model inference", "switched_from": "", "story": "IBM CEO says open weight models are now good and cheap on-prem"}], "threads": [{"url": "https://news.ycombinator.com/item?id=49953495", "title": "Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s", "mentions": 4}, {"url": "https://news.ycombinator.com/item?id=50063717", "title": "Janas: Run LLMs larger than RAM by streaming experts from SSD", "mentions": 2}, {"url": "https://news.ycombinator.com/item?id=50258538", "title": "New iMac with M6", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=50236102", "title": "OpenAI and Anthropic cut their LLM prices by up to 80% within a few months. Why?", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=50238088", "title": "IBM CEO says open weight models are now good and cheap on-prem", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=50118153", "title": "Show HN: Capy-flash \u2013 a 125B MoE model on two 16 GB consumer GPUs", "mentions": 1}], "sentiment_by_month": {"2025-11": {}, "2025-12": {}, "2026-01": {}, "2026-02": {}, "2026-03": {}, "2026-04": {}, "2026-05": {}, "2026-06": {}, "2026-07": {}, "2026-08": {}, "2026-09": {}, "2026-10": {"compare": 2, "recommend": 8, "warn": 1}}, "quotes": [{"id": 804192352, "item_id": 50262022, "url": "https://news.ycombinator.com/item?id=50262022", "by": "plqbfbv", "date": "2026-10-27", "stance": "recommend", "quote": "You can basically ignore everything in that table and use Strata instead, you can use bigger/less-quantized models and have way better results.", "excerpt": "I've benchmarked quite a few models on my 9070XT, and I basically gave up hope of using models locally: https://claude.ai/artifact/77pHgucqQNsmic8ZEnqdBA I asked Claude to come up with 7 problems to solve that were not part of any benchmarks. Besides speed, I asked it to put together a scoring that would reflect models that perform well on the 7 problems and in a reasonable time-frame (RWS, the definition is in the a", "context": "runs models on a 9070XT", "use_case": "running local LLMs on 16GB VRAM", "switched_from": "", "story": "New iMac with M6"}, {"id": 803830384, "item_id": 50239399, "url": "https://news.ycombinator.com/item?id=50239399", "by": "CamperBob2", "date": "2026-10-26", "stance": "recommend", "quote": "Take a look at the docs for https://github.com/niko1221/strata for some ideas", "excerpt": "Take a look at the docs for https://github.com/niko1221/strata for some ideas, particularly https://github.com/Niko1221/Strata/blob/main/docs/MODELS.md#pick-by-memory .", "context": "", "use_case": "running LLMs at home", "switched_from": "", "story": "OpenAI and Anthropic cut their LLM prices by up to 80% within a few months. Why?"}, {"id": 802045376, "item_id": 50127836, "url": "https://news.ycombinator.com/item?id=50127836", "by": "peter_d_sherman", "date": "2026-10-17", "stance": "compare", "quote": "Strata may have a worthy competitor here!", "excerpt": "A Strata (https://github.com/Niko1221/Strata) challenger! The first one I've seen (although I'm sure there are and will be more as more time progresses...) It seems to require a bit more system memory (70GiB free RAM as opposed to 32GB) and two newer nVidia RTX 50 series graphics cards with 16GB (as opposed to a larger choice of multi-vendor older single graphics cards with a minimum of 12GB), but on the flip side, i", "context": "", "use_case": "running large MoE models locally", "switched_from": "", "story": "Show HN: Capy-flash \u2013 a 125B MoE model on two 16 GB consumer GPUs"}, {"id": 801086897, "item_id": 50067931, "url": "https://news.ycombinator.com/item?id=50067931", "by": "MadameMinty", "date": "2026-10-13", "stance": "recommend", "quote": "I use this (tuned) and I cannot understate how satisfied I am.", "excerpt": "Then check out those: https://github.com/prabanta-dev/janas <- this, focuses on Qwen3-Next-80B-A3B. https://github.com/Niko1221/Strata <- Qwen3.8-Flash-Next. I use this (tuned) and I cannot understate how satisfied I am. https://github.com/antirez/ds4 <- DeepSeek 4 family + more.", "context": "", "use_case": "local LLM inference", "switched_from": "", "story": "Janas: Run LLMs larger than RAM by streaming experts from SSD"}, {"id": 801082544, "item_id": 50067659, "url": "https://news.ycombinator.com/item?id=50067659", "by": "sosuke", "date": "2026-10-13", "stance": "recommend", "quote": "Strata has been such a boon to my own local development", "excerpt": "In a spirit of trying to do what my imagination wants to be possible I set off to use or recreate Strata but to load Poolside Laguna S 2.1 Q3 on my system. My machine has an RTX 4060 Ti 16 GB on PCI 3.0, Threadripper 1950X with AVX2, and 128 GB quad-channel DDR4-2133 I stopped after hitting a successful 84-minute autonomous run in OMP. Just 4 bugs in the result found by a frontier-ish model. Hit 770.83 tokens/sec on ", "context": "runs local models on RTX 4060 Ti, Threadripper", "use_case": "local development with open-weight models", "switched_from": "", "story": "Tried making Strata-like work for Poolside Laguna S 2.1 Q3 moderate success"}, {"id": 801050688, "item_id": 50065668, "url": "https://news.ycombinator.com/item?id=50065668", "by": "suprjami", "date": "2026-10-13", "stance": "recommend", "quote": "Strata already does this with Qwen3.8-Flash-Next", "excerpt": "Qwen 3? (Jumanji) What year is it? What a waste of time. Strata already does this with Qwen3.8-Flash-Next: https://github.com/Niko1221/Strata", "context": "", "use_case": "running LLMs larger than RAM", "switched_from": "", "story": "Janas: Run LLMs larger than RAM by streaming experts from SSD"}, {"id": 799387696, "item_id": 49961731, "url": "https://news.ycombinator.com/item?id=49961731", "by": "Jackson__", "date": "2026-10-05", "stance": "warn", "quote": "Strata performs so much worse with the same weights", "excerpt": ">To be done right, such vision benchmark should consider the peculiarities of the vision tower As per my previous comment, I used the _exact same weights_ for the comparison. The vision tower is always kept as an unquantized BF16 file for GGUF, as I believe is the default. >Second of all, the \"find coordinates\" of something is super difficult task of any model It is in fact not (anymore), and most recent VLMs I teste", "context": "", "use_case": "running vision models locally", "switched_from": "", "story": "Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s"}, {"id": 799381185, "item_id": 49961324, "url": "https://news.ycombinator.com/item?id=49961324", "by": "Schlagbohrer", "date": "2026-10-05", "stance": "recommend", "quote": "I would love to have that To Do list interface in Pi.", "excerpt": "I would love to have that To Do list interface in Pi. Pi is great and my daily driver but it only has such nice pop-up TUI for the sub-agent management. I use it on a monitor which is tipped 90\u00b0, so I have a lot of room for TUI stuff, and a live To Do list would be awesome to glance at. Edit: I am referring to the To Do list shown in the TUI in this screenshot from the repo: https://raw.githubusercontent.com/Niko1221", "context": "", "use_case": "agent TUI with To Do list", "switched_from": "", "story": "Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s"}, {"id": 799364528, "item_id": 49960283, "url": "https://news.ycombinator.com/item?id=49960283", "by": "cjdell", "date": "2026-10-05", "stance": "recommend", "quote": "This is game changing. My R9700 32GB is now smarter and about 2x faster than using Qwen-3.8-27B.", "excerpt": "This is game changing. My R9700 32GB is now smarter and about 2x faster than using Qwen-3.8-27B. About 60 t/s when combined with my 96GB of DDR4. My motherboard limits me to PCIe Gen3 so that is likely a bottleneck. For the Nix inclined: https://github.com/cjdell/nixos-config/blob/main/hosts/zen3-nixos/ai/strata.nix Even got it running on the iGPU of a GMKTec M6 Ryzen 6600H at reasonable speed (10 t/s). Fast enough t", "context": "", "use_case": "running large local models", "switched_from": "", "story": "Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s"}, {"id": 799284000, "item_id": 49955250, "url": "https://news.ycombinator.com/item?id=49955250", "by": "latentsea", "date": "2026-10-04", "stance": "recommend", "quote": "I've switched to IQ3_XXS and am running at 60 t/s on Strata vs the 21 t/s I was getting in llama.cpp. Better outputs too.", "excerpt": "You can run IQ3_XXS, IQ3_S, and IQ4_XS too. I've switched to IQ3_XXS and am running at 60 t/s on Strata vs the 21 t/s I was getting in llama.cpp. Better outputs too.", "context": "", "use_case": "local LLM inference", "switched_from": "llama.cpp", "story": "Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s"}, {"id": 799032354, "item_id": 49939522, "url": "https://news.ycombinator.com/item?id=49939522", "by": "sleight42", "date": "2026-10-02", "stance": "compare", "quote": "I'll use a single-chat/slot runtime like Strata or Freetoken.", "excerpt": "I don't do that. I use my 2020 top end gaming PC with its 3090. I just ordered 128GB RAM for it. I'll use a single-chat/slot runtime like Strata or Freetoken. And then I'll be able to run either a blazing fast 8-bit Qwen 3.8 27b or a 4 or 5 bit Qwen 3.8 Flash Next. That's good enough for me for development. And then I'll have my current gaming rig with less RAM but better CPU and GPU run a reasonable smart tool agent", "context": "has a 2020 gaming PC with a 3090", "use_case": "local model runtime", "switched_from": "", "story": "Dots: Always-on agents"}]}