{"slug": "flash-moe", "name": "flash-moe", "category": "ai", "category_name": "AI tools and apps", "description": "", "recommenders": 3, "commenters": 6, "warners": 0, "score": 3.551, "rising": -0.98, "first": "2026-04-01", "latest": "2026-07-29", "why": "this looks like a good candidate with plenty of room for speedups", "why_is_quote": true, "has_verdict": false, "sentiment": {"recommend": 3, "neutral": 3, "compare": 0, "warn": 0}, "spark": [0, 0, 0, 0, 0, 1, 0, 0, 2, 0, 0, 0], "listed": true, "loved": [], "caveat": "", "short_quote": "this looks like a good candidate with plenty of room for speedups", "kind": "", "icon": false, "domain": null, "github_repo": "github.com/danveloper/flash-moe", "official_url": "https://github.com/danveloper/flash-moe", "pricing": "", "aliases": ["flash-moe", "Flash-MoE"], "verdict": null, "praise": [{"label": "", "summary": "", "quotes": [{"id": 785672962, "item_id": 49104560, "url": "https://news.ycombinator.com/item?id=49104560", "by": "gitpusher42", "date": "2026-07-29", "version": null, "stance": "recommend", "quote": "Check for colibri, dwarf star and flash-moe.", "excerpt": "Check for colibri, dwarf star and flash-moe. they do similar things with bigger models https://github.com/JustVugg/colibri https://github.com/antirez/ds4 https://github.com/danveloper/flash-moe", "context": "", "use_case": "Large local model inference", "switched_from": "", "story": "Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac"}, {"id": 785596769, "item_id": 49099798, "url": "https://news.ycombinator.com/item?id=49099798", "by": "gitpusher42", "date": "2026-07-29", "version": null, "stance": "recommend", "quote": "Check the Colibri and Flash-MoE projects. They\u2019re already doing that.", "excerpt": "Yeah! Check the Colibri and Flash-MoE projects. They\u2019re already doing that. https://github.com/danveloper/flash-moe https://github.com/JustVugg/colibri", "context": "", "use_case": "", "switched_from": "", "story": "Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac"}, {"id": 785594672, "item_id": 49099667, "url": "https://news.ycombinator.com/item?id=49099667", "by": "gitpusher42", "date": "2026-07-29", "version": null, "stance": "recommend", "quote": "Huge respect to them, it was a big inspiration for this project!", "excerpt": "I think I first saw Flash-MoE (https://github.com/danveloper/flash-moe) in April. Huge respect to them, it was a big inspiration for this project!", "context": "", "use_case": "Local model inference", "switched_from": "", "story": "Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac"}, {"id": 781654064, "item_id": 48853379, "url": "https://news.ycombinator.com/item?id=48853379", "by": "bahmboo", "date": "2026-07-09", "version": null, "stance": "recommend", "quote": "this looks like a good candidate with plenty of room for speedups", "excerpt": "Another recent project that runs a huge model on a 48gb Mac is https://github.com/danveloper/flash-moe - it gets over 5 tokens/sec on an M3 Max compared to this projects very impressive 1 token/sec on an M5 Max. So for anyone wanting to tackle a Mac only version that targets lower spec machines this looks like a good candidate with plenty of room for speedups [edit: because it doesn't use the gpu]. Not hijacking anyt", "context": "", "use_case": "running huge models on Mac", "switched_from": "", "story": "Getting GLM 5.2 running on my slow computer"}, {"id": 766804465, "item_id": 47925279, "url": "https://news.ycombinator.com/item?id=47925279", "by": "gibsonsmog", "date": "2026-04-27", "version": null, "stance": "recommend", "quote": "I'm sure people will adapt the Flash-MoE[2] integration to be even better soon as well", "excerpt": "Yeah, local is clearly the future. Even beyond the cheap Chinese models you can install the apfel[1] stuff if you're on a mac and want a quick available onboard cli option. And I'm sure people will adapt the Flash-MoE[2] integration to be even better soon as well. [1] https://apfel.franzai.com/ [2] https://github.com/danveloper/flash-moe", "context": "", "use_case": "MoE model inference", "switched_from": "", "story": "GitHub Copilot is moving to usage-based billing"}]}], "caveats": [], "who": [], "tags": [], "use_cases": ["large local model inference", "local model inference", "running huge models on mac", "moe model inference", "reference implementation"], "compared": [{"slug": "colibri", "name": "Colibri", "co": 2, "switched_from": 0, "switched_to": 0}], "makers": [{"id": 759902736, "item_id": 47493921, "url": "https://news.ycombinator.com/item?id=47493921", "by": "anemll", "date": "2026-03-23", "version": null, "stance": "recommend", "quote": "Check my repo", "excerpt": "Check my repo, I had added some support for GUFF/untloth, Q3,Q5/Q8 https://github.com/Anemll/flash-moe/blob/iOS-App/docs/gguf-hybrid-bringup-log.md", "context": "", "use_case": "", "switched_from": "", "story": "iPhone 17 Pro Demonstrated Running a 400B LLM"}], "threads": [{"url": "https://news.ycombinator.com/item?id=49098510", "title": "Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac", "mentions": 3}, {"url": "https://news.ycombinator.com/item?id=47604354", "title": "TurboQuant KV Compression and SSD Expert Streaming for M5 Pro and IOS", "mentions": 2}, {"url": "https://news.ycombinator.com/item?id=48842459", "title": "Getting GLM 5.2 running on my slow computer", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=47923357", "title": "GitHub Copilot is moving to usage-based billing", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=47884971", "title": "DeepSeek v4", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=47490070", "title": "iPhone 17 Pro Demonstrated Running a 400B LLM", "mentions": 1}], "sentiment_by_month": {"2025-11": {}, "2025-12": {}, "2026-01": {}, "2026-02": {}, "2026-03": {}, "2026-04": {"neutral": 3, "recommend": 1}, "2026-05": {}, "2026-06": {}, "2026-07": {"recommend": 2}, "2026-08": {}, "2026-09": {}, "2026-10": {}}, "versions": [], "version_unnamed": 0, "quotes": [{"id": 785672962, "item_id": 49104560, "url": "https://news.ycombinator.com/item?id=49104560", "by": "gitpusher42", "date": "2026-07-29", "version": null, "stance": "recommend", "quote": "Check for colibri, dwarf star and flash-moe.", "excerpt": "Check for colibri, dwarf star and flash-moe. they do similar things with bigger models https://github.com/JustVugg/colibri https://github.com/antirez/ds4 https://github.com/danveloper/flash-moe", "context": "", "use_case": "Large local model inference", "switched_from": "", "story": "Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac"}, {"id": 785596769, "item_id": 49099798, "url": "https://news.ycombinator.com/item?id=49099798", "by": "gitpusher42", "date": "2026-07-29", "version": null, "stance": "recommend", "quote": "Check the Colibri and Flash-MoE projects. They\u2019re already doing that.", "excerpt": "Yeah! Check the Colibri and Flash-MoE projects. They\u2019re already doing that. https://github.com/danveloper/flash-moe https://github.com/JustVugg/colibri", "context": "", "use_case": "", "switched_from": "", "story": "Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac"}, {"id": 785594672, "item_id": 49099667, "url": "https://news.ycombinator.com/item?id=49099667", "by": "gitpusher42", "date": "2026-07-29", "version": null, "stance": "recommend", "quote": "Huge respect to them, it was a big inspiration for this project!", "excerpt": "I think I first saw Flash-MoE (https://github.com/danveloper/flash-moe) in April. Huge respect to them, it was a big inspiration for this project!", "context": "", "use_case": "Local model inference", "switched_from": "", "story": "Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac"}, {"id": 781654064, "item_id": 48853379, "url": "https://news.ycombinator.com/item?id=48853379", "by": "bahmboo", "date": "2026-07-09", "version": null, "stance": "recommend", "quote": "this looks like a good candidate with plenty of room for speedups", "excerpt": "Another recent project that runs a huge model on a 48gb Mac is https://github.com/danveloper/flash-moe - it gets over 5 tokens/sec on an M3 Max compared to this projects very impressive 1 token/sec on an M5 Max. So for anyone wanting to tackle a Mac only version that targets lower spec machines this looks like a good candidate with plenty of room for speedups [edit: because it doesn't use the gpu]. Not hijacking anyt", "context": "", "use_case": "running huge models on Mac", "switched_from": "", "story": "Getting GLM 5.2 running on my slow computer"}, {"id": 766804465, "item_id": 47925279, "url": "https://news.ycombinator.com/item?id=47925279", "by": "gibsonsmog", "date": "2026-04-27", "version": null, "stance": "recommend", "quote": "I'm sure people will adapt the Flash-MoE[2] integration to be even better soon as well", "excerpt": "Yeah, local is clearly the future. Even beyond the cheap Chinese models you can install the apfel[1] stuff if you're on a mac and want a quick available onboard cli option. And I'm sure people will adapt the Flash-MoE[2] integration to be even better soon as well. [1] https://apfel.franzai.com/ [2] https://github.com/danveloper/flash-moe", "context": "", "use_case": "MoE model inference", "switched_from": "", "story": "GitHub Copilot is moving to usage-based billing"}, {"id": 766169712, "item_id": 47885607, "url": "https://news.ycombinator.com/item?id=47885607", "by": "inventor7777", "date": "2026-04-23", "version": null, "stance": "neutral", "quote": "these were just two that I starred when I saw them posted here", "excerpt": "Sure! Note: these were just two that I starred when I saw them posted here. I have not looked seriously at it at the moment, https://github.com/danveloper/flash-moe https://github.com/t8/hypura", "context": "", "use_case": "", "switched_from": "", "story": "DeepSeek v4"}, {"id": 761690912, "item_id": 47605682, "url": "https://news.ycombinator.com/item?id=47605682", "by": "aegis_camera", "date": "2026-04-01", "version": null, "stance": "neutral", "quote": "Yes, this is a reference project", "excerpt": "Yes, this is a reference project, the main different is we don't use os swap ( it introduces latency, will add https://github.com/danveloper/flash-moe to the original reference as well ).", "context": "", "use_case": "reference implementation", "switched_from": "", "story": "TurboQuant KV Compression and SSD Expert Streaming for M5 Pro and IOS"}, {"id": 761682032, "item_id": 47605127, "url": "https://news.ycombinator.com/item?id=47605127", "by": "boogerlad", "date": "2026-04-01", "version": null, "stance": "neutral", "quote": "Does this use anything from the flash-moe project?", "excerpt": "Does this use anything from the flash-moe project? https://github.com/Alexintosh/flash-moe", "context": "", "use_case": "", "switched_from": "", "story": "TurboQuant KV Compression and SSD Expert Streaming for M5 Pro and IOS"}]}