{"slug": "omlx", "name": "oMLX", "category": "ai", "category_name": "AI models and assistants", "description": "Local LLM inference server for Apple Silicon Macs, built on Apple's MLX framework.", "recommenders": 9, "commenters": 16, "warners": 1, "score": 9.061, "rising": 1.04, "first": "2026-04-09", "latest": "2026-10-08", "why": "Recommended as an easy way to run local models on Apple Silicon Macs, with KV caching that persists to disk and a notable speed increase in v0.7.0.", "why_is_quote": false, "has_verdict": true, "sentiment": {"recommend": 9, "neutral": 6, "compare": 0, "warn": 1}, "spark": [0, 0, 0, 0, 0, 2, 0, 4, 0, 1, 1, 1], "listed": true, "loved": ["Mac local runtime", "Ease of use"], "caveat": "Qwen tool calls", "short_quote": "I switched to oMLX today from lm studio. Really nice", "icon": true, "domain": null, "github_repo": "github.com/jundot/omlx", "official_url": "https://github.com/jundot/omlx", "pricing": "open source", "aliases": ["oMLX", "omlx"], "verdict": [{"text": "Recommended as an easy way to run local models on Apple Silicon Macs,", "cites": [{"id": 792546002, "item_id": 49534125, "url": "https://news.ycombinator.com/item?id=49534125", "by": "ch_sm", "date": "2026-09-02", "stance": "recommend", "quote": "It\u2018s easy with omlx.", "excerpt": "It depends on your use case, but the smaller Gemma 4 models or qwen3.6:9b would probably run OK on that. I recommend trying it, even just for fun. It\u2018s easy with omlx.", "context": "", "use_case": "running local models", "switched_from": "", "story": "My local model setup on an M4 Pro Mac Mini"}, {"id": 779566400, "item_id": 48722900, "url": "https://news.ycombinator.com/item?id=48722900", "by": "blopker", "date": "2026-06-29", "stance": "recommend", "quote": "if you're getting started on a Mac, I'd suggest trying out oMLX", "excerpt": "I've been working with local models for the past year. There's so many possibilities, but I don't think coding is one. Coding requires so many layers beyond inference; I spent so much time trying to replicate what Claude Code does end to end locally. Understanding all the layers and keeping up with the advancements feels like a slog. Even this article messes up and misunderstands what some of the settings are doing. ", "context": "", "use_case": "running local models on Mac", "switched_from": "", "story": "Qwen 3.6 27B is the sweet spot for local development"}, {"id": 776124400, "item_id": 48507775, "url": "https://news.ycombinator.com/item?id=48507775", "by": "namnnumbr", "date": "2026-06-12", "stance": "recommend", "quote": "oMLX (https://github.com/jundot/omlx) makes running the mlx inference server quite easy", "excerpt": "oMLX (https://github.com/jundot/omlx) makes running the mlx inference server quite easy for those interested in UI-based hosting. oMLX also supports mtp or dflash drafting.", "context": "", "use_case": "running MLX inference server", "switched_from": "", "story": "How to Setup a Local Coding Agent on macOS"}, {"id": 764493264, "item_id": 47780829, "url": "https://news.ycombinator.com/item?id=47780829", "by": "anonyfox", "date": "2026-04-15", "stance": "recommend", "quote": "using oMLX as the runtime/server", "excerpt": "Macbook air M5 32gb ram here. using oMLX as the runtime/server and right now the perceived best model is Devstral-Small-2-24B-Instruct-2512-4bit for me, within opencode.", "context": "MacBook Air M5 32GB user", "use_case": "local model runtime", "switched_from": "", "story": "Ask HN: Which LLM model and agentic CLI are you using for local development?"}]}, {"text": " with KV caching that persists to disk and a notable speed increase in v0.7.0.", "cites": [{"id": 791818483, "item_id": 49488655, "url": "https://news.ycombinator.com/item?id=49488655", "by": "solarkraft", "date": "2026-08-29", "stance": "recommend", "quote": "For that I use omlx, which can persist the KV cache to disk", "excerpt": "NGL: I don\u2019t really have a good way to find out right now. It also doesn\u2019t matter that much because the way the models use the tokes varies a lot. Qwen 3.8 is known for overthinking while Muse Glimmer may be a little slower per token, but it uses them very efficiently, caveman style. Generation speed isn\u2019t the bottleneck anyway, at least on pre M4/M5 devices (the newer chips got significant processing acceleration). ", "context": "runs local models on Apple Silicon", "use_case": "local LLM serving with KV caching", "switched_from": "", "story": "GLM-5.3 is now open-weight"}, {"id": 799012385, "item_id": 49938274, "url": "https://news.ycombinator.com/item?id=49938274", "by": "vlowther", "date": "2026-10-02", "stance": "recommend", "quote": "the speed increase they have for the v0.7.0 release is amazeballs", "excerpt": "It is pretty nifty. I spend some time over last weekend implementing fused TQ to allow for 1m context lengths on a 128 gb MacBook M5 Max when using Qwen 3.8 flash next (https://github.com/antirez/ds4/pull/1115 if you are interested). If I get bored I might port over the Metal kernels from oMLX -- the speed increase they have for the v0.7.0 release is amazeballs.", "context": "", "use_case": "Metal kernels for inference", "switched_from": "", "story": "From the creator of Redis; run LLM locally with ds4"}]}, {"text": " The most common complaint is Qwen models failing to call tools correctly,", "cites": [{"id": 776134112, "item_id": 48508382, "url": "https://news.ycombinator.com/item?id=48508382", "by": "vorticalbox", "date": "2026-06-12", "stance": "recommend", "quote": "I switched to oMLX today from lm studio. Really nice", "excerpt": "I switched to oMLX today from lm studio. Really nice but I have found qwen3.6 sometimes failing to call tools correctly.", "context": "", "use_case": "local LLM inference", "switched_from": "LM Studio", "story": "Show HN: Script to bulk delete Claude chats from the web UI"}, {"id": 776157008, "item_id": 48509813, "url": "https://news.ycombinator.com/item?id=48509813", "by": "nozzlegear", "date": "2026-06-12", "stance": "neutral", "quote": "I use oMLX and also have this issue with qwen.", "excerpt": "I use oMLX and also have this issue with qwen. Not sure what it is about oMLX + qwen but they don't seem to play well together.", "context": "", "use_case": "local model serving", "switched_from": "", "story": "Show HN: Script to bulk delete Claude chats from the web UI"}]}, {"text": " along with reports of papercut bugs and silent failures.", "cites": [{"id": 783767712, "item_id": 48985482, "url": "https://news.ycombinator.com/item?id=48985482", "by": "jdiff", "date": "2026-07-20", "stance": "warn", "quote": "oMLX keeps driving me up a wall with odd papercuts, bugs, and silent failures", "excerpt": "I use oMLX and I'm tentatively going to be giving this a shot. oMLX keeps driving me up a wall with odd papercuts, bugs, and silent failures and fallbacks that are only visible buried deep inside logs when they should be announced out loud. The maintainer of mlx-vlm being behind this as well is the main thing kicking me over into trying it, even if it is incredibly young. I'm confused and unenthused to see it chompin", "context": "", "use_case": "running local MLX models", "switched_from": "", "story": "Nativ: Run frontier open models locally on your Mac"}]}], "praise": [{"label": "Mac local runtime", "summary": "Commenters use it as the runtime or server for local models on M-series Macs.", "quotes": [{"id": 764493264, "item_id": 47780829, "url": "https://news.ycombinator.com/item?id=47780829", "by": "anonyfox", "date": "2026-04-15", "stance": "recommend", "quote": "using oMLX as the runtime/server", "excerpt": "Macbook air M5 32gb ram here. using oMLX as the runtime/server and right now the perceived best model is Devstral-Small-2-24B-Instruct-2512-4bit for me, within opencode.", "context": "MacBook Air M5 32GB user", "use_case": "local model runtime", "switched_from": "", "story": "Ask HN: Which LLM model and agentic CLI are you using for local development?"}, {"id": 776230960, "item_id": 48514435, "url": "https://news.ycombinator.com/item?id=48514435", "by": "c0rruptbytes", "date": "2026-06-13", "stance": "recommend", "quote": "omlx + gemma 12b 6 bit + pi", "excerpt": "i\u2019m running m4 pro 48gb right now omlx + gemma 12b 6 bit + pi it\u2019s feasible for sure MoEs for speed (qwen 35b, cohere 30b, gemma 26b) Dense for more methodical work (qwen 27b [reigning champ], gemma 31b, gemma 12b) MoE i recommend 5bit+ Dense i think 4 bit is okay Play with your context size, you don\u2019t really need that much, have lazy loading for tools and mcps my pi extensions for anyone looking for a skinny quick s", "context": "M4 Pro 48GB user", "use_case": "running local models on Mac", "switched_from": "", "story": "How to Setup a Local Coding Agent on macOS"}, {"id": 763244323, "item_id": 47702770, "url": "https://news.ycombinator.com/item?id=47702770", "by": "dust42", "date": "2026-04-09", "stance": "neutral", "quote": "I used Qwen3.5-35B on Mac M1 Max with oMLX.", "excerpt": "I really love it. The simplicity is key. The first play project I made with it was a public transport map with GTFS data - click on a stop and get the routes and the timetables for the stop and the surrounding ones. I used Qwen3.5-35B on Mac M1 Max with oMLX. It wrote 98% of the code with very little interaction from me. And very useful is the /tree feature to go back in history when the model is on a wrong track or ", "context": "", "use_case": "running local models on Mac", "switched_from": "", "story": "Reallocating $100/Month Claude Code Spend to Zed and OpenRouter"}]}, {"label": "Ease of use", "summary": "Commenters say it makes running an MLX inference server easy, including for Mac beginners.", "quotes": [{"id": 792546002, "item_id": 49534125, "url": "https://news.ycombinator.com/item?id=49534125", "by": "ch_sm", "date": "2026-09-02", "stance": "recommend", "quote": "It\u2018s easy with omlx.", "excerpt": "It depends on your use case, but the smaller Gemma 4 models or qwen3.6:9b would probably run OK on that. I recommend trying it, even just for fun. It\u2018s easy with omlx.", "context": "", "use_case": "running local models", "switched_from": "", "story": "My local model setup on an M4 Pro Mac Mini"}, {"id": 776124400, "item_id": 48507775, "url": "https://news.ycombinator.com/item?id=48507775", "by": "namnnumbr", "date": "2026-06-12", "stance": "recommend", "quote": "oMLX (https://github.com/jundot/omlx) makes running the mlx inference server quite easy", "excerpt": "oMLX (https://github.com/jundot/omlx) makes running the mlx inference server quite easy for those interested in UI-based hosting. oMLX also supports mtp or dflash drafting.", "context": "", "use_case": "running MLX inference server", "switched_from": "", "story": "How to Setup a Local Coding Agent on macOS"}, {"id": 779566400, "item_id": 48722900, "url": "https://news.ycombinator.com/item?id=48722900", "by": "blopker", "date": "2026-06-29", "stance": "recommend", "quote": "if you're getting started on a Mac, I'd suggest trying out oMLX", "excerpt": "I've been working with local models for the past year. There's so many possibilities, but I don't think coding is one. Coding requires so many layers beyond inference; I spent so much time trying to replicate what Claude Code does end to end locally. Understanding all the layers and keeping up with the advancements feels like a slog. Even this article messes up and misunderstands what some of the settings are doing. ", "context": "", "use_case": "running local models on Mac", "switched_from": "", "story": "Qwen 3.6 27B is the sweet spot for local development"}]}, {"label": "Coding agents", "summary": "Commenters pair it with Claude Code, opencode and pi for local coding.", "quotes": [{"id": 766822000, "item_id": 47926375, "url": "https://news.ycombinator.com/item?id=47926375", "by": "jonaustin", "date": "2026-04-27", "stance": "recommend", "quote": "Try https://github.com/jundot/omlx", "excerpt": "Try https://github.com/jundot/omlx I've been using qwen3.5 (122b) with claude code for months, and it's definitely dumber than sonnet/opus, but it works through things reasonably well (i.e. writes half-decent code and tool calls usually work), and I pretty much never run into loops now. and make sure you're following Unsloth's recommendations for temperature/etc.", "context": "Runs local models with Claude Code for months", "use_case": "running local LLMs on Mac", "switched_from": "", "story": "Running Local LLMs Offline on a Ten-Hour Flight"}, {"id": 764493264, "item_id": 47780829, "url": "https://news.ycombinator.com/item?id=47780829", "by": "anonyfox", "date": "2026-04-15", "stance": "recommend", "quote": "using oMLX as the runtime/server", "excerpt": "Macbook air M5 32gb ram here. using oMLX as the runtime/server and right now the perceived best model is Devstral-Small-2-24B-Instruct-2512-4bit for me, within opencode.", "context": "MacBook Air M5 32GB user", "use_case": "local model runtime", "switched_from": "", "story": "Ask HN: Which LLM model and agentic CLI are you using for local development?"}, {"id": 776230960, "item_id": 48514435, "url": "https://news.ycombinator.com/item?id=48514435", "by": "c0rruptbytes", "date": "2026-06-13", "stance": "recommend", "quote": "omlx + gemma 12b 6 bit + pi", "excerpt": "i\u2019m running m4 pro 48gb right now omlx + gemma 12b 6 bit + pi it\u2019s feasible for sure MoEs for speed (qwen 35b, cohere 30b, gemma 26b) Dense for more methodical work (qwen 27b [reigning champ], gemma 31b, gemma 12b) MoE i recommend 5bit+ Dense i think 4 bit is okay Play with your context size, you don\u2019t really need that much, have lazy loading for tools and mcps my pi extensions for anyone looking for a skinny quick s", "context": "M4 Pro 48GB user", "use_case": "running local models on Mac", "switched_from": "", "story": "How to Setup a Local Coding Agent on macOS"}]}, {"label": "Prompt caching", "summary": "It can persist the KV cache to disk, which helps with long system prompts.", "quotes": [{"id": 791818483, "item_id": 49488655, "url": "https://news.ycombinator.com/item?id=49488655", "by": "solarkraft", "date": "2026-08-29", "stance": "recommend", "quote": "For that I use omlx, which can persist the KV cache to disk", "excerpt": "NGL: I don\u2019t really have a good way to find out right now. It also doesn\u2019t matter that much because the way the models use the tokes varies a lot. Qwen 3.8 is known for overthinking while Muse Glimmer may be a little slower per token, but it uses them very efficiently, caveman style. Generation speed isn\u2019t the bottleneck anyway, at least on pre M4/M5 devices (the newer chips got significant processing acceleration). ", "context": "runs local models on Apple Silicon", "use_case": "local LLM serving with KV caching", "switched_from": "", "story": "GLM-5.3 is now open-weight"}]}, {"label": "Speed", "summary": "One commenter calls the v0.7.0 speed increase impressive.", "quotes": [{"id": 799012385, "item_id": 49938274, "url": "https://news.ycombinator.com/item?id=49938274", "by": "vlowther", "date": "2026-10-02", "stance": "recommend", "quote": "the speed increase they have for the v0.7.0 release is amazeballs", "excerpt": "It is pretty nifty. I spend some time over last weekend implementing fused TQ to allow for 1m context lengths on a 128 gb MacBook M5 Max when using Qwen 3.8 flash next (https://github.com/antirez/ds4/pull/1115 if you are interested). If I get bored I might port over the Metal kernels from oMLX -- the speed increase they have for the v0.7.0 release is amazeballs.", "context": "", "use_case": "Metal kernels for inference", "switched_from": "", "story": "From the creator of Redis; run LLM locally with ds4"}]}, {"label": "Speculative decoding", "summary": "It supports MTP or DFlash drafting.", "quotes": [{"id": 776124400, "item_id": 48507775, "url": "https://news.ycombinator.com/item?id=48507775", "by": "namnnumbr", "date": "2026-06-12", "stance": "recommend", "quote": "oMLX (https://github.com/jundot/omlx) makes running the mlx inference server quite easy", "excerpt": "oMLX (https://github.com/jundot/omlx) makes running the mlx inference server quite easy for those interested in UI-based hosting. oMLX also supports mtp or dflash drafting.", "context": "", "use_case": "running MLX inference server", "switched_from": "", "story": "How to Setup a Local Coding Agent on macOS"}]}], "caveats": [{"label": "Qwen tool calls", "summary": "Users report Qwen models sometimes fail to call tools correctly with it.", "quotes": [{"id": 776134112, "item_id": 48508382, "url": "https://news.ycombinator.com/item?id=48508382", "by": "vorticalbox", "date": "2026-06-12", "stance": "recommend", "quote": "I switched to oMLX today from lm studio. Really nice", "excerpt": "I switched to oMLX today from lm studio. Really nice but I have found qwen3.6 sometimes failing to call tools correctly.", "context": "", "use_case": "local LLM inference", "switched_from": "LM Studio", "story": "Show HN: Script to bulk delete Claude chats from the web UI"}, {"id": 776157008, "item_id": 48509813, "url": "https://news.ycombinator.com/item?id=48509813", "by": "nozzlegear", "date": "2026-06-12", "stance": "neutral", "quote": "I use oMLX and also have this issue with qwen.", "excerpt": "I use oMLX and also have this issue with qwen. Not sure what it is about oMLX + qwen but they don't seem to play well together.", "context": "", "use_case": "local model serving", "switched_from": "", "story": "Show HN: Script to bulk delete Claude chats from the web UI"}]}, {"label": "Bugs, silent failures", "summary": "One user reports papercuts, bugs, and silent fallbacks that show up only in logs.", "quotes": [{"id": 783767712, "item_id": 48985482, "url": "https://news.ycombinator.com/item?id=48985482", "by": "jdiff", "date": "2026-07-20", "stance": "warn", "quote": "oMLX keeps driving me up a wall with odd papercuts, bugs, and silent failures", "excerpt": "I use oMLX and I'm tentatively going to be giving this a shot. oMLX keeps driving me up a wall with odd papercuts, bugs, and silent failures and fallbacks that are only visible buried deep inside logs when they should be announced out loud. The maintainer of mlx-vlm being behind this as well is the main thing kicking me over into trying it, even if it is incredibly young. I'm confused and unenthused to see it chompin", "context": "", "use_case": "running local MLX models", "switched_from": "", "story": "Nativ: Run frontier open models locally on your Mac"}]}, {"label": "Slow generation", "summary": "One user got 11 tokens per second on an M5 Pro and found local coding too slow.", "quotes": [{"id": 765915076, "item_id": 47869692, "url": "https://news.ycombinator.com/item?id=47869692", "by": "datadrivenangel", "date": "2026-04-22", "stance": "neutral", "quote": "got 11 tokens per second on omlx on an M5 Pro with 128gb of ram", "excerpt": "So far I'm unimpressed for local inference. got 11 tokens per second on omlx on an M5 Pro with 128gb of ram, so it took an hour to write a few hundred lines of code that didn't work. Opus and Sonnet in CC the same task successfully in a matter of minutes. The 3.6:35b model seemed okay on ollama yesterday. Need to check out other harnesses for this besides claude code, but the local models are just painfully slow.", "context": "", "use_case": "local inference", "switched_from": "", "story": "Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model"}]}], "who": [{"by": "solarkraft", "context": "runs local models on Apple Silicon", "url": "https://news.ycombinator.com/item?id=49488655", "date": "2026-08-29"}, {"by": "c0rruptbytes", "context": "M4 Pro 48GB user", "url": "https://news.ycombinator.com/item?id=48514435", "date": "2026-06-13"}, {"by": "jonaustin", "context": "Runs local models with Claude Code for months", "url": "https://news.ycombinator.com/item?id=47926375", "date": "2026-04-27"}, {"by": "anonyfox", "context": "MacBook Air M5 32GB user", "url": "https://news.ycombinator.com/item?id=47780829", "date": "2026-04-15"}], "tags": ["for Mac users", "for local LLMs", "for coding agents", "for LM Studio switchers", "for code audits"], "use_cases": ["running local models on mac", "running local models", "running local models for agents", "metal kernels for inference", "local llm serving with kv caching", "local model runner"], "compared": [{"slug": "qwen-3-6", "name": "Qwen 3.6", "co": 5, "switched_from": 0, "switched_to": 0}, {"slug": "lm-studio", "name": "LM Studio", "co": 2, "switched_from": 1, "switched_to": 0}, {"slug": "opencode", "name": "OpenCode", "co": 3, "switched_from": 0, "switched_to": 0}, {"slug": "qwen-3-8", "name": "Qwen 3.8", "co": 3, "switched_from": 0, "switched_to": 0}, {"slug": "mtplx", "name": "MTPLX", "co": 1, "switched_from": 0, "switched_to": 1}, {"slug": "qwen", "name": "Qwen", "co": 3, "switched_from": 0, "switched_to": 0}, {"slug": "pi", "name": "Pi", "co": 3, "switched_from": 0, "switched_to": 0}, {"slug": "ollama", "name": "Ollama", "co": 3, "switched_from": 0, "switched_to": 0}], "makers": [], "threads": [{"url": "https://news.ycombinator.com/item?id=48507020", "title": "How to Setup a Local Coding Agent on macOS", "mentions": 2}, {"url": "https://news.ycombinator.com/item?id=48505161", "title": "Show HN: Script to bulk delete Claude chats from the web UI", "mentions": 2}, {"url": "https://news.ycombinator.com/item?id=47863217", "title": "Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model", "mentions": 2}, {"url": "https://news.ycombinator.com/item?id=50011229", "title": "Ask HN: Got a Mac for running agents? what's your setup?", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=49936575", "title": "From the creator of Redis; run LLM locally with ds4", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=49529132", "title": "My local model setup on an M4 Pro Mac Mini", "mentions": 1}], "sentiment_by_month": {"2025-11": {}, "2025-12": {}, "2026-01": {}, "2026-02": {}, "2026-03": {}, "2026-04": {"neutral": 3, "recommend": 2}, "2026-05": {}, "2026-06": {"recommend": 4, "neutral": 1}, "2026-07": {"warn": 1}, "2026-08": {"neutral": 1, "recommend": 1}, "2026-09": {"recommend": 1}, "2026-10": {"recommend": 1, "neutral": 1}}, "quotes": [{"id": 800188176, "item_id": 50011761, "url": "https://news.ycombinator.com/item?id=50011761", "by": "krees", "date": "2026-10-08", "stance": "neutral", "quote": "Mac M3, oMLX, pi. 64GB RAM. Qwen 3.8 27b / Gemma 12b", "excerpt": "Mac M3, oMLX, pi. 64GB RAM. Qwen 3.8 27b / Gemma 12b", "context": "", "use_case": "running local models for agents", "switched_from": "", "story": "Ask HN: Got a Mac for running agents? what's your setup?"}, {"id": 799012385, "item_id": 49938274, "url": "https://news.ycombinator.com/item?id=49938274", "by": "vlowther", "date": "2026-10-02", "stance": "recommend", "quote": "the speed increase they have for the v0.7.0 release is amazeballs", "excerpt": "It is pretty nifty. I spend some time over last weekend implementing fused TQ to allow for 1m context lengths on a 128 gb MacBook M5 Max when using Qwen 3.8 flash next (https://github.com/antirez/ds4/pull/1115 if you are interested). If I get bored I might port over the Metal kernels from oMLX -- the speed increase they have for the v0.7.0 release is amazeballs.", "context": "", "use_case": "Metal kernels for inference", "switched_from": "", "story": "From the creator of Redis; run LLM locally with ds4"}, {"id": 792546002, "item_id": 49534125, "url": "https://news.ycombinator.com/item?id=49534125", "by": "ch_sm", "date": "2026-09-02", "stance": "recommend", "quote": "It\u2018s easy with omlx.", "excerpt": "It depends on your use case, but the smaller Gemma 4 models or qwen3.6:9b would probably run OK on that. I recommend trying it, even just for fun. It\u2018s easy with omlx.", "context": "", "use_case": "running local models", "switched_from": "", "story": "My local model setup on an M4 Pro Mac Mini"}, {"id": 791818483, "item_id": 49488655, "url": "https://news.ycombinator.com/item?id=49488655", "by": "solarkraft", "date": "2026-08-29", "stance": "recommend", "quote": "For that I use omlx, which can persist the KV cache to disk", "excerpt": "NGL: I don\u2019t really have a good way to find out right now. It also doesn\u2019t matter that much because the way the models use the tokes varies a lot. Qwen 3.8 is known for overthinking while Muse Glimmer may be a little slower per token, but it uses them very efficiently, caveman style. Generation speed isn\u2019t the bottleneck anyway, at least on pre M4/M5 devices (the newer chips got significant processing acceleration). ", "context": "runs local models on Apple Silicon", "use_case": "local LLM serving with KV caching", "switched_from": "", "story": "GLM-5.3 is now open-weight"}, {"id": 790955508, "item_id": 49434719, "url": "https://news.ycombinator.com/item?id=49434719", "by": "jlokier", "date": "2026-08-25", "stance": "neutral", "quote": "Others use LM Studio, oMLX, etc.", "excerpt": "You can't run Linux directly on these. Asahi Linux supports up to M2 only. Linux runs very well in a VM on macOS. There are many good options for this, some free and open source (QEMU, UTM, Lima, Colima), some proprietary (VMware Fusion, Parallels). But Linux in a VM doesn't get access to the real GPU, so model performance is limited. Those running on the CPU perform well, and those needing the GPU don't. However, ma", "context": "", "use_case": "local model runner", "switched_from": "", "story": "Apple introduces M6 and M5 Ultra"}, {"id": 783767712, "item_id": 48985482, "url": "https://news.ycombinator.com/item?id=48985482", "by": "jdiff", "date": "2026-07-20", "stance": "warn", "quote": "oMLX keeps driving me up a wall with odd papercuts, bugs, and silent failures", "excerpt": "I use oMLX and I'm tentatively going to be giving this a shot. oMLX keeps driving me up a wall with odd papercuts, bugs, and silent failures and fallbacks that are only visible buried deep inside logs when they should be announced out loud. The maintainer of mlx-vlm being behind this as well is the main thing kicking me over into trying it, even if it is incredibly young. I'm confused and unenthused to see it chompin", "context": "", "use_case": "running local MLX models", "switched_from": "", "story": "Nativ: Run frontier open models locally on your Mac"}, {"id": 779566400, "item_id": 48722900, "url": "https://news.ycombinator.com/item?id=48722900", "by": "blopker", "date": "2026-06-29", "stance": "recommend", "quote": "if you're getting started on a Mac, I'd suggest trying out oMLX", "excerpt": "I've been working with local models for the past year. There's so many possibilities, but I don't think coding is one. Coding requires so many layers beyond inference; I spent so much time trying to replicate what Claude Code does end to end locally. Understanding all the layers and keeping up with the advancements feels like a slog. Even this article messes up and misunderstands what some of the settings are doing. ", "context": "", "use_case": "running local models on Mac", "switched_from": "", "story": "Qwen 3.6 27B is the sweet spot for local development"}, {"id": 776230960, "item_id": 48514435, "url": "https://news.ycombinator.com/item?id=48514435", "by": "c0rruptbytes", "date": "2026-06-13", "stance": "recommend", "quote": "omlx + gemma 12b 6 bit + pi", "excerpt": "i\u2019m running m4 pro 48gb right now omlx + gemma 12b 6 bit + pi it\u2019s feasible for sure MoEs for speed (qwen 35b, cohere 30b, gemma 26b) Dense for more methodical work (qwen 27b [reigning champ], gemma 31b, gemma 12b) MoE i recommend 5bit+ Dense i think 4 bit is okay Play with your context size, you don\u2019t really need that much, have lazy loading for tools and mcps my pi extensions for anyone looking for a skinny quick s", "context": "M4 Pro 48GB user", "use_case": "running local models on Mac", "switched_from": "", "story": "How to Setup a Local Coding Agent on macOS"}, {"id": 776157008, "item_id": 48509813, "url": "https://news.ycombinator.com/item?id=48509813", "by": "nozzlegear", "date": "2026-06-12", "stance": "neutral", "quote": "I use oMLX and also have this issue with qwen.", "excerpt": "I use oMLX and also have this issue with qwen. Not sure what it is about oMLX + qwen but they don't seem to play well together.", "context": "", "use_case": "local model serving", "switched_from": "", "story": "Show HN: Script to bulk delete Claude chats from the web UI"}, {"id": 776134112, "item_id": 48508382, "url": "https://news.ycombinator.com/item?id=48508382", "by": "vorticalbox", "date": "2026-06-12", "stance": "recommend", "quote": "I switched to oMLX today from lm studio. Really nice", "excerpt": "I switched to oMLX today from lm studio. Really nice but I have found qwen3.6 sometimes failing to call tools correctly.", "context": "", "use_case": "local LLM inference", "switched_from": "LM Studio", "story": "Show HN: Script to bulk delete Claude chats from the web UI"}, {"id": 776124400, "item_id": 48507775, "url": "https://news.ycombinator.com/item?id=48507775", "by": "namnnumbr", "date": "2026-06-12", "stance": "recommend", "quote": "oMLX (https://github.com/jundot/omlx) makes running the mlx inference server quite easy", "excerpt": "oMLX (https://github.com/jundot/omlx) makes running the mlx inference server quite easy for those interested in UI-based hosting. oMLX also supports mtp or dflash drafting.", "context": "", "use_case": "running MLX inference server", "switched_from": "", "story": "How to Setup a Local Coding Agent on macOS"}, {"id": 766822000, "item_id": 47926375, "url": "https://news.ycombinator.com/item?id=47926375", "by": "jonaustin", "date": "2026-04-27", "stance": "recommend", "quote": "Try https://github.com/jundot/omlx", "excerpt": "Try https://github.com/jundot/omlx I've been using qwen3.5 (122b) with claude code for months, and it's definitely dumber than sonnet/opus, but it works through things reasonably well (i.e. writes half-decent code and tool calls usually work), and I pretty much never run into loops now. and make sure you're following Unsloth's recommendations for temperature/etc.", "context": "Runs local models with Claude Code for months", "use_case": "running local LLMs on Mac", "switched_from": "", "story": "Running Local LLMs Offline on a Ten-Hour Flight"}, {"id": 765915076, "item_id": 47869692, "url": "https://news.ycombinator.com/item?id=47869692", "by": "datadrivenangel", "date": "2026-04-22", "stance": "neutral", "quote": "got 11 tokens per second on omlx on an M5 Pro with 128gb of ram", "excerpt": "So far I'm unimpressed for local inference. got 11 tokens per second on omlx on an M5 Pro with 128gb of ram, so it took an hour to write a few hundred lines of code that didn't work. Opus and Sonnet in CC the same task successfully in a matter of minutes. The 3.6:35b model seemed okay on ollama yesterday. Need to check out other harnesses for this besides claude code, but the local models are just painfully slow.", "context": "", "use_case": "local inference", "switched_from": "", "story": "Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model"}, {"id": 765874706, "item_id": 47867169, "url": "https://news.ycombinator.com/item?id=47867169", "by": "jedisct1", "date": "2026-04-22", "stance": "neutral", "quote": "I used Qwen3.6-27B on an M5 (oq8, using omlx)", "excerpt": "I really like local models for code reviews / security audits. Even if they don't run super fast, I can let them work overnight and get comprehensive reports in the morning. I used Qwen3.6-27B on an M5 (oq8, using omlx) and Swival (https://swival.dev) /audit command on small code bases I use for benchmarking models for security audits. It found 8 out of 10, which is excellent for a local model, produced valid patches", "context": "", "use_case": "running local models", "switched_from": "", "story": "Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model"}, {"id": 764493264, "item_id": 47780829, "url": "https://news.ycombinator.com/item?id=47780829", "by": "anonyfox", "date": "2026-04-15", "stance": "recommend", "quote": "using oMLX as the runtime/server", "excerpt": "Macbook air M5 32gb ram here. using oMLX as the runtime/server and right now the perceived best model is Devstral-Small-2-24B-Instruct-2512-4bit for me, within opencode.", "context": "MacBook Air M5 32GB user", "use_case": "local model runtime", "switched_from": "", "story": "Ask HN: Which LLM model and agentic CLI are you using for local development?"}, {"id": 763244323, "item_id": 47702770, "url": "https://news.ycombinator.com/item?id=47702770", "by": "dust42", "date": "2026-04-09", "stance": "neutral", "quote": "I used Qwen3.5-35B on Mac M1 Max with oMLX.", "excerpt": "I really love it. The simplicity is key. The first play project I made with it was a public transport map with GTFS data - click on a stop and get the routes and the timetables for the stop and the surrounding ones. I used Qwen3.5-35B on Mac M1 Max with oMLX. It wrote 98% of the code with very little interaction from me. And very useful is the /tree feature to go back in history when the model is on a wrong track or ", "context": "", "use_case": "running local models on Mac", "switched_from": "", "story": "Reallocating $100/Month Claude Code Spend to Zed and OpenRouter"}]}