oMLX
Local LLM inference server for Apple Silicon Macs, built on Apple's MLX framework.
Recommended as an easy way to run local models on Apple Silicon Macs,1234 with KV caching that persists to disk and a notable speed increase in v0.7.0.56 The most common complaint is Qwen models failing to call tools correctly,78 along with reports of papercut bugs and silent failures.9
Why people recommend it
Mac local runtime · 3
Commenters use it as the runtime or server for local models on M-series Macs.
“using oMLX as the runtime/server”
“omlx + gemma 12b 6 bit + pi”
Show 1 more
“I used Qwen3.5-35B on Mac M1 Max with oMLX.”
Ease of use · 3
Commenters say it makes running an MLX inference server easy, including for Mac beginners.
“oMLX (https://github.com/jundot/omlx) makes running the mlx inference server quite easy”
Show 1 more
“if you're getting started on a Mac, I'd suggest trying out oMLX”
Coding agents · 3
Commenters pair it with Claude Code, opencode and pi for local coding.
“Try https://github.com/jundot/omlx”
“using oMLX as the runtime/server”
Show 1 more
“omlx + gemma 12b 6 bit + pi”
Prompt caching · 1
It can persist the KV cache to disk, which helps with long system prompts.
“For that I use omlx, which can persist the KV cache to disk”
Speed · 1
One commenter calls the v0.7.0 speed increase impressive.
“the speed increase they have for the v0.7.0 release is amazeballs”
Speculative decoding · 1
It supports MTP or DFlash drafting.
“oMLX (https://github.com/jundot/omlx) makes running the mlx inference server quite easy”
Caveats and dissent
Qwen tool calls · 2
Users report Qwen models sometimes fail to call tools correctly with it.
“I switched to oMLX today from lm studio. Really nice”
“I use oMLX and also have this issue with qwen.”
Bugs, silent failures · 1
One user reports papercuts, bugs, and silent fallbacks that show up only in logs.
“oMLX keeps driving me up a wall with odd papercuts, bugs, and silent failures”
Slow generation · 1
One user got 11 tokens per second on an M5 Pro and found local coding too slow.
“got 11 tokens per second on omlx on an M5 Pro with 128gb of ram”
Often compared with
Who recommends it in their own words
Threads where it came up
- How to Setup a Local Coding Agent on macOS 2 mentions
- Show HN: Script to bulk delete Claude chats from the web UI 2 mentions
- Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model 2 mentions
- Ask HN: Got a Mac for running agents? what's your setup? 1 mention
- From the creator of Redis; run LLM locally with ds4 1 mention
- My local model setup on an M4 Pro Mac Mini 1 mention
Also written as omlx. Data: omlx.json. Quotes are verbatim; the verdict and theme labels are written by a model from these comments, and every clause links to its sources.
All comments 16
“Mac M3, oMLX, pi. 64GB RAM. Qwen 3.8 27b / Gemma 12b”
“the speed increase they have for the v0.7.0 release is amazeballs”
“It‘s easy with omlx.”
“For that I use omlx, which can persist the KV cache to disk”
“Others use LM Studio, oMLX, etc.”
“oMLX keeps driving me up a wall with odd papercuts, bugs, and silent failures”
“if you're getting started on a Mac, I'd suggest trying out oMLX”
“omlx + gemma 12b 6 bit + pi”
“I use oMLX and also have this issue with qwen.”
“I switched to oMLX today from lm studio. Really nice”
“oMLX (https://github.com/jundot/omlx) makes running the mlx inference server quite easy”
“Try https://github.com/jundot/omlx”
“got 11 tokens per second on omlx on an M5 Pro with 128gb of ram”
“I used Qwen3.6-27B on an M5 (oq8, using omlx)”
“using oMLX as the runtime/server”
“I used Qwen3.5-35B on Mac M1 Max with oMLX.”