{"slug": "onnx", "name": "ONNX", "category": "dev-libs", "category_name": "Libraries and frameworks", "description": "", "recommenders": 3, "commenters": 8, "warners": 2, "score": 1.801, "rising": 0.35, "first": "2025-12-25", "latest": "2026-10-07", "why": "We use the ONNX runtime for small models in the browser", "why_is_quote": true, "has_verdict": false, "sentiment": {"recommend": 3, "neutral": 3, "compare": 0, "warn": 2}, "spark": [0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0], "listed": true, "loved": [], "caveat": "", "short_quote": "We use the ONNX runtime for small models in the browser", "icon": true, "domain": null, "github_repo": "github.com/microsoft/onnxruntime", "official_url": "https://onnxruntime.ai", "pricing": "", "aliases": ["ONNX", "ONNX Runtime"], "verdict": null, "praise": [{"label": "", "summary": "", "quotes": [{"id": 792599744, "item_id": 49537484, "url": "https://news.ycombinator.com/item?id=49537484", "by": "seamossfet", "date": "2026-09-02", "stance": "recommend", "quote": "We use the ONNX runtime for small models in the browser", "excerpt": "We use the ONNX runtime for small models in the browser https://github.com/microsoft/onnxruntime", "context": "", "use_case": "in-browser small model inference", "switched_from": "", "story": "WebLLM: high-performance in-browser LLM inference engine"}, {"id": 775359616, "item_id": 48459976, "url": "https://news.ycombinator.com/item?id=48459976", "by": "antonvs", "date": "2026-06-09", "stance": "recommend", "quote": "We use this in production:", "excerpt": "We use this in production: https://docs.rs/onnxruntime/latest/onnxruntime/ It\u2019s a Rust wrapper around ONNX Runtime. We currently serve 5+ million inference requests per day for a highly performance-sensitive application, for a long list of major enterprise clients. We don\u2019t use GPUs for inference, because it would be cost-prohibitive. We launch tens of thousands of VMs per day to run these workloads.", "context": "serves 5+ million inference requests per day", "use_case": "production ML inference on CPU", "switched_from": "", "story": "OpenCV 5 Is Here: The Biggest Leap in Years for Computer Vision"}, {"id": 744288176, "item_id": 46518011, "url": "https://news.ycombinator.com/item?id=46518011", "by": "sroussey", "date": "2026-01-06", "stance": "recommend", "quote": "onnx on the web has the most models available and can use webgpu which is available everywhere.", "excerpt": "onnx on the web has the most models available and can use webgpu which is available everywhere. Huggingface\u2019s transformers.js uses it. And I use that for https://workglow.dev (also tensorflow mediapipe though that is using wasm). I don\u2019t think webnn has gone anywhere and is too restrictive.", "context": "", "use_case": "ML models in the browser", "switched_from": "", "story": "Show HN: Jax-JS, array library in JavaScript targeting WebGPU"}]}], "caveats": [{"label": "", "summary": "", "quotes": [{"id": 783431010, "item_id": 48964438, "url": "https://news.ycombinator.com/item?id=48964438", "by": "shade", "date": "2026-07-18", "stance": "warn", "quote": "dealing with ONNX is kind of annoying", "excerpt": "Nice - I'm definitely going to take a look at this. I've built my own cross-platform (Mac/Win/Linux) live captioning app on top of Nemotron, and it works well but dealing with ONNX is kind of annoying. With this having Rust support (I built it on Rust/Tauri) it should be a pretty solid candidate; I'll have to see if I can find a Silero VAD implementation that doesn't depend on ONNX, or maybe I'll see if the clankers ", "context": "built a cross-platform live captioning app", "use_case": "model runtime", "switched_from": "", "story": "Transcribe.cpp"}, {"id": 742153264, "item_id": 46384579, "url": "https://news.ycombinator.com/item?id=46384579", "by": "nuc1e0n", "date": "2025-12-25", "stance": "warn", "quote": "My experiences with ONNX have not been pleasant. Conversions from models written with Tensorflow and Pytorch often fail.", "excerpt": "My experiences with ONNX have not been pleasant. Conversions from models written with Tensorflow and Pytorch often fail. I recommend using TFLite or Executorch for deployment to edge devices instead.", "context": "", "use_case": "model conversion for edge deployment", "switched_from": "", "story": "ONNX Runtime and CoreML May Silently Convert Your Model to FP16"}]}], "who": [{"by": "antonvs", "context": "serves 5+ million inference requests per day", "url": "https://news.ycombinator.com/item?id=48459976", "date": "2026-06-09"}], "tags": [], "use_cases": ["entity scoring for voice control", "in-browser small model inference", "running model in rust simulator", "model runtime", "running model in browser", "production ml inference on cpu"], "compared": [], "makers": [], "threads": [{"url": "https://news.ycombinator.com/item?id=49986765", "title": "Jev-Driven SRE Diagnosis: What Worked and What Failed", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=49536411", "title": "WebLLM: high-performance in-browser LLM inference engine", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=49113999", "title": "Show HN: NegativeEV \u2013 I built a tool to help people see how bad their bets are", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=48963879", "title": "Transcribe.cpp", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=48630171", "title": "Moebius: 0.2B image inpainting model with 10B-level performance", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=48421858", "title": "OpenCV 5 Is Here: The Biggest Leap in Years for Computer Vision", "mentions": 1}], "sentiment_by_month": {"2025-11": {}, "2025-12": {"warn": 1}, "2026-01": {"recommend": 1}, "2026-02": {}, "2026-03": {}, "2026-04": {}, "2026-05": {}, "2026-06": {"recommend": 1, "neutral": 1}, "2026-07": {"warn": 1, "neutral": 1}, "2026-08": {}, "2026-09": {"recommend": 1}, "2026-10": {"neutral": 1}}, "quotes": [{"id": 799829985, "item_id": 49989374, "url": "https://news.ycombinator.com/item?id=49989374", "by": "amne", "date": "2026-10-07", "stance": "neutral", "quote": "onnx scores all my HA entities to determine which one I'm talking about", "excerpt": "I switched to a decision model for my local \"AI\" voice control thingy. faster-whisper produces the text then onnx scores all my HA entities to determine which one I'm talking about .. then it lists all its actions and a second scoring round to determine what action I want. some regex to extract numbers if I'm saying things like \"water the lawn for 10 minutes\". It's pretty dumb when compared to what an LLM can do but ", "context": "", "use_case": "entity scoring for voice control", "switched_from": "", "story": "Jev-Driven SRE Diagnosis: What Worked and What Failed"}, {"id": 792599744, "item_id": 49537484, "url": "https://news.ycombinator.com/item?id=49537484", "by": "seamossfet", "date": "2026-09-02", "stance": "recommend", "quote": "We use the ONNX runtime for small models in the browser", "excerpt": "We use the ONNX runtime for small models in the browser https://github.com/microsoft/onnxruntime", "context": "", "use_case": "in-browser small model inference", "switched_from": "", "story": "WebLLM: high-performance in-browser LLM inference engine"}, {"id": 785824307, "item_id": 49114019, "url": "https://news.ycombinator.com/item?id=49114019", "by": "qkwrv", "date": "2026-07-30", "stance": "neutral", "quote": "it's so the Rust simulator can run the model itself, with no Python in the loop", "excerpt": "I know I say it in the title, but I want to be very clear up front: the predominant output of this tool is \"don't place this bet.\" Betting apps want you to bet. If they sense you are tired of losing and might leave, they'll entice you with deposit bonuses, boosted odds and protected picks. There are even people, you see them self-promoting all over X, who sell access to \"good\" bets. Obviously, they don't refund you i", "context": "", "use_case": "running model in Rust simulator", "switched_from": "", "story": "Show HN: NegativeEV \u2013 I built a tool to help people see how bad their bets are"}, {"id": 783431010, "item_id": 48964438, "url": "https://news.ycombinator.com/item?id=48964438", "by": "shade", "date": "2026-07-18", "stance": "warn", "quote": "dealing with ONNX is kind of annoying", "excerpt": "Nice - I'm definitely going to take a look at this. I've built my own cross-platform (Mac/Win/Linux) live captioning app on top of Nemotron, and it works well but dealing with ONNX is kind of annoying. With this having Rust support (I built it on Rust/Tauri) it should be a pretty solid candidate; I'll have to see if I can find a Silero VAD implementation that doesn't depend on ONNX, or maybe I'll see if the clankers ", "context": "built a cross-platform live captioning app", "use_case": "model runtime", "switched_from": "", "story": "Transcribe.cpp"}, {"id": 778192433, "item_id": 48637027, "url": "https://news.ycombinator.com/item?id=48637027", "by": "simonw", "date": "2026-06-22", "stance": "neutral", "quote": "I got this working with ONNX", "excerpt": "I got this working with ONNX (thanks, Claude Opus 4.8) and now I have an interactive demo of the model running entirely in the browser here (~1.3GB download): https://simonw.github.io/moebius-web/ - code here: https://github.com/simonw/moebius-web (Claude Code transcript: https://gisthost.github.io/?58039ba5c1ca3ed177e8659168996ee4)", "context": "", "use_case": "running model in browser", "switched_from": "", "story": "Moebius: 0.2B image inpainting model with 10B-level performance"}, {"id": 775359616, "item_id": 48459976, "url": "https://news.ycombinator.com/item?id=48459976", "by": "antonvs", "date": "2026-06-09", "stance": "recommend", "quote": "We use this in production:", "excerpt": "We use this in production: https://docs.rs/onnxruntime/latest/onnxruntime/ It\u2019s a Rust wrapper around ONNX Runtime. We currently serve 5+ million inference requests per day for a highly performance-sensitive application, for a long list of major enterprise clients. We don\u2019t use GPUs for inference, because it would be cost-prohibitive. We launch tens of thousands of VMs per day to run these workloads.", "context": "serves 5+ million inference requests per day", "use_case": "production ML inference on CPU", "switched_from": "", "story": "OpenCV 5 Is Here: The Biggest Leap in Years for Computer Vision"}, {"id": 744288176, "item_id": 46518011, "url": "https://news.ycombinator.com/item?id=46518011", "by": "sroussey", "date": "2026-01-06", "stance": "recommend", "quote": "onnx on the web has the most models available and can use webgpu which is available everywhere.", "excerpt": "onnx on the web has the most models available and can use webgpu which is available everywhere. Huggingface\u2019s transformers.js uses it. And I use that for https://workglow.dev (also tensorflow mediapipe though that is using wasm). I don\u2019t think webnn has gone anywhere and is too restrictive.", "context": "", "use_case": "ML models in the browser", "switched_from": "", "story": "Show HN: Jax-JS, array library in JavaScript targeting WebGPU"}, {"id": 742153264, "item_id": 46384579, "url": "https://news.ycombinator.com/item?id=46384579", "by": "nuc1e0n", "date": "2025-12-25", "stance": "warn", "quote": "My experiences with ONNX have not been pleasant. Conversions from models written with Tensorflow and Pytorch often fail.", "excerpt": "My experiences with ONNX have not been pleasant. Conversions from models written with Tensorflow and Pytorch often fail. I recommend using TFLite or Executorch for deployment to edge devices instead.", "context": "", "use_case": "model conversion for edge deployment", "switched_from": "", "story": "ONNX Runtime and CoreML May Silently Convert Your Model to FP16"}]}