{"slug": "onnx-runtime", "name": "ONNX Runtime", "category": "ai", "category_name": "AI tools and apps", "description": "", "recommenders": 6, "commenters": 26, "warners": 6, "score": 3.369, "rising": -0.64, "first": "2025-11-14", "latest": "2026-10-22", "why": "We use the ONNX runtime for small models in the browser", "why_is_quote": true, "has_verdict": false, "sentiment": {"recommend": 6, "neutral": 10, "compare": 3, "warn": 8}, "spark": [1, 0, 1, 0, 1, 0, 1, 1, 0, 0, 1, 0], "listed": true, "loved": [], "caveat": "", "short_quote": "We use the ONNX runtime for small models in the browser", "kind": "", "icon": true, "domain": null, "github_repo": "github.com/microsoft/onnxruntime", "official_url": "https://onnxruntime.ai", "pricing": "", "aliases": ["ONNX Runtime", "ONNX"], "verdict": null, "praise": [{"label": "", "summary": "", "quotes": [{"id": 792599744, "item_id": 49537484, "url": "https://news.ycombinator.com/item?id=49537484", "by": "seamossfet", "date": "2026-09-02", "version": null, "stance": "recommend", "quote": "We use the ONNX runtime for small models in the browser", "excerpt": "We use the ONNX runtime for small models in the browser https://github.com/microsoft/onnxruntime", "context": "", "use_case": "in-browser small model inference", "switched_from": "", "story": "WebLLM: high-performance in-browser LLM inference engine"}, {"id": 775359616, "item_id": 48459976, "url": "https://news.ycombinator.com/item?id=48459976", "by": "antonvs", "date": "2026-06-09", "version": null, "stance": "recommend", "quote": "We use this in production:", "excerpt": "We use this in production: https://docs.rs/onnxruntime/latest/onnxruntime/ It\u2019s a Rust wrapper around ONNX Runtime. We currently serve 5+ million inference requests per day for a highly performance-sensitive application, for a long list of major enterprise clients. We don\u2019t use GPUs for inference, because it would be cost-prohibitive. We launch tens of thousands of VMs per day to run these workloads.", "context": "serves 5+ million inference requests per day", "use_case": "production ML inference on CPU", "switched_from": "", "story": "OpenCV 5 Is Here: The Biggest Leap in Years for Computer Vision"}, {"id": 767963680, "item_id": 47997730, "url": "https://news.ycombinator.com/item?id=47997730", "by": "exabrial", "date": "2026-05-03", "version": null, "stance": "recommend", "quote": "This format continues to impress me.", "excerpt": "A *2.4gb* ONNX? That is wild. This format continues to impress me. ONNX uses 32bit single precision floats I believe, so thats something like ~644m float params/constants. I recently dove deep 'traditional ML' side of the ONNX serialization format for the purposes of writing an JVM ML compiler for trees and regressions. ONNX actually quite clever the way it serializes trees into parallel arrays (which is then seriali", "context": "", "use_case": "", "switched_from": "", "story": "Show HN: Apple's Sharp Running in the Browser via ONNX Runtime Web"}, {"id": 755447072, "item_id": 47215442, "url": "https://news.ycombinator.com/item?id=47215442", "by": "marcyb5st", "date": "2026-03-02", "version": null, "stance": "recommend", "quote": "ONNX (CPU at least) was already low level code and already very optimized", "excerpt": "You can do that with Onnx. You can graft the preprocessing layers to the actual model [1] and then serve that. Honestly, I already thought that ONNX (CPU at least) was already low level code and already very optimized. @Author - if you see this is it possible to add comparisons (ie \"vanilla\" inference latencies vs timber)? [1] https://gist.github.com/msteiner-google/5f03534b0df58d32abccbaeca88837c5 <-- A gist I put t", "context": "", "use_case": "model inference serving", "switched_from": "", "story": "Show HN: Timber \u2013 Ollama for classical ML models, 336x faster than Python"}, {"id": 744288176, "item_id": 46518011, "url": "https://news.ycombinator.com/item?id=46518011", "by": "sroussey", "date": "2026-01-06", "version": null, "stance": "recommend", "quote": "onnx on the web has the most models available and can use webgpu which is available everywhere.", "excerpt": "onnx on the web has the most models available and can use webgpu which is available everywhere. Huggingface\u2019s transformers.js uses it. And I use that for https://workglow.dev (also tensorflow mediapipe though that is using wasm). I don\u2019t think webnn has gone anywhere and is too restrictive.", "context": "", "use_case": "ML models in the browser", "switched_from": "", "story": "Show HN: Jax-JS, array library in JavaScript targeting WebGPU"}]}], "caveats": [{"label": "", "summary": "", "quotes": [{"id": 783539776, "item_id": 48971236, "url": "https://news.ycombinator.com/item?id=48971236", "by": "salamo", "date": "2026-07-19", "version": null, "stance": "warn", "quote": "ORT alone was 3.4MB over the wire, so I swapped it out for a tiny wasm", "excerpt": "Yeah, I also found that for ultra low footprint models ORT is a big portion of the total payload, because it contains logic for general ONNX graph operations. In my case I found that ORT alone was 3.4MB over the wire, so I swapped it out for a tiny wasm that was 850x smaller and only contained the operations I needed: https://blog.lukesalamone.com/posts/creating-tiny-semantic-search/", "context": "", "use_case": "semantic search in browser", "switched_from": "", "story": null}, {"id": 783431010, "item_id": 48964438, "url": "https://news.ycombinator.com/item?id=48964438", "by": "shade", "date": "2026-07-18", "version": null, "stance": "warn", "quote": "dealing with ONNX is kind of annoying", "excerpt": "Nice - I'm definitely going to take a look at this. I've built my own cross-platform (Mac/Win/Linux) live captioning app on top of Nemotron, and it works well but dealing with ONNX is kind of annoying. With this having Rust support (I built it on Rust/Tauri) it should be a pretty solid candidate; I'll have to see if I can find a Silero VAD implementation that doesn't depend on ONNX, or maybe I'll see if the clankers ", "context": "built a cross-platform live captioning app", "use_case": "model runtime", "switched_from": "", "story": "Transcribe.cpp"}, {"id": 773298850, "item_id": 48331178, "url": "https://news.ycombinator.com/item?id=48331178", "by": "jkaczman", "date": "2026-05-29", "version": null, "stance": "warn", "quote": "getting rid of a massive dependency we currently rely on (ONNX Runtime)", "excerpt": "Hi u/blitzo, thanks for the reply! I'm the other contributor mentioned in the post (John Kaczman). In case you're not familiar with the Yocto Project, it's designed to be a tool/template for developers (like Ardalan and I!) to use to create custom Linux images for embedded devices (in this case, a Raspberry Pi). It works off of distributing layers/recipes (these \"templates\") in open-source repositories for re-use amo", "context": "", "use_case": "", "switched_from": "", "story": "Show HN: Open-source private home security camera system (end-to-end encryption)"}, {"id": 773015057, "item_id": 48313441, "url": "https://news.ycombinator.com/item?id=48313441", "by": "CoreEpoch", "date": "2026-05-28", "version": null, "stance": "warn", "quote": "tried to quantize my .onnx models using the ORT Python quantizer and realized the models were becoming slower", "excerpt": "I was working on a camera inference project and tried to quantize my .onnx models using the ORT Python quantizer and realized the models were becoming slower, which didn't make sense to me. So I started making my own quantize tool in Rust and as I was peeling back the layers, running tests, etc I figured out the ORT quantizer places the Dequantize node between the Conv and the ReLU, breaking kernel fusion. But if you", "context": "", "use_case": "", "switched_from": "", "story": "Kenosis \u2013 Quantize ONNX models without breaking kernel fusion"}]}], "who": [{"by": "antonvs", "context": "serves 5+ million inference requests per day", "url": "https://news.ycombinator.com/item?id=48459976", "date": "2026-06-09"}], "tags": [], "use_cases": ["model inference", "entity scoring for voice control", "in-browser small model inference", "running model in rust simulator", "semantic search in browser", "model runtime"], "compared": [{"slug": "coreml", "name": "CoreML", "co": 3, "switched_from": 0, "switched_to": 0}, {"slug": "executorch", "name": "ExecuTorch", "co": 1, "switched_from": 0, "switched_to": 1}, {"slug": "pytorch", "name": "PyTorch", "co": 3, "switched_from": 0, "switched_to": 0}, {"slug": "home-assistant", "name": "Home Assistant", "co": 2, "switched_from": 0, "switched_to": 0}, {"slug": "webllm", "name": "WebLLM", "co": 2, "switched_from": 0, "switched_to": 0}], "makers": [], "threads": [{"url": "https://news.ycombinator.com/item?id=46350075", "title": "ONNX Runtime and CoreML May Silently Convert Your Model to FP16", "mentions": 3}, {"url": "https://news.ycombinator.com/item?id=47741527", "title": "Ask HN: What Are You Working On? (April 2026)", "mentions": 2}, {"url": "https://news.ycombinator.com/item?id=47303111", "title": "Ask HN: What Are You Working On? (March 2026)", "mentions": 2}, {"url": "https://news.ycombinator.com/item?id=46516267", "title": "Show HN: Jax-JS, array library in JavaScript targeting WebGPU", "mentions": 2}, {"url": "https://news.ycombinator.com/item?id=50195488", "title": "Show HN: OnKey \u2013 I'm a bad singer, so I built an app to show where I'm off-key", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=49986765", "title": "Jev-Driven SRE Diagnosis: What Worked and What Failed", "mentions": 1}], "sentiment_by_month": {"2025-11": {"recommend": 1}, "2025-12": {"neutral": 1, "warn": 3}, "2026-01": {"recommend": 1, "neutral": 1, "compare": 1}, "2026-02": {"neutral": 1}, "2026-03": {"recommend": 1, "neutral": 2, "compare": 2}, "2026-04": {"warn": 1, "neutral": 1}, "2026-05": {"recommend": 1, "warn": 2}, "2026-06": {"recommend": 1, "neutral": 1}, "2026-07": {"warn": 2, "neutral": 1}, "2026-08": {}, "2026-09": {"recommend": 1}, "2026-10": {"neutral": 2}}, "versions": [], "version_unnamed": 0, "family_ref": null, "runs_on": null, "lines": [], "quotes": [{"id": 803189171, "item_id": 50199323, "url": "https://news.ycombinator.com/item?id=50199323", "by": "dsmurrell", "date": "2026-10-22", "version": null, "stance": "neutral", "quote": "with the model running on ONNX Runtime", "excerpt": "For the user's pitch OnKey uses CREPE-tiny (https://github.com/marl/crepe) but ported to Rust with the model running on ONNX Runtime. For the singer's pitch also CREPE but the full version. The vocal separation for the song uses UVR: https://github.com/Anjok07/ultimatevocalremovergui You can find more details here: https://singonkey.app/docs/how-it-works", "context": "", "use_case": "", "switched_from": "", "story": "Show HN: OnKey \u2013 I'm a bad singer, so I built an app to show where I'm off-key"}, {"id": 799829985, "item_id": 49989374, "url": "https://news.ycombinator.com/item?id=49989374", "by": "amne", "date": "2026-10-07", "version": null, "stance": "neutral", "quote": "onnx scores all my HA entities to determine which one I'm talking about", "excerpt": "I switched to a decision model for my local \"AI\" voice control thingy. faster-whisper produces the text then onnx scores all my HA entities to determine which one I'm talking about .. then it lists all its actions and a second scoring round to determine what action I want. some regex to extract numbers if I'm saying things like \"water the lawn for 10 minutes\". It's pretty dumb when compared to what an LLM can do but ", "context": "", "use_case": "entity scoring for voice control", "switched_from": "", "story": "Jev-Driven SRE Diagnosis: What Worked and What Failed"}, {"id": 792599744, "item_id": 49537484, "url": "https://news.ycombinator.com/item?id=49537484", "by": "seamossfet", "date": "2026-09-02", "version": null, "stance": "recommend", "quote": "We use the ONNX runtime for small models in the browser", "excerpt": "We use the ONNX runtime for small models in the browser https://github.com/microsoft/onnxruntime", "context": "", "use_case": "in-browser small model inference", "switched_from": "", "story": "WebLLM: high-performance in-browser LLM inference engine"}, {"id": 785824307, "item_id": 49114019, "url": "https://news.ycombinator.com/item?id=49114019", "by": "qkwrv", "date": "2026-07-30", "version": null, "stance": "neutral", "quote": "it's so the Rust simulator can run the model itself, with no Python in the loop", "excerpt": "I know I say it in the title, but I want to be very clear up front: the predominant output of this tool is \"don't place this bet.\" Betting apps want you to bet. If they sense you are tired of losing and might leave, they'll entice you with deposit bonuses, boosted odds and protected picks. There are even people, you see them self-promoting all over X, who sell access to \"good\" bets. Obviously, they don't refund you i", "context": "", "use_case": "running model in Rust simulator", "switched_from": "", "story": "Show HN: NegativeEV \u2013 I built a tool to help people see how bad their bets are"}, {"id": 783539776, "item_id": 48971236, "url": "https://news.ycombinator.com/item?id=48971236", "by": "salamo", "date": "2026-07-19", "version": null, "stance": "warn", "quote": "ORT alone was 3.4MB over the wire, so I swapped it out for a tiny wasm", "excerpt": "Yeah, I also found that for ultra low footprint models ORT is a big portion of the total payload, because it contains logic for general ONNX graph operations. In my case I found that ORT alone was 3.4MB over the wire, so I swapped it out for a tiny wasm that was 850x smaller and only contained the operations I needed: https://blog.lukesalamone.com/posts/creating-tiny-semantic-search/", "context": "", "use_case": "semantic search in browser", "switched_from": "", "story": null}, {"id": 783431010, "item_id": 48964438, "url": "https://news.ycombinator.com/item?id=48964438", "by": "shade", "date": "2026-07-18", "version": null, "stance": "warn", "quote": "dealing with ONNX is kind of annoying", "excerpt": "Nice - I'm definitely going to take a look at this. I've built my own cross-platform (Mac/Win/Linux) live captioning app on top of Nemotron, and it works well but dealing with ONNX is kind of annoying. With this having Rust support (I built it on Rust/Tauri) it should be a pretty solid candidate; I'll have to see if I can find a Silero VAD implementation that doesn't depend on ONNX, or maybe I'll see if the clankers ", "context": "built a cross-platform live captioning app", "use_case": "model runtime", "switched_from": "", "story": "Transcribe.cpp"}, {"id": 778192433, "item_id": 48637027, "url": "https://news.ycombinator.com/item?id=48637027", "by": "simonw", "date": "2026-06-22", "version": null, "stance": "neutral", "quote": "I got this working with ONNX", "excerpt": "I got this working with ONNX (thanks, Claude Opus 4.8) and now I have an interactive demo of the model running entirely in the browser here (~1.3GB download): https://simonw.github.io/moebius-web/ - code here: https://github.com/simonw/moebius-web (Claude Code transcript: https://gisthost.github.io/?58039ba5c1ca3ed177e8659168996ee4)", "context": "", "use_case": "running model in browser", "switched_from": "", "story": "Moebius: 0.2B image inpainting model with 10B-level performance"}, {"id": 775359616, "item_id": 48459976, "url": "https://news.ycombinator.com/item?id=48459976", "by": "antonvs", "date": "2026-06-09", "version": null, "stance": "recommend", "quote": "We use this in production:", "excerpt": "We use this in production: https://docs.rs/onnxruntime/latest/onnxruntime/ It\u2019s a Rust wrapper around ONNX Runtime. We currently serve 5+ million inference requests per day for a highly performance-sensitive application, for a long list of major enterprise clients. We don\u2019t use GPUs for inference, because it would be cost-prohibitive. We launch tens of thousands of VMs per day to run these workloads.", "context": "serves 5+ million inference requests per day", "use_case": "production ML inference on CPU", "switched_from": "", "story": "OpenCV 5 Is Here: The Biggest Leap in Years for Computer Vision"}, {"id": 773298850, "item_id": 48331178, "url": "https://news.ycombinator.com/item?id=48331178", "by": "jkaczman", "date": "2026-05-29", "version": null, "stance": "warn", "quote": "getting rid of a massive dependency we currently rely on (ONNX Runtime)", "excerpt": "Hi u/blitzo, thanks for the reply! I'm the other contributor mentioned in the post (John Kaczman). In case you're not familiar with the Yocto Project, it's designed to be a tool/template for developers (like Ardalan and I!) to use to create custom Linux images for embedded devices (in this case, a Raspberry Pi). It works off of distributing layers/recipes (these \"templates\") in open-source repositories for re-use amo", "context": "", "use_case": "", "switched_from": "", "story": "Show HN: Open-source private home security camera system (end-to-end encryption)"}, {"id": 773015057, "item_id": 48313441, "url": "https://news.ycombinator.com/item?id=48313441", "by": "CoreEpoch", "date": "2026-05-28", "version": null, "stance": "warn", "quote": "tried to quantize my .onnx models using the ORT Python quantizer and realized the models were becoming slower", "excerpt": "I was working on a camera inference project and tried to quantize my .onnx models using the ORT Python quantizer and realized the models were becoming slower, which didn't make sense to me. So I started making my own quantize tool in Rust and as I was peeling back the layers, running tests, etc I figured out the ORT quantizer places the Dequantize node between the Conv and the ReLU, breaking kernel fusion. But if you", "context": "", "use_case": "", "switched_from": "", "story": "Kenosis \u2013 Quantize ONNX models without breaking kernel fusion"}, {"id": 767963680, "item_id": 47997730, "url": "https://news.ycombinator.com/item?id=47997730", "by": "exabrial", "date": "2026-05-03", "version": null, "stance": "recommend", "quote": "This format continues to impress me.", "excerpt": "A *2.4gb* ONNX? That is wild. This format continues to impress me. ONNX uses 32bit single precision floats I believe, so thats something like ~644m float params/constants. I recently dove deep 'traditional ML' side of the ONNX serialization format for the purposes of writing an JVM ML compiler for trees and regressions. ONNX actually quite clever the way it serializes trees into parallel arrays (which is then seriali", "context": "", "use_case": "", "switched_from": "", "story": "Show HN: Apple's Sharp Running in the Browser via ONNX Runtime Web"}, {"id": 764025826, "item_id": 47751614, "url": "https://news.ycombinator.com/item?id=47751614", "by": "ctbellmar", "date": "2026-04-13", "version": null, "stance": "neutral", "quote": "Go API, ONNX Runtime", "excerpt": "PlantLab (https://plantlab.ai) - AI plant health diagnosis for cannabis. It's an API, not an app [1]. Photo in, structured JSON out - condition, confidence, growth stage, nutrient lockout analysis. The response is for machines. Light burn at 0.92 confidence? Your controller dims the light. Calcium deficiency with excess potassium flagged as the lockout cause? Dosing pump adjusts. I'm a software dev/data nerd, not a g", "context": "", "use_case": "model inference", "switched_from": "", "story": "Ask HN: What Are You Working On? (April 2026)"}, {"id": 763928945, "item_id": 47745559, "url": "https://news.ycombinator.com/item?id=47745559", "by": "exabrial", "date": "2026-04-12", "version": null, "stance": "warn", "quote": "The ONNX Runtimes are also huge (150mb), JNI, and drag gargantuan dependencies!", "excerpt": "https://github.com/exabrial/petrify Petrify is a machine learning model compiler for the the JVM. It reads your model from an ONNX or other model format, walks the Trees or Linear models, and encodes the model in equivalent JVM bytecode as a stateless class you can invoke. This differs from every other ONNX Runtime that I know of, which are essentially interpreters. The ONNX Runtimes are also huge (150mb), JNI, and d", "context": "", "use_case": "", "switched_from": "", "story": "Ask HN: What Are You Working On? (April 2026)"}, {"id": 759079312, "item_id": 47442457, "url": "https://news.ycombinator.com/item?id=47442457", "by": "rohan_joshi", "date": "2026-03-19", "version": null, "stance": "compare", "quote": "it is expected to be faster than onnx", "excerpt": "shouldn't be hard. what backend/hardware are you interested in running this with? i'll add an example for using C++ onnx model. btw check out roadmap, our inference engine will be out 1-2 weeks and it is expected to be faster than onnx.", "context": "", "use_case": "", "switched_from": "", "story": "Show HN: Three new Kitten TTS models \u2013 smallest less than 25MB"}, {"id": 756913186, "item_id": 47307074, "url": "https://news.ycombinator.com/item?id=47307074", "by": "paragarora", "date": "2026-03-09", "version": null, "stance": "neutral", "quote": "ONNX for local embeddings", "excerpt": "Building HEBBS \u2014 a memory engine for AI agents, written in Rust. The problem: every agent framework bolts together a vector DB for recall, a KV store for state, maybe a graph DB for relationships, and then hopes the duct tape holds. You get one retrieval path (similarity search), no decay, no consolidation, and the agent forgets everything the moment context gets trimmed. HEBBS replaces that stack with a single embed", "context": "", "use_case": "Local embeddings", "switched_from": "", "story": "Ask HN: What Are You Working On? (March 2026)"}, {"id": 756905315, "item_id": 47306582, "url": "https://news.ycombinator.com/item?id=47306582", "by": "mikeayles", "date": "2026-03-09", "version": null, "stance": "compare", "quote": "It's 10-50x slower than ONNX Runtime", "excerpt": "Rewriting the backend Bitwise Cloud, my semantic search for embedded systems docs Claude Code plugin from Python to Go. The problem was the ML dependencies. The backend uses BGE-small-en-v1.5 for embeddings and FAISS for vector search. Both are C++/Python. Using them from Go means CGO, which means a C toolchain in your build, platform-specific binaries, and the end of go get && go build. So I wrote both from scratch ", "context": "", "use_case": "", "switched_from": "", "story": "Ask HN: What Are You Working On? (March 2026)"}, {"id": 755491458, "item_id": 47218216, "url": "https://news.ycombinator.com/item?id=47218216", "by": "dmonroy", "date": "2026-03-02", "version": null, "stance": "neutral", "quote": "Local ONNX inference, no API calls", "excerpt": "TLDR: PostgreSQL extension that adds ai.embed() and ai.classify() as IMMUTABLE functions. Local ONNX inference, no API calls. Works in generated columns out of the box. ~117 MiB per backend ( connection pooler recommended). The design and constraints are human-driven; the implementation was AI-assisted under supervision. ~$127 in tokens spent. GitHub: https://github.com/dmonroy/ai-native-pg Post: https://insert.dev/i", "context": "", "use_case": "", "switched_from": "", "story": "ai.embed() and ai.classify() as IMMUTABLE Postgres functions. AI-coded for $127"}, {"id": 755447072, "item_id": 47215442, "url": "https://news.ycombinator.com/item?id=47215442", "by": "marcyb5st", "date": "2026-03-02", "version": null, "stance": "recommend", "quote": "ONNX (CPU at least) was already low level code and already very optimized", "excerpt": "You can do that with Onnx. You can graft the preprocessing layers to the actual model [1] and then serve that. Honestly, I already thought that ONNX (CPU at least) was already low level code and already very optimized. @Author - if you see this is it possible to add comparisons (ie \"vanilla\" inference latencies vs timber)? [1] https://gist.github.com/msteiner-google/5f03534b0df58d32abccbaeca88837c5 <-- A gist I put t", "context": "", "use_case": "model inference serving", "switched_from": "", "story": "Show HN: Timber \u2013 Ollama for classical ML models, 336x faster than Python"}, {"id": 753900337, "item_id": 47118771, "url": "https://news.ycombinator.com/item?id=47118771", "by": "vicioussquid", "date": "2026-02-22", "version": null, "stance": "neutral", "quote": "", "excerpt": "https://github.com/ViciousSquid/Dosidicus This update introduces optional AI accelerator support via ONNX Runtime while keeping NumPy fallback. I am looking for testers, thank you for your interest x", "context": "", "use_case": "AI model acceleration", "switched_from": "", "story": "Show HN: Optional AI accelerator support without PyTorch (ONNX and NumPy)"}, {"id": 749001409, "item_id": 46812588, "url": "https://news.ycombinator.com/item?id=46812588", "by": "finalkk", "date": "2026-01-29", "version": null, "stance": "compare", "quote": "They are incredible for production", "excerpt": "Hi HN, I\u2019ve spent the last 3 months building Kandle because I was frustrated with the status quo of ML on the Web. Currently, if you want to run models in the browser, you\u2019re mostly stuck with ONNX Runtime or WebLLM. They are incredible for production, but they are \"Blackboxes.\" You have almost zero control over the intermediate tensors, and implementing custom logic or hooks between layers is a nightmare. I missed t", "context": "", "use_case": "", "switched_from": "", "story": "Show HN: Kandle \u2013 A WebGPU-based ML library written from scratch in JavaScript"}, {"id": 744289104, "item_id": 46518069, "url": "https://news.ycombinator.com/item?id=46518069", "by": "ekzhang", "date": "2026-01-06", "version": null, "stance": "neutral", "quote": "", "excerpt": "Since ONNX is just a model data format, you can actually parse and run ONNX files in jax-js as well. Here\u2019s an example of running DETR ResNet-50 from Xenova\u2019s transformers.js checkpoint in jax-js https://jax-js.com/detr-resnet-50 I don\u2019t think I intend to support everything in ONNX right now, especially quant/dequant, but eventually it would be interesting to see if we can help accelerate transformers.js with a jax-j", "context": "", "use_case": "ML model format and runtime", "switched_from": "", "story": "Show HN: Jax-JS, array library in JavaScript targeting WebGPU"}, {"id": 744288176, "item_id": 46518011, "url": "https://news.ycombinator.com/item?id=46518011", "by": "sroussey", "date": "2026-01-06", "version": null, "stance": "recommend", "quote": "onnx on the web has the most models available and can use webgpu which is available everywhere.", "excerpt": "onnx on the web has the most models available and can use webgpu which is available everywhere. Huggingface\u2019s transformers.js uses it. And I use that for https://workglow.dev (also tensorflow mediapipe though that is using wasm). I don\u2019t think webnn has gone anywhere and is too restrictive.", "context": "", "use_case": "ML models in the browser", "switched_from": "", "story": "Show HN: Jax-JS, array library in JavaScript targeting WebGPU"}, {"id": 742153264, "item_id": 46384579, "url": "https://news.ycombinator.com/item?id=46384579", "by": "nuc1e0n", "date": "2025-12-25", "version": null, "stance": "warn", "quote": "My experiences with ONNX have not been pleasant. Conversions from models written with Tensorflow and Pytorch often fail.", "excerpt": "My experiences with ONNX have not been pleasant. Conversions from models written with Tensorflow and Pytorch often fail. I recommend using TFLite or Executorch for deployment to edge devices instead.", "context": "", "use_case": "model conversion for edge deployment", "switched_from": "", "story": "ONNX Runtime and CoreML May Silently Convert Your Model to FP16"}, {"id": 741638929, "item_id": 46352433, "url": "https://news.ycombinator.com/item?id=46352433", "by": "Two_hands", "date": "2025-12-22", "version": null, "stance": "neutral", "quote": "The best solution for ORT would probably be to introduce a pure MPS provider", "excerpt": "Thank you for reading. Also generally I think CoreML isn't the best. The best solution for ORT would probably be to introduce a pure MPS provider (https://github.com/microsoft/onnxruntime/issues/21271), but given they've already bought into CoreML the effort may not be worth the reward for the core team. Which fair enough as it's a pretty mammoth task", "context": "", "use_case": "", "switched_from": "", "story": "ONNX Runtime and CoreML May Silently Convert Your Model to FP16"}, {"id": 741630402, "item_id": 46351900, "url": "https://news.ycombinator.com/item?id=46351900", "by": "smcleod", "date": "2025-12-21", "version": null, "stance": "warn", "quote": "https://github.com/microsoft/onnxruntime/issues/26355", "excerpt": "This was an interesting read, thanks for sharing. I've recently been building something that uses Parakeet v2/v3 models, I'm using the parakeet-rs package (https://github.com/altunenes/parakeet-rs) which has had a few issues running models with CoreML (unrelated to the linked post), e.g. https://github.com/microsoft/onnxruntime/issues/26355", "context": "", "use_case": "", "switched_from": "", "story": "ONNX Runtime and CoreML May Silently Convert Your Model to FP16"}, {"id": 740290624, "item_id": 46268164, "url": "https://news.ycombinator.com/item?id=46268164", "by": "Two_hands", "date": "2025-12-14", "version": null, "stance": "warn", "quote": "Basically the default args lead to your model being ran in FP16 not FP32.", "excerpt": "Working on a blog post about default behaviour in ONNX Runtime when using the CoreML execution provider. Basically the default args lead to your model being ran in FP16 not FP32. You can find more details at my site soon: https://ym2132.github.io", "context": "", "use_case": "model inference", "switched_from": "", "story": "Ask HN: What Are You Working On? (December 2025)"}, {"id": 739127970, "item_id": 46195498, "url": "https://news.ycombinator.com/item?id=46195498", "by": "refulgentis", "date": "2025-12-08", "version": null, "stance": "neutral", "quote": "specific packaged & code signed libraries provided by Microsoft for ONNX", "excerpt": "My caffeinated instinct is to say basically \"yes I'd do anything in Flutter\", I honestly would rather stop coding than go back to anything I've done before (ObjC/Swift/Java/Kotlin with side journeys in C++). It boggles my mind how much of a different job dev is with true hot reload. More carefully, and dealing with what you're indicating more directly: There's stuff that we just need every millisecond of performance ", "context": "", "use_case": "", "switched_from": "", "story": "Scala 3 slowed us down?"}, {"id": 734919328, "item_id": 45932458, "url": "https://news.ycombinator.com/item?id=45932458", "by": "hamza_q_", "date": "2025-11-14", "version": null, "stance": "recommend", "quote": "Cool use of ONNX!", "excerpt": "Cool use of ONNX! Fluid Inference also have great implementations of Parakeet v2/v3 in CoreML for Apple devices and OpenVINO for Intel: https://github.com/FluidInference/FluidAudio https://github.com/FluidInference/eddy-audio", "context": "", "use_case": "", "switched_from": "", "story": "Show HN: Chirp \u2013 Local Windows dictation with ParakeetV3 no executable required"}]}