ONNX Runtime
No written verdict yet. Here is what commenters said.
Why people recommend it
“We use the ONNX runtime for small models in the browser”
“We use this in production:”
Show 3 more
“This format continues to impress me.”
“ONNX (CPU at least) was already low level code and already very optimized”
“onnx on the web has the most models available and can use webgpu which is available everywhere.”
Caveats and dissent
“ORT alone was 3.4MB over the wire, so I swapped it out for a tiny wasm”
“dealing with ONNX is kind of annoying”
Show 2 more
“getting rid of a massive dependency we currently rely on (ONNX Runtime)”
“tried to quantize my .onnx models using the ORT Python quantizer and realized the models were becoming slower”
Often compared with
Who recommends it in their own words
Threads where it came up
- ONNX Runtime and CoreML May Silently Convert Your Model to FP16 3 mentions
- Ask HN: What Are You Working On? (April 2026) 2 mentions
- Ask HN: What Are You Working On? (March 2026) 2 mentions
- Show HN: Jax-JS, array library in JavaScript targeting WebGPU 2 mentions
- Show HN: OnKey – I'm a bad singer, so I built an app to show where I'm off-key 1 mention
- Jev-Driven SRE Diagnosis: What Worked and What Failed 1 mention
Also written as ONNX. Data: onnx-runtime.json. Quotes are verbatim; the verdict and theme labels are written by a model from these comments, and every clause links to its sources.
All comments 28
“with the model running on ONNX Runtime”
“onnx scores all my HA entities to determine which one I'm talking about”
“We use the ONNX runtime for small models in the browser”
“it's so the Rust simulator can run the model itself, with no Python in the loop”
“ORT alone was 3.4MB over the wire, so I swapped it out for a tiny wasm”
“dealing with ONNX is kind of annoying”
“I got this working with ONNX”
“We use this in production:”
“getting rid of a massive dependency we currently rely on (ONNX Runtime)”
“tried to quantize my .onnx models using the ORT Python quantizer and realized the models were becoming slower”
“This format continues to impress me.”
“Go API, ONNX Runtime”
“The ONNX Runtimes are also huge (150mb), JNI, and drag gargantuan dependencies!”
“it is expected to be faster than onnx”
“ONNX for local embeddings”
“It's 10-50x slower than ONNX Runtime”
“Local ONNX inference, no API calls”
“ONNX (CPU at least) was already low level code and already very optimized”
https://github.com/ViciousSquid/Dosidicus This update introduces optional AI accelerator support via ONNX Runtime while keeping NumPy fallback. I am looking for testers, thank you for your interest x
“They are incredible for production”
Since ONNX is just a model data format, you can actually parse and run ONNX files in jax-js as well. Here’s an example of running DETR ResNet-50 from Xenova’s transformers.js checkpoint in jax-js https://jax-js.com/detr-resnet-50 I don’t think I intend to support everything in ONNX right now, especially quant/dequant, but eventually it would be interesting to see if we can help accelerate transformers.js with a jax-j
“onnx on the web has the most models available and can use webgpu which is available everywhere.”
“My experiences with ONNX have not been pleasant. Conversions from models written with Tensorflow and Pytorch often fail.”
“The best solution for ORT would probably be to introduce a pure MPS provider”
“https://github.com/microsoft/onnxruntime/issues/26355”
“Basically the default args lead to your model being ran in FP16 not FP32.”
“specific packaged & code signed libraries provided by Microsoft for ONNX”
“Cool use of ONNX!”