{"slug": "simdjson", "name": "simdjson", "category": "dev-libs", "category_name": "Libraries and frameworks", "description": "", "recommenders": 5, "commenters": 10, "warners": 1, "score": 4.452, "rising": -1.64, "first": "2025-11-12", "latest": "2026-07-23", "why": "It showcases an especially ingenious scheme for escaping json strings", "why_is_quote": true, "has_verdict": false, "sentiment": {"recommend": 5, "neutral": 3, "compare": 1, "warn": 1}, "spark": [0, 0, 2, 0, 2, 0, 0, 0, 1, 0, 0, 0], "listed": true, "loved": [], "caveat": "", "short_quote": "It showcases an especially ingenious scheme for escaping json strings", "kind": "", "icon": true, "domain": null, "github_repo": "github.com/simdjson/simdjson", "official_url": "https://simdjson.org", "pricing": "", "aliases": ["simdjson", "simd-json"], "verdict": null, "praise": [{"label": "", "summary": "", "quotes": [{"id": 784410752, "item_id": 49025672, "url": "https://news.ycombinator.com/item?id=49025672", "by": "ubedan", "date": "2026-07-23", "stance": "recommend", "quote": "It showcases an especially ingenious scheme for escaping json strings", "excerpt": "Github: https://github.com/simdjson/simdjson It showcases an especially ingenious scheme for escaping json strings, as well as the other tokens necessary to parse json. Daniel Lemire - One of the authors at QCon 2019: https://www.youtube.com/watch?v=wlvKAT7SZIQ Another Youtube video that explains it: https://www.youtube.com/watch?v=vd9J9PPmAMM", "context": "", "use_case": "JSON parsing library", "switched_from": "", "story": "GigaToken: ~1000x faster Language model tokenization"}, {"id": 760661072, "item_id": 47541317, "url": "https://news.ycombinator.com/item?id=47541317", "by": "Asmod4n", "date": "2026-03-27", "stance": "recommend", "quote": "You could just take simdjson, use its ondemand api", "excerpt": "You could just take simdjson, use its ondemand api and then navigate it with .at_path(_with_wildcard) (https://github.com/simdjson/simdjson/blob/master/doc/basics.md#jsonpath) The whole tool would be like a few dozen lines of c++ and most likely be faster than this.", "context": "", "use_case": "fast JSON parsing", "switched_from": "", "story": "A Faster Alternative to Jq"}, {"id": 759395921, "item_id": 47462245, "url": "https://news.ycombinator.com/item?id=47462245", "by": "szmarczak", "date": "2026-03-20", "stance": "recommend", "quote": "can process gigabytes per second", "excerpt": "> Attempted Fix: Skip the JSON Round-Trip > We integrated serde-wasm-bindgen So you're reinventing JSON but binary? V8 JSON nowadays is highly optimized [1] and can process gigabytes per second [2], I doubt it is a bottleneck here. [1] https://v8.dev/blog/json-stringify [2] https://github.com/simdjson/simdjson", "context": "", "use_case": "", "switched_from": "", "story": "We rewrote our Rust WASM Parser in TypeScript \u2013 and it got 3x Faster"}, {"id": 746858000, "item_id": 46678625, "url": "https://news.ycombinator.com/item?id=46678625", "by": "johndough", "date": "2026-01-19", "stance": "recommend", "quote": "You can parse JSON at several GB/s: https://github.com/simdjson/simdjson", "excerpt": "(>= 6 months of 1 gig of data per day) You can parse JSON at several GB/s: https://github.com/simdjson/simdjson And you could scale that by one or two orders of magnitude with thread-based parallelism on recent AMD Epyc or Intel Xeon CPUs. So parsing alone should not pose a problem (maybe even sub-second for 6 months of data). We would need a more precise problem statement to judge whether horizontal scaling is neede", "context": "", "use_case": "fast JSON parsing", "switched_from": "", "story": "Command-line Tools can be 235x Faster than your Hadoop Cluster (2014)"}]}], "caveats": [{"label": "", "summary": "", "quotes": [{"id": 736441184, "item_id": 46027574, "url": "https://news.ycombinator.com/item?id=46027574", "by": "zX41ZdbW", "date": "2025-11-23", "stance": "warn", "quote": "any usage of it breaks std::unordered_map in unrelated parts of the code", "excerpt": "This reminds me of an old bug in simdjson - any usage of it breaks std::unordered_map in unrelated parts of the code due to an unintentional modification of FPU flags: https://github.com/simdjson/simdjson/issues/169", "context": "", "use_case": "", "switched_from": "", "story": "A time-travelling door bug in Half Life 2"}]}], "who": [{"by": "mag123c", "context": "built a local token-tracking CLI", "url": "https://news.ycombinator.com/item?id=46832744", "date": "2026-01-30"}], "tags": [], "use_cases": ["fast json parsing", "json parsing", "json parsing library", "parsing session logs", "simd json parsing"], "compared": [], "makers": [], "threads": [{"url": "https://news.ycombinator.com/item?id=46666085", "title": "Command-line Tools can be 235x Faster than your Hadoop Cluster (2014)", "mentions": 2}, {"url": "https://news.ycombinator.com/item?id=49010167", "title": "GigaToken: ~1000x faster Language model tokenization", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=47539825", "title": "A Faster Alternative to Jq", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=47461094", "title": "We rewrote our Rust WASM Parser in TypeScript \u2013 and it got 3x Faster", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=46781011", "title": "Ask HN: How do you budget for token based AI APIs?", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=46780384", "title": "Show HN: I built a CSV parser to try Go 1.26's new SIMD package", "mentions": 1}], "sentiment_by_month": {"2025-11": {"neutral": 1, "warn": 1}, "2025-12": {}, "2026-01": {"compare": 1, "recommend": 2, "neutral": 2}, "2026-02": {}, "2026-03": {"recommend": 2}, "2026-04": {}, "2026-05": {}, "2026-06": {}, "2026-07": {"recommend": 1}, "2026-08": {}, "2026-09": {}, "2026-10": {}}, "quotes": [{"id": 784410752, "item_id": 49025672, "url": "https://news.ycombinator.com/item?id=49025672", "by": "ubedan", "date": "2026-07-23", "stance": "recommend", "quote": "It showcases an especially ingenious scheme for escaping json strings", "excerpt": "Github: https://github.com/simdjson/simdjson It showcases an especially ingenious scheme for escaping json strings, as well as the other tokens necessary to parse json. Daniel Lemire - One of the authors at QCon 2019: https://www.youtube.com/watch?v=wlvKAT7SZIQ Another Youtube video that explains it: https://www.youtube.com/watch?v=vd9J9PPmAMM", "context": "", "use_case": "JSON parsing library", "switched_from": "", "story": "GigaToken: ~1000x faster Language model tokenization"}, {"id": 760661072, "item_id": 47541317, "url": "https://news.ycombinator.com/item?id=47541317", "by": "Asmod4n", "date": "2026-03-27", "stance": "recommend", "quote": "You could just take simdjson, use its ondemand api", "excerpt": "You could just take simdjson, use its ondemand api and then navigate it with .at_path(_with_wildcard) (https://github.com/simdjson/simdjson/blob/master/doc/basics.md#jsonpath) The whole tool would be like a few dozen lines of c++ and most likely be faster than this.", "context": "", "use_case": "fast JSON parsing", "switched_from": "", "story": "A Faster Alternative to Jq"}, {"id": 759395921, "item_id": 47462245, "url": "https://news.ycombinator.com/item?id=47462245", "by": "szmarczak", "date": "2026-03-20", "stance": "recommend", "quote": "can process gigabytes per second", "excerpt": "> Attempted Fix: Skip the JSON Round-Trip > We integrated serde-wasm-bindgen So you're reinventing JSON but binary? V8 JSON nowadays is highly optimized [1] and can process gigabytes per second [2], I doubt it is a bottleneck here. [1] https://v8.dev/blog/json-stringify [2] https://github.com/simdjson/simdjson", "context": "", "use_case": "", "switched_from": "", "story": "We rewrote our Rust WASM Parser in TypeScript \u2013 and it got 3x Faster"}, {"id": 749323907, "item_id": 46832744, "url": "https://news.ycombinator.com/item?id=46832744", "by": "mag123c", "date": "2026-01-30", "stance": "neutral", "quote": "Parsing the session logs directly (JSONL/JSON) with simd-json", "excerpt": "I built a local CLI tool to track my daily token usage across Claude Code, Codex, and Gemini CLI. Parsing the session logs directly (JSONL/JSON) with simd-json gives me exact numbers without relying on any external API. Knowing my actual spend per day changed how I use these tools \u2014 I was burning 3x more on cache misses than I realized.", "context": "built a local token-tracking CLI", "use_case": "parsing session logs", "switched_from": "", "story": "Ask HN: How do you budget for token based AI APIs?"}, {"id": 748805953, "item_id": 46800372, "url": "https://news.ycombinator.com/item?id=46800372", "by": "juliusgeo", "date": "2026-01-28", "stance": "neutral", "quote": "Simdjson uses this trick, and I use it as well", "excerpt": "This is super cool! If I'm understanding your implementation correctly, you do perform bit by bit state machine logic to check whether quotes should be escaped etc. You can do that in a single pass by using carry-less polynomial multiplication instructions (_mm_clmulepi64_si128 on AVX-512 I believe), or by just computing the carryless xor directly on the quote mask and then &ing the inverse with the bitmask for quote", "context": "", "use_case": "SIMD JSON parsing", "switched_from": "", "story": "Show HN: I built a CSV parser to try Go 1.26's new SIMD package"}, {"id": 747010848, "item_id": 46688178, "url": "https://news.ycombinator.com/item?id=46688178", "by": "anshumankmr", "date": "2026-01-19", "stance": "recommend", "quote": "", "excerpt": "> https://github.com/simdjson/simdjson Was not aware of this but seems it is not there natively in Python,but seems cool. Will try out in future.", "context": "", "use_case": "", "switched_from": "", "story": "Command-line Tools can be 235x Faster than your Hadoop Cluster (2014)"}, {"id": 746858000, "item_id": 46678625, "url": "https://news.ycombinator.com/item?id=46678625", "by": "johndough", "date": "2026-01-19", "stance": "recommend", "quote": "You can parse JSON at several GB/s: https://github.com/simdjson/simdjson", "excerpt": "(>= 6 months of 1 gig of data per day) You can parse JSON at several GB/s: https://github.com/simdjson/simdjson And you could scale that by one or two orders of magnitude with thread-based parallelism on recent AMD Epyc or Intel Xeon CPUs. So parsing alone should not pose a problem (maybe even sub-second for 6 months of data). We would need a more precise problem statement to judge whether horizontal scaling is neede", "context": "", "use_case": "fast JSON parsing", "switched_from": "", "story": "Command-line Tools can be 235x Faster than your Hadoop Cluster (2014)"}, {"id": 744127169, "item_id": 46507948, "url": "https://news.ycombinator.com/item?id=46507948", "by": "eliasdejong", "date": "2026-01-05", "stance": "compare", "quote": "is able to outperform simdjson by 120x in some benchmarks", "excerpt": "Increasingly the performance limit for modern CPUs is the amount of data you can feed through a single core: basically memcpy() speed. On most x86 cores the limit is around 6 GB/s and about 20 GB/s for Apple M chips. When you see advertised numbers like '200 GB/s' that is total memory bandwidth, or all cores combined. For individual cores, the limit will still be around 6 GB/s. This means even if you write a perfect ", "context": "", "use_case": "", "switched_from": "", "story": "I/O is no longer the bottleneck? (2022)"}, {"id": 736441184, "item_id": 46027574, "url": "https://news.ycombinator.com/item?id=46027574", "by": "zX41ZdbW", "date": "2025-11-23", "stance": "warn", "quote": "any usage of it breaks std::unordered_map in unrelated parts of the code", "excerpt": "This reminds me of an old bug in simdjson - any usage of it breaks std::unordered_map in unrelated parts of the code due to an unintentional modification of FPU flags: https://github.com/simdjson/simdjson/issues/169", "context": "", "use_case": "", "switched_from": "", "story": "A time-travelling door bug in Half Life 2"}, {"id": 735399698, "item_id": 45962481, "url": "https://news.ycombinator.com/item?id=45962481", "by": "hilti", "date": "2025-11-17", "stance": "neutral", "quote": "multi-threaded parsing (simdjson)", "excerpt": "Similar tool for JSONL files: I built JSONL Viewer Pro after repeatedly crashing VS Code trying to inspect multi-GB training datasets and IoT device logs with nested objects. Native Mac/Windows app with multi-threaded parsing (simdjson), automatic nested object flattening, and handles 10M+ rows instantly. For HN: Use code HN100 for free access https://iotdatasystems.gumroad.com/ Built with C++ for native performance ", "context": "", "use_case": "JSON parsing", "switched_from": "", "story": "Show HN: Parqeye \u2013 A CLI tool to visualize and inspect Parquet files"}, {"id": 734396034, "item_id": 45899752, "url": "https://news.ycombinator.com/item?id=45899752", "by": "hilti", "date": "2025-11-12", "stance": "neutral", "quote": "Multi-threaded parsing with simdjson's on-demand API", "excerpt": "Hey HN! Author here. I built this after struggling with Excel/Pandas for large JSONL log files from IoT devices. The data had deeply nested objects and millions of rows. The key challenge was speed + usability. I wanted to just double-click a file and see my data instantly, with the ability to filter nested fields. How it works: 1. Multi-threaded parsing with simdjson's on-demand API 2. Automatic flattening of nested", "context": "", "use_case": "JSON parsing", "switched_from": "", "story": "Show HN: Jsonl Viewer \u2013 Desktop app for 10M+ rows with nested flattening"}]}