{"slug": "zstandard", "name": "Zstandard", "category": "dev-libs", "category_name": "Libraries and frameworks", "description": "", "recommenders": 19, "commenters": 44, "warners": 0, "score": 22.47, "rising": 3.75, "first": "2025-11-12", "latest": "2026-10-25", "why": "I prefer zstd, because it is like 10x faster for both compression and decompression", "why_is_quote": true, "has_verdict": false, "sentiment": {"recommend": 20, "neutral": 9, "compare": 17, "warn": 1}, "spark": [1, 0, 3, 0, 5, 1, 2, 1, 0, 1, 5, 1], "listed": true, "loved": [], "caveat": "", "short_quote": "I prefer zstd, because it is like 10x faster for both compression and decompression", "kind": "", "icon": true, "domain": null, "github_repo": "github.com/facebook/zstd", "official_url": "https://facebook.github.io/zstd/", "pricing": "", "aliases": ["zstd", "Zstandard", "Zstd", "ZSTD"], "verdict": null, "praise": [{"label": "", "summary": "", "quotes": [{"id": 793595936, "item_id": 49599746, "url": "https://news.ycombinator.com/item?id=49599746", "by": "nvme0n1p1", "date": "2026-09-07", "stance": "recommend", "quote": "zstd is the go-to compression format these days", "excerpt": "zstd is the go-to compression format these days. It's even supported in low-level software such as many linux filesystems. I don't know much about duckdb but it looks like it supports zstd too: https://duckdb.org/docs/lts/data/json/loading_json", "context": "", "use_case": "Compression format", "switched_from": "", "story": "bzip3"}, {"id": 792748112, "item_id": 49546757, "url": "https://news.ycombinator.com/item?id=49546757", "by": "chickenbig", "date": "2026-09-03", "stance": "recommend", "quote": "Zstandard is an awesome piece of technology.", "excerpt": "Zstandard is an awesome piece of technology. Even the very low compression levels -1, 1 or 2 (IIRC the default of 0 is actually a much higher number) can be very effective, especially for more situations where CPU is a bottleneck. It is particularly effective when you can 'rotate' the data to enhance compressibility; https://github.com/google/riegeli does this automatically for wire format protobufs by splitting data", "context": "", "use_case": "data compression", "switched_from": "", "story": "We could save petabytes of cache storage with Zstandard and Pingora"}, {"id": 792681680, "item_id": 49542605, "url": "https://news.ycombinator.com/item?id=49542605", "by": "gopalv", "date": "2026-09-02", "stance": "recommend", "quote": "Zstd has a seekable format for frames", "excerpt": "> I don't see how that's possible if the cache no longer stores the uncompressed data. Zstd has a seekable format for frames, similar to pigz --independent works. [1] - https://github.com/facebook/zstd/blob/dev/contrib/seekable_format/README.md", "context": "", "use_case": "", "switched_from": "", "story": "We could save petabytes of cache storage with Zstandard and Pingora"}, {"id": 792679392, "item_id": 49542462, "url": "https://news.ycombinator.com/item?id=49542462", "by": "kccqzy", "date": "2026-09-02", "stance": "recommend", "quote": "Given how fast zstd can decompress", "excerpt": "Actually zstd internally splits data into frames, and frames can indicate the decompressed data size. So if we control the compressor we can make it so that all frames have the size information; it isn\u2019t exactly seekable but at least it will not need to decompress the resource. https://python-zstandard.readthedocs.io/en/latest/concepts.html Given how fast zstd can decompress, this may or may not actually be a win: th", "context": "", "use_case": "", "switched_from": "", "story": "We could save petabytes of cache storage with Zstandard and Pingora"}, {"id": 792668800, "item_id": 49541800, "url": "https://news.ycombinator.com/item?id=49541800", "by": "theandrewbailey", "date": "2026-09-02", "stance": "recommend", "quote": "Zstd is widely supported in browsers", "excerpt": "I was thinking this. Zstd is widely supported in browsers, over 80% right now and will increase over time: https://caniuse.com/?search=zstd", "context": "", "use_case": "HTTP content compression", "switched_from": "", "story": "We could save petabytes of cache storage with Zstandard and Pingora"}]}], "caveats": [{"label": "", "summary": "", "quotes": [{"id": 793594128, "item_id": 49599633, "url": "https://news.ycombinator.com/item?id=49599633", "by": "nvme0n1p1", "date": "2026-09-07", "stance": "warn", "quote": "That always seemed annoying to me.", "excerpt": "> 8MB I believe for high levels Yep, i found it in the source here: - https://github.com/facebook/zstd/blob/d9c0c7e2cf8a8bf9fb98d3bee546dcf8dc9ac59a/lib/zstd.h#L1352 - https://github.com/facebook/zstd/blob/d9c0c7e2cf8a8bf9fb98d3bee546dcf8dc9ac59a/lib/compress/clevels.h Also, zstd docs say: > Note: If windowLog is set to larger than 27, --long=windowLog or --memory=windowSize needs to be passed to the decompressor. Th", "context": "", "use_case": "", "switched_from": "", "story": "bzip3"}]}], "who": [{"by": "davidebaldini", "context": "built a VPS search engine", "url": "https://news.ycombinator.com/item?id=49176154", "date": "2026-08-04"}], "tags": [], "use_cases": ["compression", "compression library", "data compression", "compression library and cli", "compression format", "compression with random access"], "compared": [{"slug": "gzip", "name": "gzip", "co": 9, "switched_from": 0, "switched_to": 0}, {"slug": "7-zip", "name": "7-Zip", "co": 3, "switched_from": 0, "switched_to": 0}, {"slug": "sqlite", "name": "SQLite", "co": 2, "switched_from": 0, "switched_to": 0}, {"slug": "riegeli", "name": "Riegeli", "co": 2, "switched_from": 0, "switched_to": 0}], "makers": [], "threads": [{"url": "https://news.ycombinator.com/item?id=49521909", "title": "We could save petabytes of cache storage with Zstandard and Pingora", "mentions": 6}, {"url": "https://news.ycombinator.com/item?id=47377998", "title": "An Ode to Bzip", "mentions": 6}, {"url": "https://news.ycombinator.com/item?id=49598291", "title": "bzip3", "mentions": 4}, {"url": "https://news.ycombinator.com/item?id=46717507", "title": "ISO PDF spec is getting Brotli \u2013 ~20 % smaller documents with no quality loss", "mentions": 4}, {"url": "https://news.ycombinator.com/item?id=48042434", "title": "SQLite Is a Library of Congress Recommended Storage Format", "mentions": 3}, {"url": "https://news.ycombinator.com/item?id=48633839", "title": "Hellishly Slow Level 13 Deflate Compression", "mentions": 2}], "sentiment_by_month": {"2025-11": {"recommend": 1}, "2025-12": {}, "2026-01": {"recommend": 3, "neutral": 1, "compare": 1}, "2026-02": {"compare": 3}, "2026-03": {"neutral": 2, "compare": 2, "recommend": 5}, "2026-04": {"compare": 2, "recommend": 1}, "2026-05": {"neutral": 1, "recommend": 2}, "2026-06": {"compare": 3, "recommend": 1}, "2026-07": {"compare": 4}, "2026-08": {"recommend": 1}, "2026-09": {"recommend": 5, "neutral": 4, "warn": 1, "compare": 1}, "2026-10": {"compare": 1, "recommend": 1, "neutral": 1}}, "quotes": [{"id": 803640162, "item_id": 50227510, "url": "https://news.ycombinator.com/item?id=50227510", "by": "bsutto", "date": "2026-10-25", "stance": "neutral", "quote": "Zstd compression.", "excerpt": "reVault is a new, secure-by-default archive format called a Lockbox, designed to be simple to use. I've been working on reVault for about 2 years. It was originally written in Dart, but with the improvements in AI, I decided to port it to Rust using AI. It's now in beta, so I'm looking for feedback, particularly around the different language bindings. A lockbox is encrypted, signed and compressed by default. Each can", "context": "", "use_case": "compression", "switched_from": "", "story": null}, {"id": 802761152, "item_id": 50172572, "url": "https://news.ycombinator.com/item?id=50172572", "by": "vel0city", "date": "2026-10-20", "stance": "recommend", "quote": "", "excerpt": "> knowing that it indirectly supports company > I totally get that by using zstd I'm not directly supporting these things. I'm not supporting Facebook by using a library that's freely available on the public internet and hosted by others. I'm not out there saying its worth the genocide. I actively try and avoid giving Facebook any amount of actual value from me and the things I touch. If you can point to how me using", "context": "", "use_case": "compression library", "switched_from": "", "story": "Facebook Pages require paid subscription to post more than two links per month"}, {"id": 799627457, "item_id": 49976716, "url": "https://news.ycombinator.com/item?id=49976716", "by": "exceptione", "date": "2026-10-06", "stance": "compare", "quote": "Orders faster than zstd.", "excerpt": "Link to slides at the bottom. Impressive read-speed, roughly operates at the same speed as native DRAM. Orders faster than zstd. See also: - https://www.phoronix.com/news/Linux-CRAM-Compressed-RAM - https://lwn.net/Articles/1059885/", "context": "", "use_case": "", "switched_from": "", "story": "Cram \u2013 A Compressed RAM Service"}, {"id": 796961120, "item_id": 49810070, "url": "https://news.ycombinator.com/item?id=49810070", "by": "Dylan16807", "date": "2026-09-23", "stance": "compare", "quote": "If I need speed then zstd wins.", "excerpt": "I'm not that fussed about CPU use. https://github.com/mcmilk/7-Zip-zstd By these charts, if I only need 5-10 megabytes per second of compression on a single core, LZMA2 wins significantly on ratio, and still decompresses at well over 100. If I'm doing a backup, or sending/receiving over my internet connection (which only has 2MB/s of upload), LZMA2 easily wins. If I need speed then zstd wins.", "context": "", "use_case": "", "switched_from": "", "story": "Can gzip be a language model?"}, {"id": 793742832, "item_id": 49608927, "url": "https://news.ycombinator.com/item?id=49608927", "by": "MrJohz", "date": "2026-09-08", "stance": "neutral", "quote": "implement a complete zstd decoder", "excerpt": "The setup is described in the linked post about programming language vs efficiency/token cost: https://danluu.com/pl-tokens/#zstd > For the first eval, I tried giving agents the zstd RFC (plus errata) and telling them to implement a complete zstd decoder (agents are stuck in a container without internet access). The tests were not given to agents. For something with the surface area of zstd, it's not really reasonabl", "context": "", "use_case": "Compression library", "switched_from": "", "story": "How well do agents use test/verification techniques?"}, {"id": 793645504, "item_id": 49602844, "url": "https://news.ycombinator.com/item?id=49602844", "by": "coder543", "date": "2026-09-07", "stance": "neutral", "quote": "zstd automatically learns the most efficient dictionary it can within a few kilobytes.", "excerpt": "\"Dictionary gains are mostly effective in the first few KB.\" https://facebook.github.io/zstd/index.html Pretrained dictionaries have never been intended to help with book sized or bigger compression. zstd automatically learns the most efficient dictionary it can within a few kilobytes. Pretrained dictionaries are only useful when you're independently compressing very small records.", "context": "", "use_case": "", "switched_from": "", "story": "bzip3"}, {"id": 793612032, "item_id": 49600752, "url": "https://news.ycombinator.com/item?id=49600752", "by": "cesarb", "date": "2026-09-07", "stance": "neutral", "quote": "It's 128 MiB by default in the command line decompressor", "excerpt": "> They couldn't allocate 5 more bits somewhere to let the decompressor autodetect longer window sizes? It's actually 8 bits: https://www.rfc-editor.org/rfc/rfc8878.html#name-window-descriptor These command line parameters change the maximum the decompressor will allow. It's 128 MiB by default in the command line decompressor; other uses (like the \"zstd\" content coding for HTTP in web browsers) use a lower limit of 8 ", "context": "", "use_case": "Compression library and CLI", "switched_from": "", "story": "bzip3"}, {"id": 793595936, "item_id": 49599746, "url": "https://news.ycombinator.com/item?id=49599746", "by": "nvme0n1p1", "date": "2026-09-07", "stance": "recommend", "quote": "zstd is the go-to compression format these days", "excerpt": "zstd is the go-to compression format these days. It's even supported in low-level software such as many linux filesystems. I don't know much about duckdb but it looks like it supports zstd too: https://duckdb.org/docs/lts/data/json/loading_json", "context": "", "use_case": "Compression format", "switched_from": "", "story": "bzip3"}, {"id": 793594128, "item_id": 49599633, "url": "https://news.ycombinator.com/item?id=49599633", "by": "nvme0n1p1", "date": "2026-09-07", "stance": "warn", "quote": "That always seemed annoying to me.", "excerpt": "> 8MB I believe for high levels Yep, i found it in the source here: - https://github.com/facebook/zstd/blob/d9c0c7e2cf8a8bf9fb98d3bee546dcf8dc9ac59a/lib/zstd.h#L1352 - https://github.com/facebook/zstd/blob/d9c0c7e2cf8a8bf9fb98d3bee546dcf8dc9ac59a/lib/compress/clevels.h Also, zstd docs say: > Note: If windowLog is set to larger than 27, --long=windowLog or --memory=windowSize needs to be passed to the decompressor. Th", "context": "", "use_case": "", "switched_from": "", "story": "bzip3"}, {"id": 792760144, "item_id": 49547509, "url": "https://news.ycombinator.com/item?id=49547509", "by": "NostraDavid", "date": "2026-09-03", "stance": "neutral", "quote": "the official lib at least uses 1-22", "excerpt": "> compression levels -1, 1 or 2 Are the available compression levels not 1-22? Though that might depend on the specific library used - the official lib at least uses 1-22: https://github.com/facebook/zstd/blob/dev/programs/zstd.1.md#operation-modifiers", "context": "", "use_case": "", "switched_from": "", "story": "We could save petabytes of cache storage with Zstandard and Pingora"}, {"id": 792748112, "item_id": 49546757, "url": "https://news.ycombinator.com/item?id=49546757", "by": "chickenbig", "date": "2026-09-03", "stance": "recommend", "quote": "Zstandard is an awesome piece of technology.", "excerpt": "Zstandard is an awesome piece of technology. Even the very low compression levels -1, 1 or 2 (IIRC the default of 0 is actually a much higher number) can be very effective, especially for more situations where CPU is a bottleneck. It is particularly effective when you can 'rotate' the data to enhance compressibility; https://github.com/google/riegeli does this automatically for wire format protobufs by splitting data", "context": "", "use_case": "data compression", "switched_from": "", "story": "We could save petabytes of cache storage with Zstandard and Pingora"}, {"id": 792717008, "item_id": 49544813, "url": "https://news.ycombinator.com/item?id=49544813", "by": "genxy", "date": "2026-09-03", "stance": "recommend", "quote": "", "excerpt": "Also external dictionaries https://nigeltao.github.io/blog/2022/zstandard-part-7-dictionaries.html", "context": "", "use_case": "compression with random access", "switched_from": "", "story": "We could save petabytes of cache storage with Zstandard and Pingora"}, {"id": 792681680, "item_id": 49542605, "url": "https://news.ycombinator.com/item?id=49542605", "by": "gopalv", "date": "2026-09-02", "stance": "recommend", "quote": "Zstd has a seekable format for frames", "excerpt": "> I don't see how that's possible if the cache no longer stores the uncompressed data. Zstd has a seekable format for frames, similar to pigz --independent works. [1] - https://github.com/facebook/zstd/blob/dev/contrib/seekable_format/README.md", "context": "", "use_case": "", "switched_from": "", "story": "We could save petabytes of cache storage with Zstandard and Pingora"}, {"id": 792679392, "item_id": 49542462, "url": "https://news.ycombinator.com/item?id=49542462", "by": "kccqzy", "date": "2026-09-02", "stance": "recommend", "quote": "Given how fast zstd can decompress", "excerpt": "Actually zstd internally splits data into frames, and frames can indicate the decompressed data size. So if we control the compressor we can make it so that all frames have the size information; it isn\u2019t exactly seekable but at least it will not need to decompress the resource. https://python-zstandard.readthedocs.io/en/latest/concepts.html Given how fast zstd can decompress, this may or may not actually be a win: th", "context": "", "use_case": "", "switched_from": "", "story": "We could save petabytes of cache storage with Zstandard and Pingora"}, {"id": 792668800, "item_id": 49541800, "url": "https://news.ycombinator.com/item?id=49541800", "by": "theandrewbailey", "date": "2026-09-02", "stance": "recommend", "quote": "Zstd is widely supported in browsers", "excerpt": "I was thinking this. Zstd is widely supported in browsers, over 80% right now and will increase over time: https://caniuse.com/?search=zstd", "context": "", "use_case": "HTTP content compression", "switched_from": "", "story": "We could save petabytes of cache storage with Zstandard and Pingora"}, {"id": 786818465, "item_id": 49176154, "url": "https://news.ycombinator.com/item?id=49176154", "by": "davidebaldini", "date": "2026-08-04", "stance": "recommend", "quote": "to compress long strings using a trained ZSTD dictionary", "excerpt": "This pet project of mine is a web search engine for discovering virtual private server (VPS) offers starting from a search query that the user can incrementally specialize to iteratively add constraints and narrow the corpus of results. Thru the past years I added 120 service providers to the data acquisition pipeline, which is run every 24 hours to refresh the entirety of the corpus and to index the VPS services in ", "context": "built a VPS search engine", "use_case": "compressing offer URLs", "switched_from": "", "story": "Show HN: A search engine for VPS servers with an automatic crawler pipeline"}, {"id": 785345568, "item_id": 49084098, "url": "https://news.ycombinator.com/item?id=49084098", "by": "zX41ZdbW", "date": "2026-07-28", "stance": "compare", "quote": "compress it with ZSTD, it typically will be better", "excerpt": "Yes, for example, the datasets at this page: https://adsb.exposed/ are indexed by the Morton curve: https://github.com/ClickHouse/adsb.exposed/#database-and-queries and https://reversedns.space/ as well. Also, a trivial application is image compression. Let's say you have a PNG image. PNG uses zlib, so if instead you take a raw bitmap and compress it with ZSTD, it typically will be better, but if you also sort pixels", "context": "", "use_case": "Bitmap compression", "switched_from": "", "story": "Some combinatorial applications of spacefilling curves"}, {"id": 783043745, "item_id": 48940234, "url": "https://news.ycombinator.com/item?id=48940234", "by": "wahern", "date": "2026-07-16", "stance": "compare", "quote": "embedding the zstd decompressor blew up the binary", "excerpt": "I built a benchmark harness for exploring compression of Lua script source files embedded in a Lua app binary as individual static constant C arrays. I settled on gzip, specifically deflate/inflate. Deflate does as well or better for small source files compared to xz, bzip2, and zstd. More importantly, the inflate algorithm is tiny[1]; embedding the zstd decompressor blew up the binary. I also explored building preco", "context": "", "use_case": "compressing embedded Lua sources", "switched_from": "", "story": "Optimizing Lua string literals to save 400 bytes"}, {"id": 782923456, "item_id": 48932716, "url": "https://news.ycombinator.com/item?id=48932716", "by": "paddor", "date": "2026-07-16", "stance": "compare", "quote": "Zstd already follows the pareto line very closely", "excerpt": "Zstd already follows the pareto line very closely and also decompresses pretty fast. In case you're looking for a memory-safe version in Rust that avoids the FFI penalty in hot loops, optimized to hell and back, this here is the new kid on the block: https://github.com/paddor/zrip", "context": "", "use_case": "", "switched_from": "", "story": "Show HN: misa77 - a codec that decodes 2x faster than LZ4 (at better ratios)"}, {"id": 782122195, "item_id": 48882637, "url": "https://news.ycombinator.com/item?id=48882637", "by": "not-a-llm", "date": "2026-07-12", "stance": "compare", "quote": "gzip is competitive with zstd", "excerpt": "not the greatest benchmark, bzip2 many times beats xz, gzip is competitive with zstd maybe it used default compression levels, and the defaults are not \"matched\" fumny because author also wrote this post: https://www.elliotcsmith.com/how-to-avoid-picking-terrible-metrics/", "context": "", "use_case": "", "switched_from": "", "story": "Autoresearch, Claude and Constrained Optimization"}, {"id": 779163168, "item_id": 48697698, "url": "https://news.ycombinator.com/item?id=48697698", "by": "chickenbig", "date": "2026-06-27", "stance": "recommend", "quote": "Even compression level 1 or 2 is pretty good.", "excerpt": "Even compression level 1 or 2 is pretty good. I once used https://github.com/google/riegeli and a low zstd compression level to store large quantities of protobuf data in an efficient manner (in terms of CPU, RAM and streaming to disk). Shame Riegeli is not well known, not well documented and does not have many tests.", "context": "", "use_case": "data compression", "switched_from": "", "story": "Hellishly Slow Level 13 Deflate Compression"}, {"id": 779150737, "item_id": 48696921, "url": "https://news.ycombinator.com/item?id=48696921", "by": "pella", "date": "2026-06-27", "stance": "compare", "quote": "10% faster compression speed and 70% faster decompression speed compared to Zstandard level 1", "excerpt": "Please check the OpenZL v0.2 + Silesia corpus benchmark. \"OpenZL to offer 10% faster compression speed and 70% faster decompression speed compared to Zstandard level 1 on the Silesia corpus in our benchmarks.\" \"OpenZL now ships its own LZ codec, exposed as ZL_GRAPH_LZ, and the serial profile in zli. It is still being actively developed to expand its feature set and improve performance on small inputs.\" https://github", "context": "", "use_case": "", "switched_from": "", "story": "Hellishly Slow Level 13 Deflate Compression"}, {"id": 778926449, "item_id": 48682903, "url": "https://news.ycombinator.com/item?id=48682903", "by": "spidy__", "date": "2026-06-25", "stance": "compare", "quote": "zstd - 28mb", "excerpt": "1. I wrote the content as what i want to mention in the documentation and just used AI to polish it so that its easy to understand, is it hard to understand the documentation right now? 2. Have added the link for downloading both the enwik9 slice and the nyc dataset. Apologies I forgot to add it. You can get it from here - https://github.com/samyak112/pym-particles/blob/main/README.md#benchmark-results 3. Other than ", "context": "", "use_case": "", "switched_from": "", "story": "Overfitted a 900KB Transformer to Compress a 100MB CSV into 7MB"}, {"id": 777773425, "item_id": 48610839, "url": "https://news.ycombinator.com/item?id=48610839", "by": "sltkr", "date": "2026-06-20", "stance": "compare", "quote": "practical tools that intend to balance compression ratio with limitation on CPU speed / memory, like zstd", "excerpt": "compression = prediction + entropy coding was already an insight from Claude Shannon in the 1950s Since LLM are inherently token predictors, that makes using them for losless compression almost trivial. For something close to the state of the art see e.g. Fabrice Bellard (of course) ts_zip: https://bellard.org/ts_zip/ I think some of the confusion comes from the fact that there is a pretty big difference between the ", "context": "", "use_case": "", "switched_from": "", "story": "Data Compression Explained"}, {"id": 771084320, "item_id": 48192770, "url": "https://news.ycombinator.com/item?id=48192770", "by": "KerrickStaley", "date": "2026-05-19", "stance": "recommend", "quote": "switching the compression from Deflate to Zstandard", "excerpt": "You can also make ZIP files smaller by switching the compression from Deflate to Zstandard. In the one case I tried this, this resulted in a 60% file size decrease [1]. Unfortunately Info-ZIP which provides the unzip command hasn't had a release in 18 years, so it doesn't support this newer compression/decompression method. You have to use 7-Zip instead. [1] https://github.com/UKGovernmentBEIS/inspect_ai/pull/3145", "context": "", "use_case": "smaller ZIP files", "switched_from": "", "story": "Make ZIP files smaller with ZIP Shrinker"}, {"id": 768770736, "item_id": 48048171, "url": "https://news.ycombinator.com/item?id=48048171", "by": "xxs", "date": "2026-05-07", "stance": "recommend", "quote": "", "excerpt": "I love zstd as the next guy and I do use zstd solo for the most part. I had a talk on it few years back too (incl. using the lib directly from Java, massively decreasing log storage, and so on). Why use it w/ 7-zip though. 7-zip archives multiple files/directories and supports encryption. It has the UI too.. On Windows there is NanaZip that's available in the microsoft store which has been graced by corporate for use", "context": "", "use_case": "Compression", "switched_from": "", "story": "SQLite Is a Library of Congress Recommended Storage Format"}, {"id": 768752272, "item_id": 48047017, "url": "https://news.ycombinator.com/item?id=48047017", "by": "tnelsond4", "date": "2026-05-07", "stance": "recommend", "quote": "", "excerpt": "I love using zstd, it's so fast to decompress. I especially like that the JavaScript decoder is 8kb and still really fast. Though the 25kb wasm decoders are about twice as fast. What are the advantages or reasons to use zstd in a 7z container versus just .zst?", "context": "", "use_case": "Compression for archives", "switched_from": "", "story": "SQLite Is a Library of Congress Recommended Storage Format"}, {"id": 768741425, "item_id": 48046339, "url": "https://news.ycombinator.com/item?id=48046339", "by": "xxs", "date": "2026-05-07", "stance": "neutral", "quote": "Personally I use only zstd with .7z", "excerpt": ".7z being there just discredits the entire process. The underlying compression algorithm is a free-hand one and can be anything[0], or contain bugs and exploits[1]. Personally I use only zstd with .7z which is 'non-standard' by the official (Russian) release. [0]: https://7-zip.org/7z.html [1]: CVE-2025-0411", "context": "", "use_case": "compression", "switched_from": "", "story": "SQLite Is a Library of Congress Recommended Storage Format"}, {"id": 764588737, "item_id": 47786796, "url": "https://news.ycombinator.com/item?id=47786796", "by": "tnelsond4", "date": "2026-04-15", "stance": "recommend", "quote": "compressed those with zstd and decompressed them via wasm on every load", "excerpt": "I love SQLite, I love the idea, I love having something mature and lightweight, but like the author I discovered it's overkill or dare I say insufficient for certain use cases. I was building a client side dictionary app with search functionality and thought that using sqlite's wasm port would be the perfect solution. I used SQLite for a couple years and it started to wear on me. The database files were bigger than t", "context": "", "use_case": "compressing indexes", "switched_from": "", "story": "Do You Even Need a Database?"}, {"id": 761902480, "item_id": 47618905, "url": "https://news.ycombinator.com/item?id=47618905", "by": "lights0123", "date": "2026-04-02", "stance": "compare", "quote": "", "excerpt": "They moved on to Courgette, then to Zucchini: https://chromium.googlesource.com/chromium/src/+/HEAD/components/zucchini/ These are optimized for compiled code though.", "context": "", "use_case": "Binary diff compression", "switched_from": "", "story": "Zstandard Across the Stack"}, {"id": 761868768, "item_id": 47616798, "url": "https://news.ycombinator.com/item?id=47616798", "by": "nasretdinov", "date": "2026-04-02", "stance": "compare", "quote": "I wonder how it compares to e.g. the library that Chrome uses", "excerpt": "Using zstd for binary diffs is something I would never expect. I wonder how it compares to e.g. the library that Chrome uses: https://blog.chromium.org/2009/07/smaller-is-faster-and-safer-too.html?m=1 (this is from 2009 so maybe they made it better since)", "context": "", "use_case": "", "switched_from": "", "story": "Zstandard Across the Stack"}, {"id": 759251760, "item_id": 47453235, "url": "https://news.ycombinator.com/item?id=47453235", "by": "theandrewbailey", "date": "2026-03-20", "stance": "recommend", "quote": "While it might not compress as small, it decompresses way faster", "excerpt": "> If the client passes br in the Accept-Encoding header the Go server serves the Brotli-compressed version of the static file, otherwise it serves the uncompressed version. Have you considered zstd? It's in most browsers.[0] While it might not compress as small, it decompresses way faster, and might make up for the few extra bytes sent. [0] https://caniuse.com/?search=zstd", "context": "", "use_case": "web asset compression", "switched_from": "", "story": "A static site generator and website transferring under 20kB"}, {"id": 758186881, "item_id": 47386680, "url": "https://news.ycombinator.com/item?id=47386680", "by": "adrian_b", "date": "2026-03-15", "stance": "recommend", "quote": "Of course for other kinds of files, zstd wins", "excerpt": "Any claims about compressing programs are extremely data-dependent so any general claims will be false for certain test cases. I do a lot of data compression and decompression, and I would have liked a lot to find a magic algorithm that works better than all others, to simplify my work. After extensive tests I have found no such algorithm. Depending on the input files and depending on the compromise between compressi", "context": "", "use_case": "", "switched_from": "", "story": "An Ode to Bzip"}, {"id": 758161008, "item_id": 47385063, "url": "https://news.ycombinator.com/item?id=47385063", "by": "bonzini", "date": "2026-03-15", "stance": "compare", "quote": "zstd is pretty much Pareto optimal at all settings when looking at size vs. decompression speed", "excerpt": "> meant to have good ratios and good (de)compression speeds as compared to other tools That does not mean it's Pareto optimal; Pareto-optimality forms a curve and while zstd, LZMA, LZ4, ZPAQ all want to be as close as possible to the curve, they focus on different parts of it. In particular zstd tries to stay on the part of the curve where the curve has slope close to 1, while LZMA and LZ4 focus on opposite sides ___", "context": "", "use_case": "", "switched_from": "", "story": "An Ode to Bzip"}, {"id": 758152320, "item_id": 47384520, "url": "https://news.ycombinator.com/item?id=47384520", "by": "lucb1e", "date": "2026-03-14", "stance": "neutral", "quote": "So Pareto by design, just not using that name", "excerpt": "> > Pareto optimality is a situation where no [\u2026] preference criterion can be better off without making at least one [\u2026] preference criterion worse off [\u2026]. (Omissions theirs.) Wasn't that zstandard's stated goal? Not very surprising that it has this property, also considering it's much newer (2015) than the established tools like gzip (1992), bzip2 (1996), LZMA as used by xz utils (1999) Edit: https://github.com/fac", "context": "", "use_case": "", "switched_from": "", "story": "An Ode to Bzip"}, {"id": 758121888, "item_id": 47382618, "url": "https://news.ycombinator.com/item?id=47382618", "by": "shawn_w", "date": "2026-03-14", "stance": "recommend", "quote": "And zstd is multi threaded from the beginning.", "excerpt": "There's also a parallel version of bzip2, pbzip2. https://man.archlinux.org/man/pbzip2.1.en And zstd is multi threaded from the beginning.", "context": "", "use_case": "", "switched_from": "", "story": "An Ode to Bzip"}, {"id": 758089696, "item_id": 47380606, "url": "https://news.ycombinator.com/item?id=47380606", "by": "0cf8612b2e1e", "date": "2026-03-14", "stance": "recommend", "quote": "There was an analysis which argued that zstd is pareto optimal.", "excerpt": "There was an analysis which argued that zstd is pareto optimal. https://insanity.industries/post/pareto-optimal-compression/", "context": "", "use_case": "", "switched_from": "", "story": "An Ode to Bzip"}, {"id": 758075248, "item_id": 47379703, "url": "https://news.ycombinator.com/item?id=47379703", "by": "laurencerowe", "date": "2026-03-14", "stance": "recommend", "quote": "", "excerpt": "lz4 can still be the right choice when decompression speed matters. It's almost twice as fast at decompression with similar compression ratios to zstd's fast setting. https://github.com/facebook/zstd?tab=readme-ov-file#benchmarks", "context": "", "use_case": "Compression", "switched_from": "", "story": "An Ode to Bzip"}, {"id": 756768817, "item_id": 47298051, "url": "https://news.ycombinator.com/item?id=47298051", "by": "NahumResearch09", "date": "2026-03-08", "stance": "compare", "quote": "while Zstd compresses 15 percent smaller but takes 3.4 milliseconds to answer", "excerpt": "I built STRIQ because decompressing time series CSVs just to get a simple average at work was driving me crazy. It is a pure C11 lossy compression library with zero dependencies that evaluates closed form algebra directly on compressed data using piecewise linear approximation. Timestamps are lossless and you set a max error bound for the lossy float64 values. On a 420K row climate dataset a mean query takes 0.4 micr", "context": "", "use_case": "", "switched_from": "", "story": "Show HN: Striq \u2013 Query lossy time series in C at 0.4\u00b5s without decompressing"}, {"id": 755595377, "item_id": 47224711, "url": "https://news.ycombinator.com/item?id=47224711", "by": "hbina085", "date": "2026-03-02", "stance": "neutral", "quote": "ZSTD (attempting a rewrite in Rust)", "excerpt": "Location: South East Asia Remote: Yes Willing to relocate: Yes Technologies: Rust, C/C++, Python, pretty open to anything really R\u00e9sum\u00e9/CV: On request. Email: hanif.ariffin.4326@gmail.com Blog: https://hbina.github.io/ GitHub: https://github.com/hbina I have 5 years professional software development experience with a history of open source contributions. I have experience developing low latency (<500ns) library, kern", "context": "", "use_case": "Compression library rewrite", "switched_from": "", "story": "Ask HN: Who wants to be hired? (March 2026)"}]}