{"slug": "turboquant", "name": "TurboQuant", "category": "search", "category_name": "Search", "description": "", "recommenders": 5, "commenters": 15, "warners": 8, "score": -2.285, "rising": -1.64, "first": "2026-03-25", "latest": "2026-08-18", "why": "This sounds great! TurboQuant does KV cache compression using quantization via rotations", "why_is_quote": true, "has_verdict": false, "sentiment": {"recommend": 5, "neutral": 2, "compare": 0, "warn": 8}, "spark": [0, 0, 0, 0, 2, 3, 0, 0, 0, 0, 0, 0], "listed": true, "loved": [], "caveat": "", "short_quote": "This sounds great! TurboQuant does KV cache compression using quantization via rotations", "kind": "", "icon": false, "domain": null, "github_repo": "github.com/tonbistudio/turboquant-pytorch", "official_url": "https://github.com/tonbistudio/turboquant-pytorch", "pricing": "", "aliases": ["TurboQuant"], "verdict": null, "praise": [{"label": "", "summary": "", "quotes": [{"id": 767229888, "item_id": 47951868, "url": "https://news.ycombinator.com/item?id=47951868", "by": "simjnd", "date": "2026-04-29", "version": null, "stance": "recommend", "quote": "People have successfully used TurboQuant to quantize model weights", "excerpt": "> The one thing I would want everyone curious about local LLMs to know is that being able to run a model and being able to run a model fast are two very different thresholds. You can get these models to run on a 128GB Mac, but we need to first tell if Q4 retains enough quality (models have different sensitivities to quantization) and how fast it runs. Very valid. This is an active area of research, and there are a lo", "context": "", "use_case": "model weight quantization", "switched_from": "", "story": "Mistral Medium 3.5"}, {"id": 765980080, "item_id": 47873755, "url": "https://news.ycombinator.com/item?id=47873755", "by": "sleepyeldrazi", "date": "2026-04-23", "version": null, "stance": "recommend", "quote": "I have been getting good results with IQ4_NL and TurboQuant at 4bits", "excerpt": "I have been getting good results with IQ4_NL and TurboQuant at 4bits on 24gb (3090). It easily fits 256k with that setup, but it starts slowing down quite a bit after 80-100k. Quality in my testing is also still good: - Coding task test: https://github.com/sleepyeldrazi/llm_programming_tests/ - Design task test: https://github.com/sleepyeldrazi/llm-design-showcase Coding was against minimax-m2.7 and glm-5, and the de", "context": "", "use_case": "", "switched_from": "", "story": "Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model"}, {"id": 761716016, "item_id": 47607251, "url": "https://news.ycombinator.com/item?id=47607251", "by": "aegis_camera", "date": "2026-04-01", "version": null, "stance": "recommend", "quote": "it's so cool, so we started to integrate them", "excerpt": "Thanks, we are not large R&D lab, limited resources. We were working on a product with is a Local VLM first BYOD when you want Video Security application, our users requested to have a MLX backend benchmark comparison, we tried hard to not deliver with Python in the application bundle, so we searched for a pure binary based MLX implementation the results shown we need to build one. It took us two weeks to get it work", "context": "small team with limited resources", "use_case": "", "switched_from": "", "story": "TurboQuant KV Compression and SSD Expert Streaming for M5 Pro and IOS"}, {"id": 760769632, "item_id": 47548102, "url": "https://news.ycombinator.com/item?id=47548102", "by": "skybrian", "date": "2026-03-27", "version": null, "stance": "recommend", "quote": "TurboQuant [1] looks pretty promising", "excerpt": "These \"truths\" are more like concerns. Skills do atrophy if you don't practice them, but also, refreshing your memory about some technology you haven't used in a while, or even learning something new, is easier than ever. You can ask the AI questions and try things out yourself very easily. Maybe \"just in time\" learning isn't good enough, but that's more of a concern based on speculation than a truth. AI is being sub", "context": "", "use_case": "Inference cost reduction", "switched_from": "", "story": "Some uncomfortable truths about AI coding agents"}, {"id": 760277120, "item_id": 47517320, "url": "https://news.ycombinator.com/item?id=47517320", "by": "naasking", "date": "2026-03-25", "version": null, "stance": "recommend", "quote": "This sounds great! TurboQuant does KV cache compression using quantization via rotations", "excerpt": "This sounds great! TurboQuant does KV cache compression using quantization via rotations, and ParoQuant [1] does weight compression using quantization via rotations! So we can get 4-bit weights that match bf16 precision, the KV cache goes down to 3 bits per key. This brings larger models and long contexts into the range of \"possibly runnable\" on beefy consumer hardware. [1] https://github.com/z-lab/paroquant", "context": "", "use_case": "", "switched_from": "", "story": "TurboQuant: Redefining AI efficiency with extreme compression"}]}], "caveats": [{"label": "", "summary": "", "quotes": [{"id": 789650272, "item_id": 49353142, "url": "https://news.ycombinator.com/item?id=49353142", "by": "bobmarleybiceps", "date": "2026-08-18", "version": null, "stance": "warn", "quote": "people should read turboquant's open review comments", "excerpt": "people should read turboquant's open review comments: https://openreview.net/forum?id=tO3ASKZlok", "context": "", "use_case": "", "switched_from": "", "story": "Turbovec \u2013 Google's TurboQuant for vector search in Rust"}, {"id": 767604944, "item_id": 47975309, "url": "https://news.ycombinator.com/item?id=47975309", "by": "yinksta", "date": "2026-05-01", "version": null, "stance": "warn", "quote": "the Turboquant debaucle is entirely on Google doing incredible amount of academic misconduct", "excerpt": "the Turboquant debaucle is entirely on Google doing incredible amount of academic misconduct and general incompetence. A) the QLJ thing they added is useless and the code they released didn't even include it since it makes the results worse B) the blogpost about turboquant was ai generated and stated that turboquant used a polar transformation when it doesn't, so for the first 2 weeks people thought turboquant involv", "context": "", "use_case": "", "switched_from": "", "story": "Advanced Quantization Algorithm for LLMs"}, {"id": 766688240, "item_id": 47918015, "url": "https://news.ycombinator.com/item?id=47918015", "by": "bigyabai", "date": "2026-04-26", "version": null, "stance": "warn", "quote": "TurboQuant looks like a vibe-laundered implementation of EDEN quantization", "excerpt": "> It is more a software problem and the next breakthrough will come from clever algorithms. I don't know if I can agree. The hardware side is extremely sub-optimal on raster-focused GPU architectures like Apple Silicon. If I had to bet, the hardware will improve a lot more than the software will over the next 10 years as more vendors adopt GPGPU characteristics. > You have just seen TurboQuant create promising effici", "context": "", "use_case": "Model quantization", "switched_from": "", "story": "Ask HN: Will local models on normal hardware ever compete?"}, {"id": 765183328, "item_id": 47823958, "url": "https://news.ycombinator.com/item?id=47823958", "by": "gajjanag", "date": "2026-04-19", "version": null, "stance": "warn", "quote": "TurboQuant is known across the industry to not be state of the art", "excerpt": "TurboQuant is known across the industry to not be state of the art. There are superior schemes for KV quant at every bitrate. Eg, SpectralQuant: https://github.com/Dynamis-Labs/spectralquant among many, many papers. > Given that TurboQuant results in a 6x reduction in memory usage for KV caches All depends on baseline. The \"6x\" is by stylistic comparison to a BF16 KV cache; not a state of the art 8 or 4 bit KV cache ", "context": "", "use_case": "", "switched_from": "", "story": "The RAM shortage could last years"}]}], "who": [{"by": "aegis_camera", "context": "small team with limited resources", "url": "https://news.ycombinator.com/item?id=47607251", "date": "2026-04-01"}], "tags": [], "use_cases": ["model weight quantization", "model quantization", "kv cache compression", "inference cost reduction", "kv cache quantization"], "compared": [], "makers": [{"id": 766717392, "item_id": 47919837, "url": "https://news.ycombinator.com/item?id=47919837", "by": "om8", "date": "2026-04-27", "version": null, "stance": "compare", "quote": "", "excerpt": "https://docs.vllm.ai/en/v0.20.0/api/vllm/model_executor/layers/quantization/turboquant/#vllm.model_executor.layers.quantization.turboquant `vllm.model_executor.layers.quantization.turboquant` > The technique implemented here consists of the scalar case of the HIGGS quantization method (Malinovskii et al., \"Pushing the Limits of Large Language Model Quantization via the Linearity Theorem\", NAACL 2025; preprint arXiv:2", "context": "", "use_case": "", "switched_from": "", "story": "TurboQuant: A First-Principles Walkthrough"}, {"id": 761260416, "item_id": 47578776, "url": "https://news.ycombinator.com/item?id=47578776", "by": "rettepelo", "date": "2026-03-30", "version": null, "stance": "recommend", "quote": "The memory usage is about 3x less compared to a simple array of floats.", "excerpt": "I'm developing a .Net C# database engine, that includes a vector index for semantic searches. This weekend I made a first attempt at implementing the new TurboQuant from Google: https://research.google/blog/turboquant-redefining-ai-efficiency-with-extreme-compression/ Here is a link to test it: https://turboquant.relatude.com/ The memory usage is about 3x less compared to a simple array of floats. There are links to ", "context": "", "use_case": "", "switched_from": "", "story": "A C# vector database using TurboQuant from Google Research"}], "threads": [{"url": "https://news.ycombinator.com/item?id=47513475", "title": "TurboQuant: Redefining AI efficiency with extreme compression", "mentions": 3}, {"url": "https://news.ycombinator.com/item?id=49349898", "title": "Turbovec \u2013 Google's TurboQuant for vector search in Rust", "mentions": 2}, {"url": "https://news.ycombinator.com/item?id=47949642", "title": "Mistral Medium 3.5", "mentions": 2}, {"url": "https://news.ycombinator.com/item?id=47822414", "title": "The RAM shortage could last years", "mentions": 2}, {"url": "https://news.ycombinator.com/item?id=47972659", "title": "Advanced Quantization Algorithm for LLMs", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=47916890", "title": "TurboQuant: A First-Principles Walkthrough", "mentions": 1}], "sentiment_by_month": {"2025-11": {}, "2025-12": {}, "2026-01": {}, "2026-02": {}, "2026-03": {"neutral": 1, "recommend": 2, "warn": 1}, "2026-04": {"recommend": 3, "warn": 4, "neutral": 1}, "2026-05": {"warn": 1}, "2026-06": {}, "2026-07": {}, "2026-08": {"warn": 2}, "2026-09": {}, "2026-10": {}}, "versions": [], "version_unnamed": 0, "family_ref": null, "runs_on": null, "lines": [], "quotes": [{"id": 789680800, "item_id": 49355050, "url": "https://news.ycombinator.com/item?id=49355050", "by": "esafak", "date": "2026-08-18", "version": null, "stance": "warn", "quote": "", "excerpt": "tl,dr: there is an allegedly better alternative, and it's already implemented everywhere: https://github.com/VectorDB-NTU/RaBitQ-Library#rabitq-in-industry", "context": "", "use_case": "", "switched_from": "", "story": "Turbovec \u2013 Google's TurboQuant for vector search in Rust"}, {"id": 789650272, "item_id": 49353142, "url": "https://news.ycombinator.com/item?id=49353142", "by": "bobmarleybiceps", "date": "2026-08-18", "version": null, "stance": "warn", "quote": "people should read turboquant's open review comments", "excerpt": "people should read turboquant's open review comments: https://openreview.net/forum?id=tO3ASKZlok", "context": "", "use_case": "", "switched_from": "", "story": "Turbovec \u2013 Google's TurboQuant for vector search in Rust"}, {"id": 767604944, "item_id": 47975309, "url": "https://news.ycombinator.com/item?id=47975309", "by": "yinksta", "date": "2026-05-01", "version": null, "stance": "warn", "quote": "the Turboquant debaucle is entirely on Google doing incredible amount of academic misconduct", "excerpt": "the Turboquant debaucle is entirely on Google doing incredible amount of academic misconduct and general incompetence. A) the QLJ thing they added is useless and the code they released didn't even include it since it makes the results worse B) the blogpost about turboquant was ai generated and stated that turboquant used a polar transformation when it doesn't, so for the first 2 weeks people thought turboquant involv", "context": "", "use_case": "", "switched_from": "", "story": "Advanced Quantization Algorithm for LLMs"}, {"id": 767307504, "item_id": 47956719, "url": "https://news.ycombinator.com/item?id=47956719", "by": "sroussey", "date": "2026-04-29", "version": null, "stance": "neutral", "quote": "", "excerpt": "Actually, they published a year ago. Recent was being on official Google blog. https://arxiv.org/abs/2504.19874 https://research.google/blog/turboquant-redefining-ai-efficiency-with-extreme-compression/", "context": "", "use_case": "", "switched_from": "", "story": "Mistral Medium 3.5"}, {"id": 767229888, "item_id": 47951868, "url": "https://news.ycombinator.com/item?id=47951868", "by": "simjnd", "date": "2026-04-29", "version": null, "stance": "recommend", "quote": "People have successfully used TurboQuant to quantize model weights", "excerpt": "> The one thing I would want everyone curious about local LLMs to know is that being able to run a model and being able to run a model fast are two very different thresholds. You can get these models to run on a 128GB Mac, but we need to first tell if Q4 retains enough quality (models have different sensitivities to quantization) and how fast it runs. Very valid. This is an active area of research, and there are a lo", "context": "", "use_case": "model weight quantization", "switched_from": "", "story": "Mistral Medium 3.5"}, {"id": 766688240, "item_id": 47918015, "url": "https://news.ycombinator.com/item?id=47918015", "by": "bigyabai", "date": "2026-04-26", "version": null, "stance": "warn", "quote": "TurboQuant looks like a vibe-laundered implementation of EDEN quantization", "excerpt": "> It is more a software problem and the next breakthrough will come from clever algorithms. I don't know if I can agree. The hardware side is extremely sub-optimal on raster-focused GPU architectures like Apple Silicon. If I had to bet, the hardware will improve a lot more than the software will over the next 10 years as more vendors adopt GPGPU characteristics. > You have just seen TurboQuant create promising effici", "context": "", "use_case": "Model quantization", "switched_from": "", "story": "Ask HN: Will local models on normal hardware ever compete?"}, {"id": 765980080, "item_id": 47873755, "url": "https://news.ycombinator.com/item?id=47873755", "by": "sleepyeldrazi", "date": "2026-04-23", "version": null, "stance": "recommend", "quote": "I have been getting good results with IQ4_NL and TurboQuant at 4bits", "excerpt": "I have been getting good results with IQ4_NL and TurboQuant at 4bits on 24gb (3090). It easily fits 256k with that setup, but it starts slowing down quite a bit after 80-100k. Quality in my testing is also still good: - Coding task test: https://github.com/sleepyeldrazi/llm_programming_tests/ - Design task test: https://github.com/sleepyeldrazi/llm-design-showcase Coding was against minimax-m2.7 and glm-5, and the de", "context": "", "use_case": "", "switched_from": "", "story": "Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model"}, {"id": 765183328, "item_id": 47823958, "url": "https://news.ycombinator.com/item?id=47823958", "by": "gajjanag", "date": "2026-04-19", "version": null, "stance": "warn", "quote": "TurboQuant is known across the industry to not be state of the art", "excerpt": "TurboQuant is known across the industry to not be state of the art. There are superior schemes for KV quant at every bitrate. Eg, SpectralQuant: https://github.com/Dynamis-Labs/spectralquant among many, many papers. > Given that TurboQuant results in a 6x reduction in memory usage for KV caches All depends on baseline. The \"6x\" is by stylistic comparison to a BF16 KV cache; not a state of the art 8 or 4 bit KV cache ", "context": "", "use_case": "", "switched_from": "", "story": "The RAM shortage could last years"}, {"id": 765168032, "item_id": 47823002, "url": "https://news.ycombinator.com/item?id=47823002", "by": "lhl", "date": "2026-04-19", "version": null, "stance": "warn", "quote": "TurboQuant is fine, but not a magic bullet.", "excerpt": "BTW, a number of corrections. The TurboQuant paper was submitted to Arxiv back in April 2025: https://arxiv.org/abs/2504.19874 Current \"TurboQuant\" implementations are about 3.8X-4.9X on compression (w/ the higher end taking some significant hits of GSM8K performance) and with about 80-100% baseline speed (no improvement, regression): https://github.com/vllm-project/vllm/pull/38479 For those not paying attention, it'", "context": "", "use_case": "KV cache compression", "switched_from": "", "story": "The RAM shortage could last years"}, {"id": 762306944, "item_id": 47644184, "url": "https://news.ycombinator.com/item?id=47644184", "by": "netdur", "date": "2026-04-04", "version": null, "stance": "warn", "quote": "my findings is not good, it is not worth it if you cannot use GPU", "excerpt": "I tried TQ for vector search and my findings is not good, it is not worth it if you cannot use GPU, however I got same quality of search as 32f using 8bit quant I wrote ann ext for sqlite, using tq, I do save a lot on space but 32f is still faster despite everything I have tried code here https://github.com/netdur/munind/tree/main/src/tq", "context": "", "use_case": "", "switched_from": "", "story": "Show HN: TurboQuant-WASM \u2013 Google's vector quantization in the browser"}, {"id": 761716016, "item_id": 47607251, "url": "https://news.ycombinator.com/item?id=47607251", "by": "aegis_camera", "date": "2026-04-01", "version": null, "stance": "recommend", "quote": "it's so cool, so we started to integrate them", "excerpt": "Thanks, we are not large R&D lab, limited resources. We were working on a product with is a Local VLM first BYOD when you want Video Security application, our users requested to have a MLX backend benchmark comparison, we tried hard to not deliver with Python in the application bundle, so we searched for a pure binary based MLX implementation the results shown we need to build one. It took us two weeks to get it work", "context": "small team with limited resources", "use_case": "", "switched_from": "", "story": "TurboQuant KV Compression and SSD Expert Streaming for M5 Pro and IOS"}, {"id": 761155824, "item_id": 47572239, "url": "https://news.ycombinator.com/item?id=47572239", "by": "mskkm", "date": "2026-03-30", "version": null, "stance": "warn", "quote": "seems to be a scam", "excerpt": "seems to be a scam \"The TurboQuant paper (ICLR 2026) contains serious issues in how it describes RaBitQ, including incorrect technical claims and misleading theory/experiment comparisons. We flagged these issues to the authors before submission. They acknowledged them, but chose not to fix them. The paper was later accepted and widely promoted by Google, reaching tens of millions of views. We\u2019re speaking up now becau", "context": "", "use_case": "", "switched_from": "", "story": "TurboQuant: Redefining AI efficiency with extreme compression"}, {"id": 760769632, "item_id": 47548102, "url": "https://news.ycombinator.com/item?id=47548102", "by": "skybrian", "date": "2026-03-27", "version": null, "stance": "recommend", "quote": "TurboQuant [1] looks pretty promising", "excerpt": "These \"truths\" are more like concerns. Skills do atrophy if you don't practice them, but also, refreshing your memory about some technology you haven't used in a while, or even learning something new, is easier than ever. You can ask the AI questions and try things out yourself very easily. Maybe \"just in time\" learning isn't good enough, but that's more of a concern based on speculation than a truth. AI is being sub", "context": "", "use_case": "Inference cost reduction", "switched_from": "", "story": "Some uncomfortable truths about AI coding agents"}, {"id": 760277120, "item_id": 47517320, "url": "https://news.ycombinator.com/item?id=47517320", "by": "naasking", "date": "2026-03-25", "version": null, "stance": "recommend", "quote": "This sounds great! TurboQuant does KV cache compression using quantization via rotations", "excerpt": "This sounds great! TurboQuant does KV cache compression using quantization via rotations, and ParoQuant [1] does weight compression using quantization via rotations! So we can get 4-bit weights that match bf16 precision, the KV cache goes down to 3 bits per key. This brings larger models and long contexts into the range of \"possibly runnable\" on beefy consumer hardware. [1] https://github.com/z-lab/paroquant", "context": "", "use_case": "", "switched_from": "", "story": "TurboQuant: Redefining AI efficiency with extreme compression"}, {"id": 760264800, "item_id": 47516550, "url": "https://news.ycombinator.com/item?id=47516550", "by": "carlosvega", "date": "2026-03-25", "version": null, "stance": "neutral", "quote": "https://mesuvash.github.io/blog/2026/turboquant-interactive/", "excerpt": "Is the number of bits per coordinate. So, 1 bit is 2x2 grid. 3 bit is a 64 cell grid (2^3 x 2^3). Here you have a demo. https://mesuvash.github.io/blog/2026/turboquant-interactive/", "context": "", "use_case": "KV cache quantization", "switched_from": "", "story": "TurboQuant: Redefining AI efficiency with extreme compression"}]}