{"slug": "ik-llama-cpp", "name": "ik_llama.cpp", "category": "ai", "category_name": "AI models and assistants", "description": "", "recommenders": 3, "commenters": 11, "warners": 0, "score": 4.591, "rising": -0.98, "first": "2026-01-06", "latest": "2026-08-23", "why": "ikllama.cpp is a fork of llama.cpp which specializes on CPU inference", "why_is_quote": true, "has_verdict": false, "sentiment": {"recommend": 3, "neutral": 7, "compare": 2, "warn": 0}, "spark": [0, 0, 2, 0, 0, 0, 0, 1, 0, 0, 0, 0], "listed": true, "loved": [], "caveat": "", "short_quote": "ikllama.cpp is a fork of llama.cpp which specializes on CPU inference", "kind": "", "icon": false, "domain": null, "github_repo": "github.com/ikawrakow/ik_llama.cpp", "official_url": "https://github.com/ikawrakow/ik_llama.cpp", "pricing": "", "aliases": ["ik_llama.cpp"], "verdict": null, "praise": [{"label": "", "summary": "", "quotes": [{"id": 779558656, "item_id": 48722416, "url": "https://news.ycombinator.com/item?id=48722416", "by": "Catloafdev", "date": "2026-06-29", "stance": "recommend", "quote": "Hello, it's the internet calling, today is that day.", "excerpt": "Hello, it's the internet calling, today is that day. https://github.com/ikawrakow/ik_llama.cpp Edit: it's gonna be slow if you're not using any VRAM. But it's possible. Software isn't going to speed that up anytime soon, it's just a hardware bandwidth limit.", "context": "", "use_case": "CPU inference for large models", "switched_from": "", "story": "Qwen 3.6 27B is the sweet spot for local development"}, {"id": 748811696, "item_id": 46800731, "url": "https://news.ycombinator.com/item?id=46800731", "by": "kgeist", "date": "2026-01-28", "stance": "recommend", "quote": "ikllama.cpp is a fork of llama.cpp which specializes on CPU inference", "excerpt": "ikllama.cpp is a fork of llama.cpp which specializes on CPU inference, some benchmarks from 1 year ago: https://github.com/ikawrakow/ik_llama.cpp/discussions/164", "context": "", "use_case": "CPU LLM inference", "switched_from": "", "story": "I have written gemma3 inference in pure C"}, {"id": 744339729, "item_id": 46521233, "url": "https://news.ycombinator.com/item?id=46521233", "by": "LargoLasskhyfv", "date": "2026-01-06", "stance": "recommend", "quote": "Have you tried anything with https://codeberg.org/ikawrakow/illama", "excerpt": "Have you tried anything with https://codeberg.org/ikawrakow/illama https://github.com/ikawrakow/ik_llama.cpp and their 4Bit-quants? Or maybe even Microsofts Bitnet? https://github.com/microsoft/BitNet https://github.com/ikawrakow/ik_llama.cpp/pull/337 https://huggingface.co/microsoft/bitnet-b1.58-2B-4T-gguf ? That would be an interesting comparison for running local LLMs on such low-end/edge-devices. Or common office", "context": "", "use_case": "", "switched_from": "", "story": "A 30B Qwen Model Walks into a Raspberry Pi and Runs in Real Time"}]}], "caveats": [], "who": [], "tags": [], "use_cases": ["local llm inference", "running local llms", "running llms on cpu", "cpu inference for large models", "cpu llm inference"], "compared": [{"slug": "llama-cpp", "name": "llama.cpp", "co": 6, "switched_from": 0, "switched_to": 0}, {"slug": "qwen", "name": "Qwen", "co": 3, "switched_from": 0, "switched_to": 0}, {"slug": "vllm", "name": "vLLM", "co": 2, "switched_from": 0, "switched_to": 0}, {"slug": "pi", "name": "Pi", "co": 2, "switched_from": 0, "switched_to": 0}], "makers": [], "threads": [{"url": "https://news.ycombinator.com/item?id=49402232", "title": "Why your local LLM feels dumber than it is", "mentions": 2}, {"url": "https://news.ycombinator.com/item?id=48995261", "title": "Laguna S 2.1", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=48922434", "title": "Running Gemma 4 26B at 5 tokens/SEC on a 13-year-old Xeon with no GPU", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=48721903", "title": "Qwen 3.6 27B is the sweet spot for local development", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=48542100", "title": "Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=48353348", "title": "A 10 year old Xeon is all you need (for 26B-A4B MTP Drafters without GPU)", "mentions": 1}], "sentiment_by_month": {"2025-11": {}, "2025-12": {}, "2026-01": {"recommend": 2}, "2026-02": {}, "2026-03": {"neutral": 1}, "2026-04": {"neutral": 2}, "2026-05": {}, "2026-06": {"neutral": 1, "compare": 1, "recommend": 1}, "2026-07": {"neutral": 2}, "2026-08": {"neutral": 1, "compare": 1}, "2026-09": {}, "2026-10": {}}, "quotes": [{"id": 790488193, "item_id": 49405512, "url": "https://news.ycombinator.com/item?id=49405512", "by": "throwdbaaway", "date": "2026-08-23", "stance": "compare", "quote": "Q8_0 from llama.cpp / ik_llama.cpp should also work better than FP8 from vllm", "excerpt": "As for KV cache quantization, Q8_0 from llama.cpp / ik_llama.cpp should also work better than FP8 from vllm (see https://github.com/vllm-project/vllm/issues/33480#issuecomment-4201272360).", "context": "", "use_case": "", "switched_from": "", "story": "Why your local LLM feels dumber than it is"}, {"id": 790471298, "item_id": 49404456, "url": "https://news.ycombinator.com/item?id=49404456", "by": "zepearl", "date": "2026-08-22", "stance": "neutral", "quote": "(and \"ik_llama.cpp\" in very few cases)", "excerpt": "I know nothing about \"vLLM\" -> anything better compared to \"llama.cpp\"? I started with \"Ollama\" (precompiled version) and it worked and was good enough to understand the very basics. Then I downloaded the sourcecode of \"llama.cpp\", compiled it with specific compilation options for my GPUs (CUDA/nVidia using proprietary module on Gentoo Linux) & CPU (AMD), and the same model ran twice as fast -> since then I stuck wit", "context": "", "use_case": "running local LLMs", "switched_from": "", "story": "Why your local LLM feels dumber than it is"}, {"id": 784059104, "item_id": 49003694, "url": "https://news.ycombinator.com/item?id=49003694", "by": "throwdbaaway", "date": "2026-07-22", "stance": "neutral", "quote": "It works, thanks to https://github.com/ikawrakow/ik_llama.cpp/pull/1911", "excerpt": "It works, thanks to https://github.com/ikawrakow/ik_llama.cpp/pull/1911, which got merged in early June. However, there might still be some issue with the chat template.", "context": "", "use_case": "", "switched_from": "", "story": "Laguna S 2.1"}, {"id": 782760384, "item_id": 48922524, "url": "https://news.ycombinator.com/item?id=48922524", "by": "neomindryan", "date": "2026-07-15", "stance": "neutral", "quote": "The fix is open upstream as PR #2138", "excerpt": "Author here, it looks like my original comment was flagged for some reason. The fix is open upstream as PR #2138 (https://github.com/ikawrakow/ik_llama.cpp/pull/2138)", "context": "", "use_case": "running LLMs on CPU", "switched_from": "", "story": "Running Gemma 4 26B at 5 tokens/SEC on a 13-year-old Xeon with no GPU"}, {"id": 779558656, "item_id": 48722416, "url": "https://news.ycombinator.com/item?id=48722416", "by": "Catloafdev", "date": "2026-06-29", "stance": "recommend", "quote": "Hello, it's the internet calling, today is that day.", "excerpt": "Hello, it's the internet calling, today is that day. https://github.com/ikawrakow/ik_llama.cpp Edit: it's gonna be slow if you're not using any VRAM. But it's possible. Software isn't going to speed that up anytime soon, it's just a hardware bandwidth limit.", "context": "", "use_case": "CPU inference for large models", "switched_from": "", "story": "Qwen 3.6 27B is the sweet spot for local development"}, {"id": 776786227, "item_id": 48549139, "url": "https://news.ycombinator.com/item?id=48549139", "by": "henrixd", "date": "2026-06-15", "stance": "compare", "quote": "switching to ik_llama.cpp fork", "excerpt": "I have been heavily relying on Qwen3.6-27B-UD-Q4_K_XL.gguf -model and Pi agent (https://pi.dev/) for local tasks and coding. I have used llama-cpp-turboquant fork with some custom cherrypicked MTP patches from another fork. I'm running this on V100 32GB (~900GB/s memory bandwidth) with 200,000 context window, --spec-type mpt --spec-draft-n-max 3 --spec-draft-n-min 0 --cache-type-k turbo3 --cache-type-v turbo3 to ment", "context": "", "use_case": "", "switched_from": "", "story": "Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?"}, {"id": 773735265, "item_id": 48358454, "url": "https://news.ycombinator.com/item?id=48358454", "by": "Aurornis", "date": "2026-06-01", "stance": "neutral", "quote": "ik_llama includes llama-sweep-bench", "excerpt": "llama.cpp includes a benchmarking tool called llama-bench https://github.com/ggml-org/llama.cpp/blob/master/tools/llama-bench/README.md ik_llama includes llama-sweep-bench https://github.com/ikawrakow/ik_llama.cpp/blob/main/examples/sweep-bench/README.md When comparing hardware, the output of these tools is very helpful to let others put it into context. The post says the output is \"reading speed\" but knowing the pre", "context": "", "use_case": "Local LLM inference", "switched_from": "", "story": "A 10 year old Xeon is all you need (for 26B-A4B MTP Drafters without GPU)"}, {"id": 763996226, "item_id": 47749764, "url": "https://news.ycombinator.com/item?id=47749764", "by": "intothemild", "date": "2026-04-13", "stance": "neutral", "quote": "backends (llama.cpp, ik_llama.cpp) have been updated", "excerpt": "Koji, a AI Backend Orchestration/Managment/Proxy layer: https://github.com/danielcherubini/koji I've been running models on my homelab for a bit now, but none of the available options out there was what I wanted. I wanted something that I could command from the CLI, API, or Web, so have an agent go in and do work remotely via SSH or myself via a web interface. I wanted the ability to know if models have been updated,", "context": "", "use_case": "", "switched_from": "", "story": "Ask HN: What Are You Working On? (April 2026)"}, {"id": 761767185, "item_id": 47610449, "url": "https://news.ycombinator.com/item?id=47610449", "by": "3abiton", "date": "2026-04-01", "stance": "neutral", "quote": "did tests on the quality of turboquant-3 compared to IQ4 quantized models", "excerpt": "Their paper TurboQuant (TQ) is not new per say. It's released last year, and heavily rehash of old ideas that were released a year prior (RabitQ). There is also [a bit of drama](https://openreview.net/forum?id=tO3ASKZlok) there that boils down to what it seems a bit of malpractice for google's researchers. TQ does few things: it claims better compression quality and speed, and better KV cache handling. Currently KV c", "context": "", "use_case": "", "switched_from": "", "story": "Anthropic: Claude Code users hitting usage limits 'way faster than expected'"}, {"id": 756043808, "item_id": 47252738, "url": "https://news.ycombinator.com/item?id=47252738", "by": "aruggirello", "date": "2026-03-04", "stance": "neutral", "quote": "I went with ik_llama.cpp", "excerpt": "I'm getting ~27 tok/s on my 5060 Ti 16Gb (just bought for ~\u20ac600) on an old 8th gen Core 7 (2017, with DDR4 memory), Kubuntu 24.04, proprietary nVidia drivers (590); I'm using the iGPU for the graphics, so the card is free for inference - I'm using Qwen3.5-27B-UD-IQ3_XXS, and almost 72k context - I went with ik_llama.cpp [0] - Qwen3.5 is very promising, I'm definitely going to use it. I also tried ZSE [1] but experien", "context": "", "use_case": "local LLM inference", "switched_from": "", "story": "Something is afoot in the land of Qwen"}, {"id": 748811696, "item_id": 46800731, "url": "https://news.ycombinator.com/item?id=46800731", "by": "kgeist", "date": "2026-01-28", "stance": "recommend", "quote": "ikllama.cpp is a fork of llama.cpp which specializes on CPU inference", "excerpt": "ikllama.cpp is a fork of llama.cpp which specializes on CPU inference, some benchmarks from 1 year ago: https://github.com/ikawrakow/ik_llama.cpp/discussions/164", "context": "", "use_case": "CPU LLM inference", "switched_from": "", "story": "I have written gemma3 inference in pure C"}, {"id": 744339729, "item_id": 46521233, "url": "https://news.ycombinator.com/item?id=46521233", "by": "LargoLasskhyfv", "date": "2026-01-06", "stance": "recommend", "quote": "Have you tried anything with https://codeberg.org/ikawrakow/illama", "excerpt": "Have you tried anything with https://codeberg.org/ikawrakow/illama https://github.com/ikawrakow/ik_llama.cpp and their 4Bit-quants? Or maybe even Microsofts Bitnet? https://github.com/microsoft/BitNet https://github.com/ikawrakow/ik_llama.cpp/pull/337 https://huggingface.co/microsoft/bitnet-b1.58-2B-4T-gguf ? That would be an interesting comparison for running local LLMs on such low-end/edge-devices. Or common office", "context": "", "use_case": "", "switched_from": "", "story": "A 30B Qwen Model Walks into a Raspberry Pi and Runs in Real Time"}]}