{"slug": "llama-swap", "name": "llama-swap", "category": "ai", "category_name": "AI models and assistants", "description": "Proxy server that swaps local LLM models on demand for llama.cpp and similar backends.", "recommenders": 7, "commenters": 9, "warners": 0, "score": 7.049, "rising": 0.36, "first": "2026-04-15", "latest": "2026-09-24", "why": "Used alongside llama.cpp to swap local models on demand, serving several models from a single endpoint and giving an Ollama-like experience.", "why_is_quote": false, "has_verdict": true, "sentiment": {"recommend": 7, "neutral": 3, "compare": 0, "warn": 0}, "spark": [0, 0, 0, 0, 0, 0, 0, 5, 0, 1, 1, 0], "listed": true, "loved": ["llama.cpp companion", "Dynamic model swapping"], "caveat": "Lemonade alternative", "short_quote": "llama-swap (running as a podman quadlet exposed via tailscale svc) allows running multiple models on a single endpoint.", "icon": true, "domain": null, "github_repo": "github.com/mostlygeek/llama-swap", "official_url": "https://github.com/mostlygeek/llama-swap", "pricing": "open source", "aliases": ["llama-swap"], "verdict": [{"text": "Used alongside llama.cpp to swap local models on demand", "cites": [{"id": 797379844, "item_id": 49836240, "url": "https://news.ycombinator.com/item?id=49836240", "by": "LeBit", "date": "2026-09-24", "stance": "recommend", "quote": "llama.cpp + llama-swap", "excerpt": "I use the following models: - Qwen3.5 9B - Qwen3.6 35B A3B - Qwen3.8 27B - Gemma 4 26B A4B - Gemma 31B - Muse Glimmer 30B I have 48G. The MacMini is used solely for inference. llama.cpp + llama-swap. People are saying local models are crap and serves no purposes. I use them to help me spell check, write emails, write text messages, write JIRA tickets, write PR comments, etc. I also use them in coding agents to comple", "context": "Mac Mini with 48G", "use_case": "model swapping", "switched_from": "", "story": "Best LLM for every budget, updated daily"}, {"id": 787964705, "item_id": 49247794, "url": "https://news.ycombinator.com/item?id=49247794", "by": "computershit", "date": "2026-08-10", "stance": "recommend", "quote": "I use llama.cpp w/ llama-swap", "excerpt": "I use llama.cpp w/ llama-swap https://github.com/ggml-org/llama.cpp https://github.com/mostlygeek/llama-swap", "context": "", "use_case": "swap local models", "switched_from": "", "story": "Meta Muse Glimmer \u2013 open weights 30B local coding model"}, {"id": 776914816, "item_id": 48557176, "url": "https://news.ycombinator.com/item?id=48557176", "by": "jborak", "date": "2026-06-16", "stance": "recommend", "quote": "I use llama-swap to dynamically swap models.", "excerpt": "I already had 2x 5070's that I had purchased a year or more ago, so getting an additional two to fill up the PCIe slots on the motherboard seemed reasonable. I did some math/shopping as well. To get 48GB of VRAM you can get 2x 3090s but that is $3k. A single 5090 is $4k but has 32GB, great for running models like Qwen 27B but maybe nothing else depending on your model settings. Already having 2x 5070, where each card", "context": "", "use_case": "swapping local models", "switched_from": "", "story": "Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?"}]}, {"text": ", serving several models from a single endpoint", "cites": [{"id": 776719777, "item_id": 48544986, "url": "https://news.ycombinator.com/item?id=48544986", "by": "porkloin", "date": "2026-06-15", "stance": "recommend", "quote": "llama-swap (running as a podman quadlet exposed via tailscale svc) allows running multiple models on a single endpoint.", "excerpt": "I have good results with this setup: Hardware: - GPU: AMD 7900xtx, 24gb vram - CPU: AMD 5950x, AM4 - RAM: 64gb DDR4 3600 Software: - OS: Bazzite (atomic fedora - this machine is running Steam \"big picture\" mode on my TV when not in use for LLM tasks) - Virtualization: Podman Quadlets, which allows me to run container images as managed systemd units - Network: tailscale - Inference: llama.cpp vulkan (better performanc", "context": "", "use_case": "multi-model endpoint", "switched_from": "", "story": "Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?"}]}, {"text": " and giving an Ollama-like experience.", "cites": [{"id": 776749922, "item_id": 48546870, "url": "https://news.ycombinator.com/item?id=48546870", "by": "jboss10", "date": "2026-06-15", "stance": "recommend", "quote": "I use llama-swap to emulate the Ollama experience.", "excerpt": "llama.cpp It's faster and more open source. Ollama has some mixed history. I use llama-swap to emulate the Ollama experience.", "context": "", "use_case": "model swapping", "switched_from": "", "story": "Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?"}, {"id": 776056539, "item_id": 48503533, "url": "https://news.ycombinator.com/item?id=48503533", "by": "trollbridge", "date": "2026-06-12", "stance": "recommend", "quote": "Use ollama or llama-swap which is fairly easy to use.", "excerpt": "I am extremely happy with ohmypi, but you could use OpenCode or just keep using Claude Code! DeepSeek-V4-Pro is adequate plus use DS4-Flash for tasks or other small activity you\u2019d use Haiku or Sonnet for. Go sign up with $10 prepaid. OpenCode Go - go sign up with $5 for a month and use Qwen-3.7-Max for design/plan/architecture or difficult troubleshooting. Feels closer to Opus 3.6 or 3.7 than DeepSeek, closest I\u2019ve f", "context": "leads a team of 3 developers, 1 designer, 1 non-technical user", "use_case": "local models on GPU", "switched_from": "", "story": "Kimi K2.7-Code: open-source coding model with better token efficiency"}]}, {"text": " The most common caveat is that some former users switched to lemonade", "cites": [{"id": 779753282, "item_id": 48734580, "url": "https://news.ycombinator.com/item?id=48734580", "by": "data-ottawa", "date": "2026-06-30", "stance": "neutral", "quote": "previously I used llama-swap", "excerpt": "I have the Framework Desktop with 395+ 128gb RAM Today I am pretty happy with it. LLMs are finally good enough (fast enough with MTP+MoE, but also just much better in capability) that I can fit local ones into real tasks, and I've used image generation with invokeAI to do some genuinely useful things like rendering concepts for a renovation. I mostly use lemonade-server and invokeAI for my workloads, previously I use", "context": "", "use_case": "local LLM serving", "switched_from": "", "story": "DGX Spark vs. Mac Studio and Halo"}, {"id": 781817217, "item_id": 48863576, "url": "https://news.ycombinator.com/item?id=48863576", "by": "data-ottawa", "date": "2026-07-10", "stance": "neutral", "quote": "I used llama swap for a while before leaning into lemonade.", "excerpt": "I second this. I used llama swap for a while before leaning into lemonade. The UI has improved a lot, but be careful as the most of the models default to very small 4K context windows by default. They\u2019re doing some nice things with their Halo models, which load an ensemble of different types of model at the same time. With high vram it\u2019s easy to keep them all in memory, so even though the compute is limited the conte", "context": "", "use_case": "running local LLMs", "switched_from": "", "story": "Hands-On with the AMD Ryzen AI Halo"}]}, {"text": ", which one found easier to manage.", "cites": [{"id": 779753282, "item_id": 48734580, "url": "https://news.ycombinator.com/item?id=48734580", "by": "data-ottawa", "date": "2026-06-30", "stance": "neutral", "quote": "previously I used llama-swap", "excerpt": "I have the Framework Desktop with 395+ 128gb RAM Today I am pretty happy with it. LLMs are finally good enough (fast enough with MTP+MoE, but also just much better in capability) that I can fit local ones into real tasks, and I've used image generation with invokeAI to do some genuinely useful things like rendering concepts for a renovation. I mostly use lemonade-server and invokeAI for my workloads, previously I use", "context": "", "use_case": "local LLM serving", "switched_from": "", "story": "DGX Spark vs. Mac Studio and Halo"}]}], "praise": [{"label": "llama.cpp companion", "summary": "Commenters pair it with llama.cpp for their local inference setups.", "quotes": [{"id": 797379844, "item_id": 49836240, "url": "https://news.ycombinator.com/item?id=49836240", "by": "LeBit", "date": "2026-09-24", "stance": "recommend", "quote": "llama.cpp + llama-swap", "excerpt": "I use the following models: - Qwen3.5 9B - Qwen3.6 35B A3B - Qwen3.8 27B - Gemma 4 26B A4B - Gemma 31B - Muse Glimmer 30B I have 48G. The MacMini is used solely for inference. llama.cpp + llama-swap. People are saying local models are crap and serves no purposes. I use them to help me spell check, write emails, write text messages, write JIRA tickets, write PR comments, etc. I also use them in coding agents to comple", "context": "Mac Mini with 48G", "use_case": "model swapping", "switched_from": "", "story": "Best LLM for every budget, updated daily"}, {"id": 787964705, "item_id": 49247794, "url": "https://news.ycombinator.com/item?id=49247794", "by": "computershit", "date": "2026-08-10", "stance": "recommend", "quote": "I use llama.cpp w/ llama-swap", "excerpt": "I use llama.cpp w/ llama-swap https://github.com/ggml-org/llama.cpp https://github.com/mostlygeek/llama-swap", "context": "", "use_case": "swap local models", "switched_from": "", "story": "Meta Muse Glimmer \u2013 open weights 30B local coding model"}, {"id": 776162352, "item_id": 48510147, "url": "https://news.ycombinator.com/item?id=48510147", "by": "monksy", "date": "2026-06-12", "stance": "recommend", "quote": "I just switched from Llama.cpp to Llama swap with the help of codex. It was great.", "excerpt": "I just switched from Llama.cpp to Llama swap with the help of codex. It was great. I need to try the DSv4 stuff sometime.", "context": "", "use_case": "local model serving", "switched_from": "llama.cpp", "story": "Kimi K2.7-Code: open-source coding model with better token efficiency"}]}, {"label": "Dynamic model swapping", "summary": "It swaps models dynamically and exposes multiple models on one endpoint.", "quotes": [{"id": 776914816, "item_id": 48557176, "url": "https://news.ycombinator.com/item?id=48557176", "by": "jborak", "date": "2026-06-16", "stance": "recommend", "quote": "I use llama-swap to dynamically swap models.", "excerpt": "I already had 2x 5070's that I had purchased a year or more ago, so getting an additional two to fill up the PCIe slots on the motherboard seemed reasonable. I did some math/shopping as well. To get 48GB of VRAM you can get 2x 3090s but that is $3k. A single 5090 is $4k but has 32GB, great for running models like Qwen 27B but maybe nothing else depending on your model settings. Already having 2x 5070, where each card", "context": "", "use_case": "swapping local models", "switched_from": "", "story": "Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?"}, {"id": 776719777, "item_id": 48544986, "url": "https://news.ycombinator.com/item?id=48544986", "by": "porkloin", "date": "2026-06-15", "stance": "recommend", "quote": "llama-swap (running as a podman quadlet exposed via tailscale svc) allows running multiple models on a single endpoint.", "excerpt": "I have good results with this setup: Hardware: - GPU: AMD 7900xtx, 24gb vram - CPU: AMD 5950x, AM4 - RAM: 64gb DDR4 3600 Software: - OS: Bazzite (atomic fedora - this machine is running Steam \"big picture\" mode on my TV when not in use for LLM tasks) - Virtualization: Podman Quadlets, which allows me to run container images as managed systemd units - Network: tailscale - Inference: llama.cpp vulkan (better performanc", "context": "", "use_case": "multi-model endpoint", "switched_from": "", "story": "Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?"}]}, {"label": "Ollama-like ease", "summary": "Commenters describe it as emulating the Ollama experience and as fairly easy to use.", "quotes": [{"id": 776749922, "item_id": 48546870, "url": "https://news.ycombinator.com/item?id=48546870", "by": "jboss10", "date": "2026-06-15", "stance": "recommend", "quote": "I use llama-swap to emulate the Ollama experience.", "excerpt": "llama.cpp It's faster and more open source. Ollama has some mixed history. I use llama-swap to emulate the Ollama experience.", "context": "", "use_case": "model swapping", "switched_from": "", "story": "Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?"}, {"id": 776056539, "item_id": 48503533, "url": "https://news.ycombinator.com/item?id=48503533", "by": "trollbridge", "date": "2026-06-12", "stance": "recommend", "quote": "Use ollama or llama-swap which is fairly easy to use.", "excerpt": "I am extremely happy with ohmypi, but you could use OpenCode or just keep using Claude Code! DeepSeek-V4-Pro is adequate plus use DS4-Flash for tasks or other small activity you\u2019d use Haiku or Sonnet for. Go sign up with $10 prepaid. OpenCode Go - go sign up with $5 for a month and use Qwen-3.7-Max for design/plan/architecture or difficult troubleshooting. Feels closer to Opus 3.6 or 3.7 than DeepSeek, closest I\u2019ve f", "context": "leads a team of 3 developers, 1 designer, 1 non-technical user", "use_case": "local models on GPU", "switched_from": "", "story": "Kimi K2.7-Code: open-source coding model with better token efficiency"}]}], "caveats": [{"label": "Lemonade alternative", "summary": "Some former users moved to lemonade, which one found easier to manage.", "quotes": [{"id": 779753282, "item_id": 48734580, "url": "https://news.ycombinator.com/item?id=48734580", "by": "data-ottawa", "date": "2026-06-30", "stance": "neutral", "quote": "previously I used llama-swap", "excerpt": "I have the Framework Desktop with 395+ 128gb RAM Today I am pretty happy with it. LLMs are finally good enough (fast enough with MTP+MoE, but also just much better in capability) that I can fit local ones into real tasks, and I've used image generation with invokeAI to do some genuinely useful things like rendering concepts for a renovation. I mostly use lemonade-server and invokeAI for my workloads, previously I use", "context": "", "use_case": "local LLM serving", "switched_from": "", "story": "DGX Spark vs. Mac Studio and Halo"}, {"id": 781817217, "item_id": 48863576, "url": "https://news.ycombinator.com/item?id=48863576", "by": "data-ottawa", "date": "2026-07-10", "stance": "neutral", "quote": "I used llama swap for a while before leaning into lemonade.", "excerpt": "I second this. I used llama swap for a while before leaning into lemonade. The UI has improved a lot, but be careful as the most of the models default to very small 4K context windows by default. They\u2019re doing some nice things with their Halo models, which load an ensemble of different types of model at the same time. With high vram it\u2019s easy to keep them all in memory, so even though the compute is limited the conte", "context": "", "use_case": "running local LLMs", "switched_from": "", "story": "Hands-On with the AMD Ryzen AI Halo"}]}, {"label": "Manual model setup", "summary": "One user built custom scripts to create hardlinks so llama-swap could load GGUF files.", "quotes": [{"id": 764630290, "item_id": 47789393, "url": "https://news.ycombinator.com/item?id=47789393", "by": "0xbadcafebee", "date": "2026-04-15", "stance": "neutral", "quote": "populate a separate cache directory of hardlinks so llama-swap can load the gguf's into llama.cpp", "excerpt": "No mention of the fact that Ollama is about 1000x easier to use. Llama.cpp is a great project, but it's also one of the least user friendly pieces of software I've used. I don't think anyone in the project cares about normal users. I started with Ollama, and it was great. But I moved to llama.cpp to have more up-to-date fixes. I still use Ollama to pull and list my models because it's so easy. I then built my own set", "context": "", "use_case": "model swapping for llama.cpp", "switched_from": "", "story": "Stop Using Ollama"}]}], "who": [{"by": "LeBit", "context": "Mac Mini with 48G", "url": "https://news.ycombinator.com/item?id=49836240", "date": "2026-09-24"}, {"by": "trollbridge", "context": "leads a team of 3 developers, 1 designer, 1 non-technical user", "url": "https://news.ycombinator.com/item?id=48503533", "date": "2026-06-12"}], "tags": ["for llama.cpp users", "for local LLM hosting", "self-hostable", "for multi-GPU rigs"], "use_cases": ["model swapping", "swap local models", "running local llms", "local llm serving", "swapping local models", "multi-model endpoint"], "compared": [{"slug": "llama-cpp", "name": "llama.cpp", "co": 6, "switched_from": 1, "switched_to": 0}, {"slug": "lemonade", "name": "Lemonade", "co": 2, "switched_from": 0, "switched_to": 2}, {"slug": "ollama", "name": "Ollama", "co": 3, "switched_from": 0, "switched_to": 0}, {"slug": "qwen-3-6", "name": "Qwen 3.6", "co": 3, "switched_from": 0, "switched_to": 0}, {"slug": "rocm", "name": "ROCm", "co": 2, "switched_from": 0, "switched_to": 0}], "makers": [], "threads": [{"url": "https://news.ycombinator.com/item?id=48542100", "title": "Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?", "mentions": 3}, {"url": "https://news.ycombinator.com/item?id=48502347", "title": "Kimi K2.7-Code: open-source coding model with better token efficiency", "mentions": 2}, {"url": "https://news.ycombinator.com/item?id=49830866", "title": "Best LLM for every budget, updated daily", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=49241679", "title": "Meta Muse Glimmer \u2013 open weights 30B local coding model", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=48861833", "title": "Hands-On with the AMD Ryzen AI Halo", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=48732679", "title": "DGX Spark vs. Mac Studio and Halo", "mentions": 1}], "sentiment_by_month": {"2025-11": {}, "2025-12": {}, "2026-01": {}, "2026-02": {}, "2026-03": {}, "2026-04": {"neutral": 1}, "2026-05": {}, "2026-06": {"recommend": 5, "neutral": 1}, "2026-07": {"neutral": 1}, "2026-08": {"recommend": 1}, "2026-09": {"recommend": 1}, "2026-10": {}}, "quotes": [{"id": 797379844, "item_id": 49836240, "url": "https://news.ycombinator.com/item?id=49836240", "by": "LeBit", "date": "2026-09-24", "stance": "recommend", "quote": "llama.cpp + llama-swap", "excerpt": "I use the following models: - Qwen3.5 9B - Qwen3.6 35B A3B - Qwen3.8 27B - Gemma 4 26B A4B - Gemma 31B - Muse Glimmer 30B I have 48G. The MacMini is used solely for inference. llama.cpp + llama-swap. People are saying local models are crap and serves no purposes. I use them to help me spell check, write emails, write text messages, write JIRA tickets, write PR comments, etc. I also use them in coding agents to comple", "context": "Mac Mini with 48G", "use_case": "model swapping", "switched_from": "", "story": "Best LLM for every budget, updated daily"}, {"id": 787964705, "item_id": 49247794, "url": "https://news.ycombinator.com/item?id=49247794", "by": "computershit", "date": "2026-08-10", "stance": "recommend", "quote": "I use llama.cpp w/ llama-swap", "excerpt": "I use llama.cpp w/ llama-swap https://github.com/ggml-org/llama.cpp https://github.com/mostlygeek/llama-swap", "context": "", "use_case": "swap local models", "switched_from": "", "story": "Meta Muse Glimmer \u2013 open weights 30B local coding model"}, {"id": 781817217, "item_id": 48863576, "url": "https://news.ycombinator.com/item?id=48863576", "by": "data-ottawa", "date": "2026-07-10", "stance": "neutral", "quote": "I used llama swap for a while before leaning into lemonade.", "excerpt": "I second this. I used llama swap for a while before leaning into lemonade. The UI has improved a lot, but be careful as the most of the models default to very small 4K context windows by default. They\u2019re doing some nice things with their Halo models, which load an ensemble of different types of model at the same time. With high vram it\u2019s easy to keep them all in memory, so even though the compute is limited the conte", "context": "", "use_case": "running local LLMs", "switched_from": "", "story": "Hands-On with the AMD Ryzen AI Halo"}, {"id": 779753282, "item_id": 48734580, "url": "https://news.ycombinator.com/item?id=48734580", "by": "data-ottawa", "date": "2026-06-30", "stance": "neutral", "quote": "previously I used llama-swap", "excerpt": "I have the Framework Desktop with 395+ 128gb RAM Today I am pretty happy with it. LLMs are finally good enough (fast enough with MTP+MoE, but also just much better in capability) that I can fit local ones into real tasks, and I've used image generation with invokeAI to do some genuinely useful things like rendering concepts for a renovation. I mostly use lemonade-server and invokeAI for my workloads, previously I use", "context": "", "use_case": "local LLM serving", "switched_from": "", "story": "DGX Spark vs. Mac Studio and Halo"}, {"id": 776914816, "item_id": 48557176, "url": "https://news.ycombinator.com/item?id=48557176", "by": "jborak", "date": "2026-06-16", "stance": "recommend", "quote": "I use llama-swap to dynamically swap models.", "excerpt": "I already had 2x 5070's that I had purchased a year or more ago, so getting an additional two to fill up the PCIe slots on the motherboard seemed reasonable. I did some math/shopping as well. To get 48GB of VRAM you can get 2x 3090s but that is $3k. A single 5090 is $4k but has 32GB, great for running models like Qwen 27B but maybe nothing else depending on your model settings. Already having 2x 5070, where each card", "context": "", "use_case": "swapping local models", "switched_from": "", "story": "Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?"}, {"id": 776749922, "item_id": 48546870, "url": "https://news.ycombinator.com/item?id=48546870", "by": "jboss10", "date": "2026-06-15", "stance": "recommend", "quote": "I use llama-swap to emulate the Ollama experience.", "excerpt": "llama.cpp It's faster and more open source. Ollama has some mixed history. I use llama-swap to emulate the Ollama experience.", "context": "", "use_case": "model swapping", "switched_from": "", "story": "Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?"}, {"id": 776719777, "item_id": 48544986, "url": "https://news.ycombinator.com/item?id=48544986", "by": "porkloin", "date": "2026-06-15", "stance": "recommend", "quote": "llama-swap (running as a podman quadlet exposed via tailscale svc) allows running multiple models on a single endpoint.", "excerpt": "I have good results with this setup: Hardware: - GPU: AMD 7900xtx, 24gb vram - CPU: AMD 5950x, AM4 - RAM: 64gb DDR4 3600 Software: - OS: Bazzite (atomic fedora - this machine is running Steam \"big picture\" mode on my TV when not in use for LLM tasks) - Virtualization: Podman Quadlets, which allows me to run container images as managed systemd units - Network: tailscale - Inference: llama.cpp vulkan (better performanc", "context": "", "use_case": "multi-model endpoint", "switched_from": "", "story": "Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?"}, {"id": 776162352, "item_id": 48510147, "url": "https://news.ycombinator.com/item?id=48510147", "by": "monksy", "date": "2026-06-12", "stance": "recommend", "quote": "I just switched from Llama.cpp to Llama swap with the help of codex. It was great.", "excerpt": "I just switched from Llama.cpp to Llama swap with the help of codex. It was great. I need to try the DSv4 stuff sometime.", "context": "", "use_case": "local model serving", "switched_from": "llama.cpp", "story": "Kimi K2.7-Code: open-source coding model with better token efficiency"}, {"id": 776056539, "item_id": 48503533, "url": "https://news.ycombinator.com/item?id=48503533", "by": "trollbridge", "date": "2026-06-12", "stance": "recommend", "quote": "Use ollama or llama-swap which is fairly easy to use.", "excerpt": "I am extremely happy with ohmypi, but you could use OpenCode or just keep using Claude Code! DeepSeek-V4-Pro is adequate plus use DS4-Flash for tasks or other small activity you\u2019d use Haiku or Sonnet for. Go sign up with $10 prepaid. OpenCode Go - go sign up with $5 for a month and use Qwen-3.7-Max for design/plan/architecture or difficult troubleshooting. Feels closer to Opus 3.6 or 3.7 than DeepSeek, closest I\u2019ve f", "context": "leads a team of 3 developers, 1 designer, 1 non-technical user", "use_case": "local models on GPU", "switched_from": "", "story": "Kimi K2.7-Code: open-source coding model with better token efficiency"}, {"id": 764630290, "item_id": 47789393, "url": "https://news.ycombinator.com/item?id=47789393", "by": "0xbadcafebee", "date": "2026-04-15", "stance": "neutral", "quote": "populate a separate cache directory of hardlinks so llama-swap can load the gguf's into llama.cpp", "excerpt": "No mention of the fact that Ollama is about 1000x easier to use. Llama.cpp is a great project, but it's also one of the least user friendly pieces of software I've used. I don't think anyone in the project cares about normal users. I started with Ollama, and it was great. But I moved to llama.cpp to have more up-to-date fixes. I still use Ollama to pull and list my models because it's so easy. I then built my own set", "context": "", "use_case": "model swapping for llama.cpp", "switched_from": "", "story": "Stop Using Ollama"}]}