{"slug": "litellm", "name": "LiteLLM", "category": "ai", "category_name": "AI models and assistants", "description": "Open-source Python SDK and proxy gateway exposing many LLM providers through an OpenAI-style API.", "recommenders": 5, "commenters": 20, "warners": 7, "score": 0.012, "rising": -1.64, "first": "2026-01-07", "latest": "2026-10-22", "why": "HN commenters use it as a proxy that puts multiple LLM providers behind one API base for coding assistants and other tools, and for monitoring usage.", "why_is_quote": false, "has_verdict": true, "sentiment": {"recommend": 5, "neutral": 7, "compare": 1, "warn": 7}, "spark": [0, 0, 2, 0, 0, 0, 0, 1, 2, 0, 0, 0], "listed": true, "loved": ["Unified LLM proxy", "Usage monitoring"], "caveat": "Supply-chain security", "short_quote": "I use litellm (slightly modified to allow cloud code telemetry pass through) and langfuse.", "icon": true, "domain": null, "github_repo": "github.com/berriai/litellm", "official_url": "https://www.litellm.ai", "pricing": "open source", "aliases": ["LiteLLM"], "verdict": [{"text": "HN commenters use it as a proxy that puts multiple LLM providers behind one API base", "cites": [{"id": 775893553, "item_id": 48493347, "url": "https://news.ycombinator.com/item?id=48493347", "by": "msradam", "date": "2026-06-11", "stance": "neutral", "quote": "I use a LiteLLM proxy with Claude Code", "excerpt": "The inertia of switching. At least for me, being able to maintain the same UX and configuration for agentic coding is a benefit. I have access to internal models at work where I use a LiteLLM proxy with Claude Code, so no Anthropic models, but I can maintain the same .claude between personal and work machines. Though I'm definitely open to a non-Anthropic harness if it's a clear win.", "context": "", "use_case": "proxy to internal models", "switched_from": "", "story": "Running Claude Code Offline on an M3 Pro with Qwen3.6"}, {"id": 749017344, "item_id": 46813584, "url": "https://news.ycombinator.com/item?id=46813584", "by": "Roark66", "date": "2026-01-29", "stance": "recommend", "quote": "I use litellm (slightly modified to allow cloud code telemetry pass through) and langfuse.", "excerpt": "I use litellm (slightly modified to allow cloud code telemetry pass through) and langfuse. There is no need for MitM, you can set Api base address to your own proxy in all the coding assistants (at least all I know - Claude Code, opencode, gemini, vc plugin). The changes I made allow use of the models endpoint in litellm at the same base url as telemetry and passing through Claude Max auth. This is not about using yo", "context": "", "use_case": "proxy to record LLM tool traffic", "switched_from": "", "story": "A MitM proxy to see what your LLM tools are sending"}, {"id": 744891170, "item_id": 46555698, "url": "https://news.ycombinator.com/item?id=46555698", "by": "atmosx", "date": "2026-01-09", "stance": "neutral", "quote": "through vim-ai and liteLLM", "excerpt": "I am using the mistral API for personal projects through vim-ai and liteLLM. I must say that so far I don\u2019t miss ChatGPT or Gemini (which I use at work).", "context": "", "use_case": "LLM API proxy", "switched_from": "", "story": "Ask HN: How would you decouple from the US?"}, {"id": 780137184, "item_id": 48758574, "url": "https://news.ycombinator.com/item?id=48758574", "by": "lbreakjai", "date": "2026-07-02", "stance": "recommend", "quote": "We just cancelled everyone's plans and rolled liteLLM out internally.", "excerpt": "We just cancelled everyone's plans and rolled liteLLM out internally. We kept it for the insanely cheap tokens, but now that they've switched to the new pricing, they're just like openrouter, just with far fewer models.", "context": "company admin", "use_case": "internal LLM gateway", "switched_from": "GitHub Copilot", "story": "Kimi K2.7 Code is generally available in GitHub Copilot"}]}, {"text": " for coding assistants and other tools, and for monitoring usage.", "cites": [{"id": 775893553, "item_id": 48493347, "url": "https://news.ycombinator.com/item?id=48493347", "by": "msradam", "date": "2026-06-11", "stance": "neutral", "quote": "I use a LiteLLM proxy with Claude Code", "excerpt": "The inertia of switching. At least for me, being able to maintain the same UX and configuration for agentic coding is a benefit. I have access to internal models at work where I use a LiteLLM proxy with Claude Code, so no Anthropic models, but I can maintain the same .claude between personal and work machines. Though I'm definitely open to a non-Anthropic harness if it's a clear win.", "context": "", "use_case": "proxy to internal models", "switched_from": "", "story": "Running Claude Code Offline on an M3 Pro with Qwen3.6"}, {"id": 747341073, "item_id": 46708817, "url": "https://news.ycombinator.com/item?id=46708817", "by": "photios", "date": "2026-01-21", "stance": "warn", "quote": "I got Claude Code running with Copilot APIs via the LiteLLM proxy, but it was a pain in the butt.", "excerpt": "I feel your pain. I used to work for a bank and the security team only approved Copilot use. OpenCode can use Copilot natively: https://opencode.ai/docs/providers/#github-copilot I got Claude Code running with Copilot APIs via the LiteLLM proxy, but it was a pain in the butt. Just use OpenCode.", "context": "used to work for a bank", "use_case": "proxying Copilot APIs", "switched_from": "", "story": "The Agentic AI Handbook: Production-Ready Patterns"}, {"id": 800391058, "item_id": 50024441, "url": "https://news.ycombinator.com/item?id=50024441", "by": "K0IN", "date": "2026-10-09", "stance": "neutral", "quote": "litellm as ai bridge to monitor usage", "excerpt": "TeamSpeak, git backup(I use gittea) to backup public repos, backups in general, my website/blog, game servers, ssh(obv) for local app port forward, litellm as ai bridge to monitor usage, some of my own apps: (notification broker to send messages to my devices using a simple http push), everything behind traefik (with let's e encrypt)", "context": "", "use_case": "AI gateway usage monitoring", "switched_from": "", "story": "Ask HN: What do you run on a $5 VPS that's worth keeping online 24/7?"}, {"id": 749017344, "item_id": 46813584, "url": "https://news.ycombinator.com/item?id=46813584", "by": "Roark66", "date": "2026-01-29", "stance": "recommend", "quote": "I use litellm (slightly modified to allow cloud code telemetry pass through) and langfuse.", "excerpt": "I use litellm (slightly modified to allow cloud code telemetry pass through) and langfuse. There is no need for MitM, you can set Api base address to your own proxy in all the coding assistants (at least all I know - Claude Code, opencode, gemini, vc plugin). The changes I made allow use of the models endpoint in litellm at the same base url as telemetry and passing through Claude Max auth. This is not about using yo", "context": "", "use_case": "proxy to record LLM tool traffic", "switched_from": "", "story": "A MitM proxy to see what your LLM tools are sending"}]}, {"text": " The most common complaint is supply-chain security after a credential-stealer incident in its builds,", "cites": [{"id": 802920688, "item_id": 50182543, "url": "https://news.ycombinator.com/item?id=50182543", "by": "petesergeant", "date": "2026-10-21", "stance": "warn", "quote": "Announcing that there's a private enterprise repo you should be paying for if you don't want a credential stealer in your build is quite the optic", "excerpt": "> There currently exists a private LiteLLM registry for enterprise customers, in which we are distributing an embargoed release that remediates a vulnerability whose disclosure date is scheduled for October 27. https://github.com/BerriAI/litellm/issues/47826 oh boy. Announcing that there's a private enterprise repo you should be paying for if you don't want a credential stealer in your build is quite the optic, and t", "context": "", "use_case": "LLM gateway", "switched_from": "", "story": "Compromised LiteLLM 1.107.0 on PyPI Installs a Credential Stealer and Backdoor"}, {"id": 803041968, "item_id": 50190123, "url": "https://news.ycombinator.com/item?id=50190123", "by": "joshheitzman", "date": "2026-10-22", "stance": "warn", "quote": "I'd like just flatout block litellm so I don't have to worry about picking it up as transitive dependency.", "excerpt": "Seems like pip doesn't have a way to exclude packages although it was requested at least once: https://github.com/pypa/pip/issues/10797 Just checked if litellm crept in with any packages I installed recently. Fortunately it didn't, but I'm working on an OSS AI agent harness so its certainly a concern and I'd like just flatout block litellm so I don't have to worry about picking it up as transitive dependency. Guess I", "context": "working on an OSS AI agent harness", "use_case": "", "switched_from": "", "story": "Compromised LiteLLM 1.107.0 on PyPI Installs a Credential Stealer and Backdoor"}, {"id": 802936832, "item_id": 50183552, "url": "https://news.ycombinator.com/item?id=50183552", "by": "1-6", "date": "2026-10-21", "stance": "warn", "quote": "This is the 2nd time. I'm glad I moved away from this platform.", "excerpt": "This is the 2nd time. I'm glad I moved away from this platform.", "context": "", "use_case": "LLM gateway", "switched_from": "", "story": "Compromised LiteLLM 1.107.0 on PyPI Installs a Credential Stealer and Backdoor"}, {"id": 802994994, "item_id": 50187187, "url": "https://news.ycombinator.com/item?id=50187187", "by": "isityettime", "date": "2026-10-21", "stance": "neutral", "quote": "You sandbox LiteLLM separately from your agents, and you give it access to much less.", "excerpt": "> All the sandbox-exec and all the Lulu rules are not going to protect against something like this. No, you can block this with sandboxing tools. For my LLM agents, I use a tool called Nono, whose macOS implementation is built on the same foundation as sandbox-exec. One of the key features it supports (and it's not the only tool that does this) is proxying and the injection of credentials via the proxy. For example, ", "context": "", "use_case": "LLM proxy", "switched_from": "", "story": "Compromised LiteLLM 1.107.0 on PyPI Installs a Credential Stealer and Backdoor"}]}, {"text": " and others report flakiness and outages.", "cites": [{"id": 760037649, "item_id": 47502353, "url": "https://news.ycombinator.com/item?id=47502353", "by": "chillfox", "date": "2026-03-24", "stance": "warn", "quote": "LiteLLM was incredible flaky and kept causing outages", "excerpt": "Now I feel lucky that I switched to just using OpenRouter a year ago because LiteLLM was incredible flaky and kept causing outages.", "context": "", "use_case": "LLM API routing", "switched_from": "", "story": "LiteLLM Python package compromised by supply-chain attack"}, {"id": 794763136, "item_id": 49672696, "url": "https://news.ycombinator.com/item?id=49672696", "by": "santiagobasulto", "date": "2026-09-12", "stance": "warn", "quote": "LiteLLM and LangChain are AWFUL pieces of software and should be avoided at ALL costs.", "excerpt": "Cool project, I had to implement the same for my experimental AI Agent: https://github.com/quantized-ai/luca-py LiteLLM and LangChain are AWFUL pieces of software and should be avoided at ALL costs. Btw, you should update httpx to httpx2, and I think it's not much effort to remove openai's SDK compatibility. I'd love to have a provider-agnostic LLM router (almost) dependency free (aside from httpx2).", "context": "", "use_case": "LLM routing", "switched_from": "", "story": "Litelm: LiteLLM Without the Bloat"}]}], "praise": [{"label": "Unified LLM proxy", "summary": "Lets tools like Claude Code and vim-ai reach different model providers through one proxy endpoint.", "quotes": [{"id": 775893553, "item_id": 48493347, "url": "https://news.ycombinator.com/item?id=48493347", "by": "msradam", "date": "2026-06-11", "stance": "neutral", "quote": "I use a LiteLLM proxy with Claude Code", "excerpt": "The inertia of switching. At least for me, being able to maintain the same UX and configuration for agentic coding is a benefit. I have access to internal models at work where I use a LiteLLM proxy with Claude Code, so no Anthropic models, but I can maintain the same .claude between personal and work machines. Though I'm definitely open to a non-Anthropic harness if it's a clear win.", "context": "", "use_case": "proxy to internal models", "switched_from": "", "story": "Running Claude Code Offline on an M3 Pro with Qwen3.6"}, {"id": 749017344, "item_id": 46813584, "url": "https://news.ycombinator.com/item?id=46813584", "by": "Roark66", "date": "2026-01-29", "stance": "recommend", "quote": "I use litellm (slightly modified to allow cloud code telemetry pass through) and langfuse.", "excerpt": "I use litellm (slightly modified to allow cloud code telemetry pass through) and langfuse. There is no need for MitM, you can set Api base address to your own proxy in all the coding assistants (at least all I know - Claude Code, opencode, gemini, vc plugin). The changes I made allow use of the models endpoint in litellm at the same base url as telemetry and passing through Claude Max auth. This is not about using yo", "context": "", "use_case": "proxy to record LLM tool traffic", "switched_from": "", "story": "A MitM proxy to see what your LLM tools are sending"}, {"id": 744891170, "item_id": 46555698, "url": "https://news.ycombinator.com/item?id=46555698", "by": "atmosx", "date": "2026-01-09", "stance": "neutral", "quote": "through vim-ai and liteLLM", "excerpt": "I am using the mistral API for personal projects through vim-ai and liteLLM. I must say that so far I don\u2019t miss ChatGPT or Gemini (which I use at work).", "context": "", "use_case": "LLM API proxy", "switched_from": "", "story": "Ask HN: How would you decouple from the US?"}]}, {"label": "Usage monitoring", "summary": "Used as a bridge to monitor AI usage and record traffic for observability or auditing.", "quotes": [{"id": 800391058, "item_id": 50024441, "url": "https://news.ycombinator.com/item?id=50024441", "by": "K0IN", "date": "2026-10-09", "stance": "neutral", "quote": "litellm as ai bridge to monitor usage", "excerpt": "TeamSpeak, git backup(I use gittea) to backup public repos, backups in general, my website/blog, game servers, ssh(obv) for local app port forward, litellm as ai bridge to monitor usage, some of my own apps: (notification broker to send messages to my devices using a simple http push), everything behind traefik (with let's e encrypt)", "context": "", "use_case": "AI gateway usage monitoring", "switched_from": "", "story": "Ask HN: What do you run on a $5 VPS that's worth keeping online 24/7?"}, {"id": 749017344, "item_id": 46813584, "url": "https://news.ycombinator.com/item?id=46813584", "by": "Roark66", "date": "2026-01-29", "stance": "recommend", "quote": "I use litellm (slightly modified to allow cloud code telemetry pass through) and langfuse.", "excerpt": "I use litellm (slightly modified to allow cloud code telemetry pass through) and langfuse. There is no need for MitM, you can set Api base address to your own proxy in all the coding assistants (at least all I know - Claude Code, opencode, gemini, vc plugin). The changes I made allow use of the models endpoint in litellm at the same base url as telemetry and passing through Claude Max auth. This is not about using yo", "context": "", "use_case": "proxy to record LLM tool traffic", "switched_from": "", "story": "A MitM proxy to see what your LLM tools are sending"}, {"id": 780470609, "item_id": 48779413, "url": "https://news.ycombinator.com/item?id=48779413", "by": "westurner", "date": "2026-07-03", "stance": "recommend", "quote": "Aegis and LiteLLM, for example, are pre-execution firewalls that add a cryptographic audit trail.", "excerpt": "Remote debugging and post-mortem debugging support might be useful. There are many AI auditability proxies; awesome-auditable-ai: \"A curated list of papers, tools, datasets, benchmarks, and standards for building, evaluating, and auditing reliable AI agents\" https://github.com/yzhao062/awesome-auditable-ai Aegis and LiteLLM, for example, are pre-execution firewalls that add a cryptographic audit trail. https://github", "context": "", "use_case": "AI agent audit trail", "switched_from": "", "story": "Show HN: Mcpsnoop \u2013 Wireshark for MCP (transparent proxy and live TUI)"}]}, {"label": "Internal rollout", "summary": "One company admin cancelled individual plans and rolled it out internally for staff.", "quotes": [{"id": 780137184, "item_id": 48758574, "url": "https://news.ycombinator.com/item?id=48758574", "by": "lbreakjai", "date": "2026-07-02", "stance": "recommend", "quote": "We just cancelled everyone's plans and rolled liteLLM out internally.", "excerpt": "We just cancelled everyone's plans and rolled liteLLM out internally. We kept it for the insanely cheap tokens, but now that they've switched to the new pricing, they're just like openrouter, just with far fewer models.", "context": "company admin", "use_case": "internal LLM gateway", "switched_from": "GitHub Copilot", "story": "Kimi K2.7 Code is generally available in GitHub Copilot"}]}, {"label": "Building block", "summary": "Suggested as a base layer to build on, and one commenter was glad to have used it.", "quotes": [{"id": 744513569, "item_id": 46532098, "url": "https://news.ycombinator.com/item?id=46532098", "by": "billylo", "date": "2026-01-07", "stance": "recommend", "quote": "I would suggest to code it on top of ollama or LiteLLM", "excerpt": "Each distro is doing their own thing. If you are targeting Linux mainly, I would suggest to code it on top of ollama or LiteLLM", "context": "", "use_case": "LLM completion backend for Linux apps", "switched_from": "", "story": "Ask HN: What's a standard way for apps to request text completion as a service?"}, {"id": 776286656, "item_id": 48517916, "url": "https://news.ycombinator.com/item?id=48517916", "by": "SamDc73", "date": "2026-06-13", "stance": "recommend", "quote": "I'm glad I used litellm in my last project", "excerpt": "I'm glad I used litellm in my last project https://github.com/BerriAI/litellm/", "context": "", "use_case": "LLM API proxy", "switched_from": "", "story": "AI OSS tool repo goes archived over night after raising $7.3M Seed"}]}], "caveats": [{"label": "Supply-chain security", "summary": "A credential-stealer incident in builds led some commenters to block it as a dependency or leave it.", "quotes": [{"id": 802920688, "item_id": 50182543, "url": "https://news.ycombinator.com/item?id=50182543", "by": "petesergeant", "date": "2026-10-21", "stance": "warn", "quote": "Announcing that there's a private enterprise repo you should be paying for if you don't want a credential stealer in your build is quite the optic", "excerpt": "> There currently exists a private LiteLLM registry for enterprise customers, in which we are distributing an embargoed release that remediates a vulnerability whose disclosure date is scheduled for October 27. https://github.com/BerriAI/litellm/issues/47826 oh boy. Announcing that there's a private enterprise repo you should be paying for if you don't want a credential stealer in your build is quite the optic, and t", "context": "", "use_case": "LLM gateway", "switched_from": "", "story": "Compromised LiteLLM 1.107.0 on PyPI Installs a Credential Stealer and Backdoor"}, {"id": 803041968, "item_id": 50190123, "url": "https://news.ycombinator.com/item?id=50190123", "by": "joshheitzman", "date": "2026-10-22", "stance": "warn", "quote": "I'd like just flatout block litellm so I don't have to worry about picking it up as transitive dependency.", "excerpt": "Seems like pip doesn't have a way to exclude packages although it was requested at least once: https://github.com/pypa/pip/issues/10797 Just checked if litellm crept in with any packages I installed recently. Fortunately it didn't, but I'm working on an OSS AI agent harness so its certainly a concern and I'd like just flatout block litellm so I don't have to worry about picking it up as transitive dependency. Guess I", "context": "working on an OSS AI agent harness", "use_case": "", "switched_from": "", "story": "Compromised LiteLLM 1.107.0 on PyPI Installs a Credential Stealer and Backdoor"}, {"id": 802936832, "item_id": 50183552, "url": "https://news.ycombinator.com/item?id=50183552", "by": "1-6", "date": "2026-10-21", "stance": "warn", "quote": "This is the 2nd time. I'm glad I moved away from this platform.", "excerpt": "This is the 2nd time. I'm glad I moved away from this platform.", "context": "", "use_case": "LLM gateway", "switched_from": "", "story": "Compromised LiteLLM 1.107.0 on PyPI Installs a Credential Stealer and Backdoor"}]}, {"label": "Enterprise-first fixes", "summary": "A fix for a further vulnerability was distributed first through a private registry for enterprise customers.", "quotes": [{"id": 802920688, "item_id": 50182543, "url": "https://news.ycombinator.com/item?id=50182543", "by": "petesergeant", "date": "2026-10-21", "stance": "warn", "quote": "Announcing that there's a private enterprise repo you should be paying for if you don't want a credential stealer in your build is quite the optic", "excerpt": "> There currently exists a private LiteLLM registry for enterprise customers, in which we are distributing an embargoed release that remediates a vulnerability whose disclosure date is scheduled for October 27. https://github.com/BerriAI/litellm/issues/47826 oh boy. Announcing that there's a private enterprise repo you should be paying for if you don't want a credential stealer in your build is quite the optic, and t", "context": "", "use_case": "LLM gateway", "switched_from": "", "story": "Compromised LiteLLM 1.107.0 on PyPI Installs a Credential Stealer and Backdoor"}]}, {"label": "Reliability", "summary": "Some commenters describe it as flaky, outage-prone or poorly built.", "quotes": [{"id": 760037649, "item_id": 47502353, "url": "https://news.ycombinator.com/item?id=47502353", "by": "chillfox", "date": "2026-03-24", "stance": "warn", "quote": "LiteLLM was incredible flaky and kept causing outages", "excerpt": "Now I feel lucky that I switched to just using OpenRouter a year ago because LiteLLM was incredible flaky and kept causing outages.", "context": "", "use_case": "LLM API routing", "switched_from": "", "story": "LiteLLM Python package compromised by supply-chain attack"}, {"id": 794763136, "item_id": 49672696, "url": "https://news.ycombinator.com/item?id=49672696", "by": "santiagobasulto", "date": "2026-09-12", "stance": "warn", "quote": "LiteLLM and LangChain are AWFUL pieces of software and should be avoided at ALL costs.", "excerpt": "Cool project, I had to implement the same for my experimental AI Agent: https://github.com/quantized-ai/luca-py LiteLLM and LangChain are AWFUL pieces of software and should be avoided at ALL costs. Btw, you should update httpx to httpx2, and I think it's not much effort to remove openai's SDK compatibility. I'd love to have a provider-agnostic LLM router (almost) dependency free (aside from httpx2).", "context": "", "use_case": "LLM routing", "switched_from": "", "story": "Litelm: LiteLLM Without the Bloat"}]}, {"label": "Setup friction", "summary": "Getting Claude Code running with Copilot APIs through the proxy was called a pain.", "quotes": [{"id": 747341073, "item_id": 46708817, "url": "https://news.ycombinator.com/item?id=46708817", "by": "photios", "date": "2026-01-21", "stance": "warn", "quote": "I got Claude Code running with Copilot APIs via the LiteLLM proxy, but it was a pain in the butt.", "excerpt": "I feel your pain. I used to work for a bank and the security team only approved Copilot use. OpenCode can use Copilot natively: https://opencode.ai/docs/providers/#github-copilot I got Claude Code running with Copilot APIs via the LiteLLM proxy, but it was a pain in the butt. Just use OpenCode.", "context": "used to work for a bank", "use_case": "proxying Copilot APIs", "switched_from": "", "story": "The Agentic AI Handbook: Production-Ready Patterns"}]}, {"label": "Routing latency", "summary": "One user noticed a significant delay in the routing process.", "quotes": [{"id": 786552483, "item_id": 49159530, "url": "https://news.ycombinator.com/item?id=49159530", "by": "Swapnoneel", "date": "2026-08-03", "stance": "warn", "quote": "I noticed a significant delay in the routing process", "excerpt": "So, I recently started using the Hermes agent and hooked it up with Step 3.5 Flash. It\u2019s a cheap and fast model, so I thought it would be enough for my workflow. However, I soon found that it hallucinates during a lot of tasks, and the output quality isn\u2019t something to write home about. On the other hand, using an expensive or frontier-level model means I\u2019ll burn through cash fast. I went down the rabbit hole to find", "context": "", "use_case": "model routing", "switched_from": "", "story": "Is Hermes Agent unusable without a model router?"}]}, {"label": "Needs sandboxing", "summary": "A commenter advises sandboxing it separately from agents with minimal access.", "quotes": [{"id": 802994994, "item_id": 50187187, "url": "https://news.ycombinator.com/item?id=50187187", "by": "isityettime", "date": "2026-10-21", "stance": "neutral", "quote": "You sandbox LiteLLM separately from your agents, and you give it access to much less.", "excerpt": "> All the sandbox-exec and all the Lulu rules are not going to protect against something like this. No, you can block this with sandboxing tools. For my LLM agents, I use a tool called Nono, whose macOS implementation is built on the same foundation as sandbox-exec. One of the key features it supports (and it's not the only tool that does this) is proxying and the injection of credentials via the proxy. For example, ", "context": "", "use_case": "LLM proxy", "switched_from": "", "story": "Compromised LiteLLM 1.107.0 on PyPI Installs a Credential Stealer and Backdoor"}]}], "who": [{"by": "sainishiva", "context": "AI agent engineer", "url": "https://news.ycombinator.com/item?id=50053994", "date": "2026-10-12"}, {"by": "lbreakjai", "context": "company admin", "url": "https://news.ycombinator.com/item?id=48758574", "date": "2026-07-02"}], "tags": ["for coding agents", "for usage monitoring", "for teams", "self-hostable"], "use_cases": ["llm gateway", "model routing", "llm api proxy", "llm proxy", "ai gateway usage monitoring", "llm routing"], "compared": [{"slug": "openrouter", "name": "OpenRouter", "co": 3, "switched_from": 0, "switched_to": 1}, {"slug": "bifrost", "name": "Bifrost", "co": 2, "switched_from": 0, "switched_to": 1}, {"slug": "gomodel", "name": "GoModel", "co": 3, "switched_from": 0, "switched_to": 0}, {"slug": "github-copilot", "name": "GitHub Copilot", "co": 1, "switched_from": 1, "switched_to": 0}, {"slug": "opencode", "name": "OpenCode", "co": 2, "switched_from": 0, "switched_to": 0}, {"slug": "claude-code", "name": "Claude Code", "co": 2, "switched_from": 0, "switched_to": 0}], "makers": [], "threads": [{"url": "https://news.ycombinator.com/item?id=50181114", "title": "Compromised LiteLLM 1.107.0 on PyPI Installs a Credential Stealer and Backdoor", "mentions": 4}, {"url": "https://news.ycombinator.com/item?id=49922568", "title": "Ask HN: Who wants to be hired? (October 2026)", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=49985548", "title": "Ask HN: What do you run on a $5 VPS that's worth keeping online 24/7?", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=49586552", "title": "Should I reconsider my decision of self-hosting LLM Gateway?", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=49662767", "title": "Litelm: LiteLLM Without the Bloat", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=49471407", "title": "Show HN: We built open OpenRouter that turns usage into a better model", "mentions": 1}], "sentiment_by_month": {"2025-11": {}, "2025-12": {}, "2026-01": {"recommend": 2, "neutral": 1, "warn": 1}, "2026-02": {}, "2026-03": {"warn": 1}, "2026-04": {}, "2026-05": {}, "2026-06": {"neutral": 1, "recommend": 1}, "2026-07": {"recommend": 2, "neutral": 1}, "2026-08": {"warn": 1, "compare": 1}, "2026-09": {"warn": 1, "neutral": 1}, "2026-10": {"neutral": 3, "warn": 3}}, "quotes": [{"id": 803041968, "item_id": 50190123, "url": "https://news.ycombinator.com/item?id=50190123", "by": "joshheitzman", "date": "2026-10-22", "stance": "warn", "quote": "I'd like just flatout block litellm so I don't have to worry about picking it up as transitive dependency.", "excerpt": "Seems like pip doesn't have a way to exclude packages although it was requested at least once: https://github.com/pypa/pip/issues/10797 Just checked if litellm crept in with any packages I installed recently. Fortunately it didn't, but I'm working on an OSS AI agent harness so its certainly a concern and I'd like just flatout block litellm so I don't have to worry about picking it up as transitive dependency. Guess I", "context": "working on an OSS AI agent harness", "use_case": "", "switched_from": "", "story": "Compromised LiteLLM 1.107.0 on PyPI Installs a Credential Stealer and Backdoor"}, {"id": 802994994, "item_id": 50187187, "url": "https://news.ycombinator.com/item?id=50187187", "by": "isityettime", "date": "2026-10-21", "stance": "neutral", "quote": "You sandbox LiteLLM separately from your agents, and you give it access to much less.", "excerpt": "> All the sandbox-exec and all the Lulu rules are not going to protect against something like this. No, you can block this with sandboxing tools. For my LLM agents, I use a tool called Nono, whose macOS implementation is built on the same foundation as sandbox-exec. One of the key features it supports (and it's not the only tool that does this) is proxying and the injection of credentials via the proxy. For example, ", "context": "", "use_case": "LLM proxy", "switched_from": "", "story": "Compromised LiteLLM 1.107.0 on PyPI Installs a Credential Stealer and Backdoor"}, {"id": 802936832, "item_id": 50183552, "url": "https://news.ycombinator.com/item?id=50183552", "by": "1-6", "date": "2026-10-21", "stance": "warn", "quote": "This is the 2nd time. I'm glad I moved away from this platform.", "excerpt": "This is the 2nd time. I'm glad I moved away from this platform.", "context": "", "use_case": "LLM gateway", "switched_from": "", "story": "Compromised LiteLLM 1.107.0 on PyPI Installs a Credential Stealer and Backdoor"}, {"id": 802920688, "item_id": 50182543, "url": "https://news.ycombinator.com/item?id=50182543", "by": "petesergeant", "date": "2026-10-21", "stance": "warn", "quote": "Announcing that there's a private enterprise repo you should be paying for if you don't want a credential stealer in your build is quite the optic", "excerpt": "> There currently exists a private LiteLLM registry for enterprise customers, in which we are distributing an embargoed release that remediates a vulnerability whose disclosure date is scheduled for October 27. https://github.com/BerriAI/litellm/issues/47826 oh boy. Announcing that there's a private enterprise repo you should be paying for if you don't want a credential stealer in your build is quite the optic, and t", "context": "", "use_case": "LLM gateway", "switched_from": "", "story": "Compromised LiteLLM 1.107.0 on PyPI Installs a Credential Stealer and Backdoor"}, {"id": 800863906, "item_id": 50053994, "url": "https://news.ycombinator.com/item?id=50053994", "by": "sainishiva", "date": "2026-10-12", "stance": "neutral", "quote": "LiteLLM model routing", "excerpt": "Location: India (IST) Remote: Yes, remote only. Contract/freelance or full-time. Can overlap US mornings or EU afternoons. Willing to relocate: No Technologies: TypeScript, Python, Node.js, React/Next.js, LLM APIs (Claude, GPT, Gemini), LiteLLM, MCP, LangGraph, RAG, Docker, Playwright, WhatsApp integrations Resume/CV: https://linkedin.com/in/shiva-saini-066157253 | https://github.com/worldofaryavart Email: shivasaini", "context": "AI agent engineer", "use_case": "model routing", "switched_from": "", "story": "Ask HN: Who wants to be hired? (October 2026)"}, {"id": 800391058, "item_id": 50024441, "url": "https://news.ycombinator.com/item?id=50024441", "by": "K0IN", "date": "2026-10-09", "stance": "neutral", "quote": "litellm as ai bridge to monitor usage", "excerpt": "TeamSpeak, git backup(I use gittea) to backup public repos, backups in general, my website/blog, game servers, ssh(obv) for local app port forward, litellm as ai bridge to monitor usage, some of my own apps: (notification broker to send messages to my devices using a simple http push), everything behind traefik (with let's e encrypt)", "context": "", "use_case": "AI gateway usage monitoring", "switched_from": "", "story": "Ask HN: What do you run on a $5 VPS that's worth keeping online 24/7?"}, {"id": 795640865, "item_id": 49727554, "url": "https://news.ycombinator.com/item?id=49727554", "by": "leanroute_ai", "date": "2026-09-16", "stance": "neutral", "quote": "was using LiteLLM for the company i work for. we were doing a poc.", "excerpt": "was using LiteLLM for the company i work for. we were doing a poc. But now we switched to leanroute.dev, we use our own provider keys. Gateway helps with the routing, caching, spend cap and monitoring usage.", "context": "", "use_case": "LLM gateway", "switched_from": "", "story": "Should I reconsider my decision of self-hosting LLM Gateway?"}, {"id": 794763136, "item_id": 49672696, "url": "https://news.ycombinator.com/item?id=49672696", "by": "santiagobasulto", "date": "2026-09-12", "stance": "warn", "quote": "LiteLLM and LangChain are AWFUL pieces of software and should be avoided at ALL costs.", "excerpt": "Cool project, I had to implement the same for my experimental AI Agent: https://github.com/quantized-ai/luca-py LiteLLM and LangChain are AWFUL pieces of software and should be avoided at ALL costs. Btw, you should update httpx to httpx2, and I think it's not much effort to remove openai's SDK compatibility. I'd love to have a provider-agnostic LLM router (almost) dependency free (aside from httpx2).", "context": "", "use_case": "LLM routing", "switched_from": "", "story": "Litelm: LiteLLM Without the Bloat"}, {"id": 791633634, "item_id": 49477102, "url": "https://news.ycombinator.com/item?id=49477102", "by": "mongrelion", "date": "2026-08-28", "stance": "compare", "quote": "to compare it with these other products that seem to be competing directly", "excerpt": "Congrats on the launching of your product. I will be taking it for a spin to compare it with these other products that seem to be competing directly with what you have to offer: - https://github.com/ENTERPILOT/GoModel - https://github.com/maximhq/bifrost - https://github.com/BerriAI/litellm Would you care to share what makes experiential different?", "context": "", "use_case": "LLM gateway", "switched_from": "", "story": "Show HN: We built open OpenRouter that turns usage into a better model"}, {"id": 786552483, "item_id": 49159530, "url": "https://news.ycombinator.com/item?id=49159530", "by": "Swapnoneel", "date": "2026-08-03", "stance": "warn", "quote": "I noticed a significant delay in the routing process", "excerpt": "So, I recently started using the Hermes agent and hooked it up with Step 3.5 Flash. It\u2019s a cheap and fast model, so I thought it would be enough for my workflow. However, I soon found that it hallucinates during a lot of tasks, and the output quality isn\u2019t something to write home about. On the other hand, using an expensive or frontier-level model means I\u2019ll burn through cash fast. I went down the rabbit hole to find", "context": "", "use_case": "model routing", "switched_from": "", "story": "Is Hermes Agent unusable without a model router?"}, {"id": 781271921, "item_id": 48829495, "url": "https://news.ycombinator.com/item?id=48829495", "by": "jarodrh", "date": "2026-07-08", "stance": "neutral", "quote": "offline estimate priced against LiteLLM", "excerpt": "I started leaning in on AI heavily this year, as I wanted to get more done autonomously, but then my token usage climbed dramatically to the point where my weekly quota would run out before the end of the week, sometimes a couple of days into the week. I realised I had to do something about it else I'd have to double my spend. So I decided to start tracking my cost per task type. This revealed that a lot of my spend ", "context": "", "use_case": "model pricing data", "switched_from": "", "story": null}, {"id": 780470609, "item_id": 48779413, "url": "https://news.ycombinator.com/item?id=48779413", "by": "westurner", "date": "2026-07-03", "stance": "recommend", "quote": "Aegis and LiteLLM, for example, are pre-execution firewalls that add a cryptographic audit trail.", "excerpt": "Remote debugging and post-mortem debugging support might be useful. There are many AI auditability proxies; awesome-auditable-ai: \"A curated list of papers, tools, datasets, benchmarks, and standards for building, evaluating, and auditing reliable AI agents\" https://github.com/yzhao062/awesome-auditable-ai Aegis and LiteLLM, for example, are pre-execution firewalls that add a cryptographic audit trail. https://github", "context": "", "use_case": "AI agent audit trail", "switched_from": "", "story": "Show HN: Mcpsnoop \u2013 Wireshark for MCP (transparent proxy and live TUI)"}, {"id": 780137184, "item_id": 48758574, "url": "https://news.ycombinator.com/item?id=48758574", "by": "lbreakjai", "date": "2026-07-02", "stance": "recommend", "quote": "We just cancelled everyone's plans and rolled liteLLM out internally.", "excerpt": "We just cancelled everyone's plans and rolled liteLLM out internally. We kept it for the insanely cheap tokens, but now that they've switched to the new pricing, they're just like openrouter, just with far fewer models.", "context": "company admin", "use_case": "internal LLM gateway", "switched_from": "GitHub Copilot", "story": "Kimi K2.7 Code is generally available in GitHub Copilot"}, {"id": 776286656, "item_id": 48517916, "url": "https://news.ycombinator.com/item?id=48517916", "by": "SamDc73", "date": "2026-06-13", "stance": "recommend", "quote": "I'm glad I used litellm in my last project", "excerpt": "I'm glad I used litellm in my last project https://github.com/BerriAI/litellm/", "context": "", "use_case": "LLM API proxy", "switched_from": "", "story": "AI OSS tool repo goes archived over night after raising $7.3M Seed"}, {"id": 775893553, "item_id": 48493347, "url": "https://news.ycombinator.com/item?id=48493347", "by": "msradam", "date": "2026-06-11", "stance": "neutral", "quote": "I use a LiteLLM proxy with Claude Code", "excerpt": "The inertia of switching. At least for me, being able to maintain the same UX and configuration for agentic coding is a benefit. I have access to internal models at work where I use a LiteLLM proxy with Claude Code, so no Anthropic models, but I can maintain the same .claude between personal and work machines. Though I'm definitely open to a non-Anthropic harness if it's a clear win.", "context": "", "use_case": "proxy to internal models", "switched_from": "", "story": "Running Claude Code Offline on an M3 Pro with Qwen3.6"}, {"id": 760037649, "item_id": 47502353, "url": "https://news.ycombinator.com/item?id=47502353", "by": "chillfox", "date": "2026-03-24", "stance": "warn", "quote": "LiteLLM was incredible flaky and kept causing outages", "excerpt": "Now I feel lucky that I switched to just using OpenRouter a year ago because LiteLLM was incredible flaky and kept causing outages.", "context": "", "use_case": "LLM API routing", "switched_from": "", "story": "LiteLLM Python package compromised by supply-chain attack"}, {"id": 749017344, "item_id": 46813584, "url": "https://news.ycombinator.com/item?id=46813584", "by": "Roark66", "date": "2026-01-29", "stance": "recommend", "quote": "I use litellm (slightly modified to allow cloud code telemetry pass through) and langfuse.", "excerpt": "I use litellm (slightly modified to allow cloud code telemetry pass through) and langfuse. There is no need for MitM, you can set Api base address to your own proxy in all the coding assistants (at least all I know - Claude Code, opencode, gemini, vc plugin). The changes I made allow use of the models endpoint in litellm at the same base url as telemetry and passing through Claude Max auth. This is not about using yo", "context": "", "use_case": "proxy to record LLM tool traffic", "switched_from": "", "story": "A MitM proxy to see what your LLM tools are sending"}, {"id": 747341073, "item_id": 46708817, "url": "https://news.ycombinator.com/item?id=46708817", "by": "photios", "date": "2026-01-21", "stance": "warn", "quote": "I got Claude Code running with Copilot APIs via the LiteLLM proxy, but it was a pain in the butt.", "excerpt": "I feel your pain. I used to work for a bank and the security team only approved Copilot use. OpenCode can use Copilot natively: https://opencode.ai/docs/providers/#github-copilot I got Claude Code running with Copilot APIs via the LiteLLM proxy, but it was a pain in the butt. Just use OpenCode.", "context": "used to work for a bank", "use_case": "proxying Copilot APIs", "switched_from": "", "story": "The Agentic AI Handbook: Production-Ready Patterns"}, {"id": 744891170, "item_id": 46555698, "url": "https://news.ycombinator.com/item?id=46555698", "by": "atmosx", "date": "2026-01-09", "stance": "neutral", "quote": "through vim-ai and liteLLM", "excerpt": "I am using the mistral API for personal projects through vim-ai and liteLLM. I must say that so far I don\u2019t miss ChatGPT or Gemini (which I use at work).", "context": "", "use_case": "LLM API proxy", "switched_from": "", "story": "Ask HN: How would you decouple from the US?"}, {"id": 744513569, "item_id": 46532098, "url": "https://news.ycombinator.com/item?id=46532098", "by": "billylo", "date": "2026-01-07", "stance": "recommend", "quote": "I would suggest to code it on top of ollama or LiteLLM", "excerpt": "Each distro is doing their own thing. If you are targeting Linux mainly, I would suggest to code it on top of ollama or LiteLLM", "context": "", "use_case": "LLM completion backend for Linux apps", "switched_from": "", "story": "Ask HN: What's a standard way for apps to request text completion as a service?"}]}