{"slug": "marginlab", "name": "MarginLab", "category": "ai", "category_name": "AI tools and apps", "description": "", "recommenders": 4, "commenters": 6, "warners": 0, "score": 4.12, "rising": -1.31, "first": "2026-04-12", "latest": "2026-09-21", "why": "To detect nerfing of a model, projects like https://marginlab.ai/trackers/claude-code/ are much much better", "why_is_quote": true, "has_verdict": false, "sentiment": {"recommend": 4, "neutral": 2, "compare": 0, "warn": 0}, "spark": [0, 0, 0, 0, 0, 3, 1, 0, 0, 0, 0, 0], "listed": true, "loved": [], "caveat": "", "short_quote": "To detect nerfing of a model, projects like https://marginlab.ai/trackers/claude-code/ are much much better", "kind": "", "icon": true, "domain": "marginlab.ai", "github_repo": null, "official_url": "https://marginlab.ai", "pricing": "", "aliases": ["MarginLab", "marginlab", "Margin Lab", "Marginlab"], "verdict": null, "praise": [{"label": "", "summary": "", "quotes": [{"id": 770120736, "item_id": 48132546, "url": "https://news.ycombinator.com/item?id=48132546", "by": "eis", "date": "2026-05-14", "version": null, "stance": "recommend", "quote": "To detect nerfing of a model, projects like https://marginlab.ai/trackers/claude-code/ are much much better", "excerpt": "Advantage for what exactly though? I'm not saying Elo Ranking doesn't give any information. It just doesn't give the information that the OP's project claims to be able to give: that models get nerfed over time. You could extract this kind of information from the raw results of each evaluation round between two models, ignoring any new model entries and compare these over time but not from the resulting Elo scores wi", "context": "", "use_case": "", "switched_from": "", "story": null}, {"id": 766284962, "item_id": 47892810, "url": "https://news.ycombinator.com/item?id=47892810", "by": "conception", "date": "2026-04-24", "version": null, "stance": "recommend", "quote": "https://marginlab.ai is one that has been around for a while.", "excerpt": "This is cool - just wanted to note https://marginlab.ai is one that has been around for a while.", "context": "", "use_case": "", "switched_from": "", "story": "I Cancelled Claude: Token Issues, Declining Quality, and Poor Support"}, {"id": 766069376, "item_id": 47879336, "url": "https://news.ycombinator.com/item?id=47879336", "by": "Keeeeeeeks", "date": "2026-04-23", "version": null, "stance": "recommend", "quote": "(no affiliation)", "excerpt": "https://marginlab.ai/ (no affiliation) There are a number of projects working on evals that can check how 'smart' a model is, but the methodology is tricky. One would want to run the exact same prompt, every day, at different times of the day, but if the eval prompt(s) are complex, the frontier lab could have a 'meta-cognitive' layer that looks for repetitive prompts, and either: a) feeds the model a pre-written outp", "context": "", "use_case": "Tracking model quality", "switched_from": "", "story": "An update on recent Claude Code quality reports"}, {"id": 763841248, "item_id": 47740078, "url": "https://news.ycombinator.com/item?id=47740078", "by": "armchairhacker", "date": "2026-04-12", "version": null, "stance": "recommend", "quote": "Make an AI usage tracker like https://marginlab.ai/trackers/codex/", "excerpt": "Make an AI usage tracker like https://marginlab.ai/trackers/codex/. These hearsay anecdotes prove nothing.", "context": "", "use_case": "", "switched_from": "", "story": "Pro Max 5x Quota Exhausted in 1.5 Hours Despite Moderate Usage"}]}], "caveats": [], "who": [], "tags": [], "use_cases": ["tracking llm performance", "tracking model quality"], "compared": [], "makers": [], "threads": [{"url": "https://news.ycombinator.com/item?id=49789224", "title": "Fable 5 \u2013 Median thinking declined in August", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=48640196", "title": "Will It Mythos?", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=48130711", "title": "item 48130711", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=47892019", "title": "I Cancelled Claude: Token Issues, Declining Quality, and Poor Support", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=47878905", "title": "An update on recent Claude Code quality reports", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=47739260", "title": "Pro Max 5x Quota Exhausted in 1.5 Hours Despite Moderate Usage", "mentions": 1}], "sentiment_by_month": {"2025-11": {}, "2025-12": {}, "2026-01": {}, "2026-02": {}, "2026-03": {}, "2026-04": {"recommend": 3}, "2026-05": {"recommend": 1}, "2026-06": {"neutral": 1}, "2026-07": {}, "2026-08": {}, "2026-09": {"neutral": 1}, "2026-10": {}}, "versions": [], "version_unnamed": 0, "family_ref": null, "runs_on": null, "lines": [], "quotes": [{"id": 796646161, "item_id": 49790385, "url": "https://news.ycombinator.com/item?id=49790385", "by": "arcanemachiner", "date": "2026-09-21", "version": null, "stance": "neutral", "quote": "Or as a loss leader to get eyeballs to your service (like Margin Lab[1])", "excerpt": "The only revenue model I for this is ads (like AI Stupid Level[0]). Or as a loss leader to get eyeballs to your service (like Margin Lab[1]). EDIT: I forgot (and am shocked) that HN still doesn't seem to support Markdown-style links. [0] https://aistupidlevel.info/ [1] https://marginlab.ai/trackers/claude-code/", "context": "", "use_case": "", "switched_from": "", "story": "Fable 5 \u2013 Median thinking declined in August"}, {"id": 778319952, "item_id": 48644997, "url": "https://news.ycombinator.com/item?id=48644997", "by": "imxyy_soope_", "date": "2026-06-23", "version": null, "stance": "neutral", "quote": "There are LLM performance trackers in the wild, for instance https://marginlab.ai", "excerpt": "There are LLM performance trackers in the wild, for instance https://marginlab.ai You may notice that the performance of the old model tends to decline before each new model release.", "context": "", "use_case": "Tracking LLM performance", "switched_from": "", "story": "Will It Mythos?"}, {"id": 770120736, "item_id": 48132546, "url": "https://news.ycombinator.com/item?id=48132546", "by": "eis", "date": "2026-05-14", "version": null, "stance": "recommend", "quote": "To detect nerfing of a model, projects like https://marginlab.ai/trackers/claude-code/ are much much better", "excerpt": "Advantage for what exactly though? I'm not saying Elo Ranking doesn't give any information. It just doesn't give the information that the OP's project claims to be able to give: that models get nerfed over time. You could extract this kind of information from the raw results of each evaluation round between two models, ignoring any new model entries and compare these over time but not from the resulting Elo scores wi", "context": "", "use_case": "", "switched_from": "", "story": null}, {"id": 766284962, "item_id": 47892810, "url": "https://news.ycombinator.com/item?id=47892810", "by": "conception", "date": "2026-04-24", "version": null, "stance": "recommend", "quote": "https://marginlab.ai is one that has been around for a while.", "excerpt": "This is cool - just wanted to note https://marginlab.ai is one that has been around for a while.", "context": "", "use_case": "", "switched_from": "", "story": "I Cancelled Claude: Token Issues, Declining Quality, and Poor Support"}, {"id": 766069376, "item_id": 47879336, "url": "https://news.ycombinator.com/item?id=47879336", "by": "Keeeeeeeks", "date": "2026-04-23", "version": null, "stance": "recommend", "quote": "(no affiliation)", "excerpt": "https://marginlab.ai/ (no affiliation) There are a number of projects working on evals that can check how 'smart' a model is, but the methodology is tricky. One would want to run the exact same prompt, every day, at different times of the day, but if the eval prompt(s) are complex, the frontier lab could have a 'meta-cognitive' layer that looks for repetitive prompts, and either: a) feeds the model a pre-written outp", "context": "", "use_case": "Tracking model quality", "switched_from": "", "story": "An update on recent Claude Code quality reports"}, {"id": 763841248, "item_id": 47740078, "url": "https://news.ycombinator.com/item?id=47740078", "by": "armchairhacker", "date": "2026-04-12", "version": null, "stance": "recommend", "quote": "Make an AI usage tracker like https://marginlab.ai/trackers/codex/", "excerpt": "Make an AI usage tracker like https://marginlab.ai/trackers/codex/. These hearsay anecdotes prove nothing.", "context": "", "use_case": "", "switched_from": "", "story": "Pro Max 5x Quota Exhausted in 1.5 Hours Despite Moderate Usage"}]}