{"slug": "nanochat", "name": "nanochat", "category": "ai", "category_name": "AI tools and apps", "description": "", "recommenders": 6, "commenters": 13, "warners": 1, "score": 5.807, "rising": -1.96, "first": "2025-11-02", "latest": "2026-08-24", "why": "The recently released nanochat https://github.com/karpathy/nanochat is fantastic.", "why_is_quote": true, "has_verdict": false, "sentiment": {"recommend": 6, "neutral": 6, "compare": 0, "warn": 1}, "spark": [1, 1, 1, 1, 0, 1, 1, 0, 0, 0, 0, 0], "listed": true, "loved": [], "caveat": "", "short_quote": "The recently released nanochat https://github.com/karpathy/nanochat is fantastic.", "kind": "", "icon": false, "domain": null, "github_repo": "github.com/karpathy/nanochat", "official_url": "https://github.com/karpathy/nanochat", "pricing": "", "aliases": ["nanochat", "Nanochat"], "verdict": null, "praise": [{"label": "", "summary": "", "quotes": [{"id": 771984960, "item_id": 48249060, "url": "https://news.ycombinator.com/item?id=48249060", "by": "lancekey", "date": "2026-05-23", "version": null, "stance": "recommend", "quote": "I used the repo, claude and shadeform to train my own mini model for about $300", "excerpt": "Also check out his nanochat repo. I used the repo, claude and shadeform to train my own mini model for about $300. Would have been less but I screwed up and let the cloud gpu rental run for a few hours even though the training run errored out. Of course the model was dumber than GPT2 but still it was a great learning experience.", "context": "", "use_case": "training a mini model", "switched_from": "", "story": "Making Deep Learning Go Brrrr from First Principles"}, {"id": 762442288, "item_id": 47652643, "url": "https://news.ycombinator.com/item?id=47652643", "by": "wongarsu", "date": "2026-04-05", "version": null, "stance": "recommend", "quote": "Kaparthy's notes on improving nanochat [1] are one of my favorite blog-like things to read.", "excerpt": "It's also a great base for experimentation. If you have an idea for an architecture improvement you can try it for $36 on the 20 layer nanocode setting, then for another $200 see how it holds up on the \"full scale\" nanocode Kaparthy's notes on improving nanochat [1] are one of my favorite blog-like things to read. Really neat to see which features have how much influence, and how the scaling laws evolve as you improv", "context": "", "use_case": "LLM training experiments", "switched_from": "", "story": "Nanocode: The best Claude Code that $200 can buy in pure JAX on TPUs"}, {"id": 750112288, "item_id": 46882018, "url": "https://news.ycombinator.com/item?id=46882018", "by": "mips_avatar", "date": "2026-02-03", "version": null, "stance": "recommend", "quote": "You should check out nanochat.", "excerpt": "You should check out nanochat. I would personally appreciate it if someone implemented hardware optimized flash attention for my 3090", "context": "", "use_case": "", "switched_from": "", "story": "FlashAttention-T: Towards Tensorized Attention"}, {"id": 748567792, "item_id": 46785487, "url": "https://news.ycombinator.com/item?id=46785487", "by": "reducesuffering", "date": "2026-01-27", "version": null, "stance": "recommend", "quote": "https://github.com/karpathy/nanochat", "excerpt": "https://github.com/karpathy/nanochat https://github.com/karpathy/llm.c The proof is in the pudding. Let's see your code", "context": "", "use_case": "", "switched_from": "", "story": "Karpathy: A few random notes from Claude coding quite a bit last few weeks"}, {"id": 739839265, "item_id": 46239954, "url": "https://news.ycombinator.com/item?id=46239954", "by": "huevosabio", "date": "2025-12-11", "version": null, "stance": "recommend", "quote": "If you relax the SOTA aspect, Karpathy's nanochat has you covered", "excerpt": "The Olmo team is AFAIK the only SOTA-ish model that has fully open source code and data. Their report is fantastic: https://www.datocms-assets.com/64837/1763662397-1763646865-olmo_3_technical_report-1.pdf It should give you an idea of how hard it is to do a SOTA model from scratch! If you relax the SOTA aspect, Karpathy's nanochat has you covered: https://github.com/karpathy/nanochat", "context": "", "use_case": "", "switched_from": "", "story": "Ask HN: Relatively SoTA LLM Agents from Scratch?"}]}], "caveats": [{"label": "", "summary": "", "quotes": [{"id": 736013056, "item_id": 46000816, "url": "https://news.ycombinator.com/item?id=46000816", "by": "AnonHP", "date": "2025-11-20", "version": null, "stance": "warn", "quote": "this is like a kindergartner in some ways", "excerpt": "As it says on the GitHub page, [1] this is like a kindergartner in some ways. I just started with the same question as on that page, \u201cwhy is the sky blue?\u201d and followed up with why it\u2019s orange during sunrise or sunset. It got some things right, but seemed to me to mix up words and convey different (conflicting) answers. So I finally told it the following. > Me: This has been a total waste of time and no clear knowled", "context": "", "use_case": "", "switched_from": "", "story": "nanochat.karpathy.ai"}]}], "who": [{"by": "deepllm", "context": "17-year-old with an 8GB NVIDIA GPU", "url": "https://news.ycombinator.com/item?id=49417517", "date": "2026-08-24"}], "tags": [], "use_cases": ["learning to train llms", "training a mini model", "llm training experiments", "training gpt-2-level model", "training small gpt-2-level models", "minimal llm training examples"], "compared": [], "makers": [], "threads": [{"url": "https://news.ycombinator.com/item?id=45997425", "title": "nanochat.karpathy.ai", "mentions": 2}, {"url": "https://news.ycombinator.com/item?id=45787993", "title": "Backpropagation is a leaky abstraction (2016)", "mentions": 2}, {"url": "https://news.ycombinator.com/item?id=49412396", "title": "I were 17, I'd learn how to build LLMs from scratch", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=48246889", "title": "Making Deep Learning Go Brrrr from First Principles", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=47649742", "title": "Nanocode: The best Claude Code that $200 can buy in pure JAX on TPUs", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=47291123", "title": "Autoresearch: Agents researching on single-GPU nanochat training automatically", "mentions": 1}], "sentiment_by_month": {"2025-11": {"recommend": 1, "neutral": 2, "warn": 1}, "2025-12": {"recommend": 1}, "2026-01": {"recommend": 1}, "2026-02": {"recommend": 1}, "2026-03": {"neutral": 3}, "2026-04": {"recommend": 1}, "2026-05": {"recommend": 1}, "2026-06": {}, "2026-07": {}, "2026-08": {"neutral": 1}, "2026-09": {}, "2026-10": {}}, "versions": [], "version_unnamed": 0, "quotes": [{"id": 790680272, "item_id": 49417517, "url": "https://news.ycombinator.com/item?id=49417517", "by": "deepllm", "date": "2026-08-24", "version": null, "stance": "neutral", "quote": "torturing yourself working the GPU for days per iteration trying to run things like https://github.com/karpathy/nanochat, which is hardly an educational experience", "excerpt": "As someone who's at a similar age and was interested in learning how to do this, there just aren't enough resources to do so. Most LLM research is in the form of academic papers, and there isn't any 'popular' way to learn these things, and besides that all said research assumes you have a B200 cluster ready to go. If you have weaker hardware (say an 8GB nVidia GPU, which is what I have) you're going to be limited to ", "context": "17-year-old with an 8GB NVIDIA GPU", "use_case": "learning to train LLMs", "switched_from": "", "story": "I were 17, I'd learn how to build LLMs from scratch"}, {"id": 771984960, "item_id": 48249060, "url": "https://news.ycombinator.com/item?id=48249060", "by": "lancekey", "date": "2026-05-23", "version": null, "stance": "recommend", "quote": "I used the repo, claude and shadeform to train my own mini model for about $300", "excerpt": "Also check out his nanochat repo. I used the repo, claude and shadeform to train my own mini model for about $300. Would have been less but I screwed up and let the cloud gpu rental run for a few hours even though the training run errored out. Of course the model was dumber than GPT2 but still it was a great learning experience.", "context": "", "use_case": "training a mini model", "switched_from": "", "story": "Making Deep Learning Go Brrrr from First Principles"}, {"id": 762442288, "item_id": 47652643, "url": "https://news.ycombinator.com/item?id=47652643", "by": "wongarsu", "date": "2026-04-05", "version": null, "stance": "recommend", "quote": "Kaparthy's notes on improving nanochat [1] are one of my favorite blog-like things to read.", "excerpt": "It's also a great base for experimentation. If you have an idea for an architecture improvement you can try it for $36 on the 20 layer nanocode setting, then for another $200 see how it holds up on the \"full scale\" nanocode Kaparthy's notes on improving nanochat [1] are one of my favorite blog-like things to read. Really neat to see which features have how much influence, and how the scaling laws evolve as you improv", "context": "", "use_case": "LLM training experiments", "switched_from": "", "story": "Nanocode: The best Claude Code that $200 can buy in pure JAX on TPUs"}, {"id": 756714080, "item_id": 47294630, "url": "https://news.ycombinator.com/item?id=47294630", "by": "categoricalrift", "date": "2026-03-07", "version": null, "stance": "neutral", "quote": "", "excerpt": "How about the very last \"Kept Improvement\" in the plot? It's titled \"random seed 42 -> 137\". I do think this project is quite conceptually interesting, but the model literally choosing a different random seed to achieve lower loss feels pretty far removed from the flowery sci-fi writing at the top of the readme.", "context": "", "use_case": "", "switched_from": "", "story": "Autoresearch: Agents researching on single-GPU nanochat training automatically"}, {"id": 756235696, "item_id": 47264731, "url": "https://news.ycombinator.com/item?id=47264731", "by": "k2so", "date": "2026-03-05", "version": null, "stance": "neutral", "quote": "I wanted to train Nanochat d26 to GPT-2 level", "excerpt": "Author here. I wanted to train Nanochat d26 to GPT-2 level and had to pick between three H100 variants on Runpod. SXM was the most expensive per hour but cheapest to finish: SXM: 702ms/step - ~$37 (using vast.ai) PCIe: 1,412ms/step - ~$112 (runpod) NVL: 2,032ms/step - ~$181 (runpod) My first SXM run hit 1,295ms. Barely faster than PCIe. Nsight OS runtime summary led me to suspect CPU starvation. I found a higher vCPU", "context": "", "use_case": "training GPT-2-level model", "switched_from": "", "story": "Which H100 Instance to Train Nanochat \u2013 Benchmarking PCIe, SXM, and NVL"}, {"id": 755298224, "item_id": 47206139, "url": "https://news.ycombinator.com/item?id=47206139", "by": "killerstorm", "date": "2026-03-01", "version": null, "stance": "neutral", "quote": "Karpathy has other projects, e.g. : https://github.com/karpathy/nanochat", "excerpt": "You're missing the point. Karpathy has other projects, e.g. : https://github.com/karpathy/nanochat You can train a model with GPT-2 level of capability for $20-$100. But, guess what, that's exactly what thousands of AI researchers have been doing for the past 5+ years. They've been training smallish models. And while these smallish models might be good for classification and whatnot, people strongly prefer big-ass fr", "context": "", "use_case": "Training small GPT-2-level models", "switched_from": "", "story": "Microgpt"}, {"id": 750112288, "item_id": 46882018, "url": "https://news.ycombinator.com/item?id=46882018", "by": "mips_avatar", "date": "2026-02-03", "version": null, "stance": "recommend", "quote": "You should check out nanochat.", "excerpt": "You should check out nanochat. I would personally appreciate it if someone implemented hardware optimized flash attention for my 3090", "context": "", "use_case": "", "switched_from": "", "story": "FlashAttention-T: Towards Tensorized Attention"}, {"id": 748567792, "item_id": 46785487, "url": "https://news.ycombinator.com/item?id=46785487", "by": "reducesuffering", "date": "2026-01-27", "version": null, "stance": "recommend", "quote": "https://github.com/karpathy/nanochat", "excerpt": "https://github.com/karpathy/nanochat https://github.com/karpathy/llm.c The proof is in the pudding. Let's see your code", "context": "", "use_case": "", "switched_from": "", "story": "Karpathy: A few random notes from Claude coding quite a bit last few weeks"}, {"id": 739839265, "item_id": 46239954, "url": "https://news.ycombinator.com/item?id=46239954", "by": "huevosabio", "date": "2025-12-11", "version": null, "stance": "recommend", "quote": "If you relax the SOTA aspect, Karpathy's nanochat has you covered", "excerpt": "The Olmo team is AFAIK the only SOTA-ish model that has fully open source code and data. Their report is fantastic: https://www.datocms-assets.com/64837/1763662397-1763646865-olmo_3_technical_report-1.pdf It should give you an idea of how hard it is to do a SOTA model from scratch! If you relax the SOTA aspect, Karpathy's nanochat has you covered: https://github.com/karpathy/nanochat", "context": "", "use_case": "", "switched_from": "", "story": "Ask HN: Relatively SoTA LLM Agents from Scratch?"}, {"id": 736013056, "item_id": 46000816, "url": "https://news.ycombinator.com/item?id=46000816", "by": "AnonHP", "date": "2025-11-20", "version": null, "stance": "warn", "quote": "this is like a kindergartner in some ways", "excerpt": "As it says on the GitHub page, [1] this is like a kindergartner in some ways. I just started with the same question as on that page, \u201cwhy is the sky blue?\u201d and followed up with why it\u2019s orange during sunrise or sunset. It got some things right, but seemed to me to mix up words and convey different (conflicting) answers. So I finally told it the following. > Me: This has been a total waste of time and no clear knowled", "context": "", "use_case": "", "switched_from": "", "story": "nanochat.karpathy.ai"}, {"id": 735971376, "item_id": 45998211, "url": "https://news.ycombinator.com/item?id=45998211", "by": "conqrr", "date": "2025-11-20", "version": null, "stance": "neutral", "quote": "Context: https://github.com/karpathy/nanochat", "excerpt": "Context: https://github.com/karpathy/nanochat", "context": "", "use_case": "", "switched_from": "", "story": "nanochat.karpathy.ai"}, {"id": 732805968, "item_id": 45800373, "url": "https://news.ycombinator.com/item?id=45800373", "by": "leobg", "date": "2025-11-03", "version": null, "stance": "neutral", "quote": "he published nanochat 3 weeks ago", "excerpt": "Also, he published nanochat 3 weeks ago [0], and he says in the readme: > nanochat will become the capstone project of the course LLM101n being developed by Eureka Labs. [0] https://github.com/karpathy/nanochat", "context": "", "use_case": "", "switched_from": "", "story": "Backpropagation is a leaky abstraction (2016)"}, {"id": 732615488, "item_id": 45788468, "url": "https://news.ycombinator.com/item?id=45788468", "by": "gchadwick", "date": "2025-11-02", "version": null, "stance": "recommend", "quote": "The recently released nanochat https://github.com/karpathy/nanochat is fantastic.", "excerpt": "Karpathy's contribution to teaching around deep learning is just immense. He's got a mountain of fantastic material from short articles like this, longer writing like https://karpathy.github.io/2015/05/21/rnn-effectiveness/ (on recurrent neural networks) and all of the stuff on YouTube. Plus his GitHub. The recently released nanochat https://github.com/karpathy/nanochat is fantastic. Having minimal, understandable an", "context": "", "use_case": "minimal LLM training examples", "switched_from": "", "story": "Backpropagation is a leaky abstraction (2016)"}]}