{"slug": "bert", "name": "BERT", "category": "ai-models", "category_name": "AI models", "description": "", "recommenders": 3, "commenters": 12, "warners": 1, "score": 4.073, "rising": 3.0, "first": "2025-11-20", "latest": "2026-10-24", "why": "it's still worthwhile to review these models and codebases", "why_is_quote": true, "has_verdict": false, "sentiment": {"recommend": 3, "neutral": 7, "compare": 1, "warn": 2}, "spark": [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1], "listed": true, "loved": [], "caveat": "", "short_quote": "it's still worthwhile to review these models and codebases", "kind": "", "icon": false, "domain": null, "github_repo": "github.com/google-research/bert", "official_url": "https://github.com/google-research/bert", "pricing": "", "aliases": ["BERT"], "verdict": null, "praise": [{"label": "", "summary": "", "quotes": [{"id": 803145712, "item_id": 50196607, "url": "https://news.ycombinator.com/item?id=50196607", "by": "PaulHoule", "date": "2026-10-22", "version": null, "stance": "recommend", "quote": "[BERT] -> [pooling] -> [probability calibrated SVM]", "excerpt": "There's a science behind calibrating machine learning models to compute probability estimates https://scikit-learn.org/stable/modules/calibration.html I've had very good luck with [BERT] -> [pooling] -> [probability calibrated SVM] but you can use simpler models like [Bag of words] -> [logistic regression] or put the classical ML model after an LLM-based model that includes a prompt to give a \"yes\"/\"no\" answer repres", "context": "", "use_case": "", "switched_from": "", "story": "Using LLM for confidence scoring is a bad idea"}, {"id": 798164384, "item_id": 49885274, "url": "https://news.ycombinator.com/item?id=49885274", "by": "prometheus1992", "date": "2026-09-28", "version": null, "stance": "recommend", "quote": "we used various bert variants to satisfy our usecases and still in uses", "excerpt": "nothing in utility. we used various bert variants to satisfy our usecases and still in uses. free and they run locally.", "context": "", "use_case": "text classification", "switched_from": "", "story": "Jeff \u2013 Jev-compatible 0.8B decision models, trained at home, ~30 ms"}, {"id": 787986546, "item_id": 49249159, "url": "https://news.ycombinator.com/item?id=49249159", "by": "canyon289", "date": "2026-08-10", "version": null, "stance": "recommend", "quote": "it's still worthwhile to review these models and codebases", "excerpt": "Disclaimer, I work on Gemma and open models at Deepmind and the opinions here are my own There were open models from EleutherAI (GPT-Neo), Google Brain (T5X, Bert), and HuggingFace was promoting open models (and others doing open work I haven't listed here) all prior to 2023 and the big Chatgpt moment. https://github.com/EleutherAI/gpt-neo/releases https://github.com/google-research/bert https://github.com/google-res", "context": "", "use_case": "", "switched_from": "", "story": "Mark Zuckerberg attacks 'closed' AI rivals as Meta returns to open models"}]}], "caveats": [{"label": "", "summary": "", "quotes": [{"id": 743827986, "item_id": 46489249, "url": "https://news.ycombinator.com/item?id=46489249", "by": "PaulHoule", "date": "2026-01-04", "version": null, "stance": "warn", "quote": "don't screw around with fine-tuned BERT", "excerpt": "It's pretty easy. SBERT + classical classifiers from scikit-learn, don't forget the probability calibration. I get diversity by clustering on k-Means and taking the best N/k from k=20 clusters and I also blend in about 30% random items to keep the system honest. It's on my agenda to make a general-purpose text classifier with a \"better\" model (better sensitivity to word order) but I don't think a better AUC-ROC would", "context": "", "use_case": "", "switched_from": "", "story": "Nike's Crisis and the Economics of Brand Decay"}]}], "who": [], "tags": [], "use_cases": ["text classification", "classification embeddings", "named entity recognition", "text generation via diffusion", "reranking markov chain output"], "compared": [{"slug": "jev", "name": "Jev", "co": 5, "switched_from": 0, "switched_to": 0}], "makers": [], "threads": [{"url": "https://news.ycombinator.com/item?id=50222365", "title": "The $25B Eigenvector \u2013 The Linear Algebra Behind Google (2006) [pdf]", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=50176082", "title": "Using LLM for confidence scoring is a bad idea", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=49933476", "title": "Decision models like Jev don't beat LLM-as-a-judge or traditional classifiers", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=49883844", "title": "Jeff \u2013 Jev-compatible 0.8B decision models, trained at home, ~30 ms", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=49783999", "title": "Kev: Tiny Jev-like family of decision models built on top of Qwen3.5", "mentions": 1}, {"url": "https://news.ycombinator.com/item?id=49765348", "title": "I built non-autoregressive decision models with RL a year ago", "mentions": 1}], "sentiment_by_month": {"2025-11": {"neutral": 1}, "2025-12": {}, "2026-01": {"warn": 1}, "2026-02": {}, "2026-03": {"neutral": 1}, "2026-04": {"neutral": 1}, "2026-05": {}, "2026-06": {}, "2026-07": {"neutral": 1}, "2026-08": {"recommend": 1}, "2026-09": {"neutral": 2, "warn": 1, "recommend": 1}, "2026-10": {"neutral": 1, "recommend": 1, "compare": 1}}, "versions": [], "version_unnamed": 12, "family_ref": null, "runs_on": null, "lines": [], "quotes": [{"id": 803559969, "item_id": 50222498, "url": "https://news.ycombinator.com/item?id=50222498", "by": "armcat", "date": "2026-10-24", "version": null, "stance": "compare", "quote": "testing TextRank against BERT for 2-3 sentence article summaries", "excerpt": "Sometimes the simplest ideas are the best. One method I employed a lot \"back in the day\" for extractive summarisation of text, is TextRank [1], which applies that multi-billion dollar linear algebra procedure against text. I remember working at a news media company and testing TextRank against BERT for 2-3 sentence article summaries, and TextRank was better. [1] https://aclanthology.org/W04-3252.pdf", "context": "", "use_case": "", "switched_from": "", "story": "The $25B Eigenvector \u2013 The Linear Algebra Behind Google (2006) [pdf]"}, {"id": 803145712, "item_id": 50196607, "url": "https://news.ycombinator.com/item?id=50196607", "by": "PaulHoule", "date": "2026-10-22", "version": null, "stance": "recommend", "quote": "[BERT] -> [pooling] -> [probability calibrated SVM]", "excerpt": "There's a science behind calibrating machine learning models to compute probability estimates https://scikit-learn.org/stable/modules/calibration.html I've had very good luck with [BERT] -> [pooling] -> [probability calibrated SVM] but you can use simpler models like [Bag of words] -> [logistic regression] or put the classical ML model after an LLM-based model that includes a prompt to give a \"yes\"/\"no\" answer repres", "context": "", "use_case": "", "switched_from": "", "story": "Using LLM for confidence scoring is a bad idea"}, {"id": 799001731, "item_id": 49937608, "url": "https://news.ycombinator.com/item?id=49937608", "by": "AnthusAI", "date": "2026-10-02", "version": null, "stance": "neutral", "quote": "you can use BERT with sequence classification for simple classification tasks", "excerpt": "That was a pretty simple task they gave it, and sure you can use BERT with sequence classification for simple classification tasks. In our benchmarks, Jev did a LOT better at multi-step reasoning tasks than any open decision model we have tested so far, and it was also better than GLiDE which was specifically designed for that kind of task. And also better than Luna. On accuracy and also confidence calibration but al", "context": "", "use_case": "", "switched_from": "", "story": "Decision models like Jev don't beat LLM-as-a-judge or traditional classifiers"}, {"id": 798164384, "item_id": 49885274, "url": "https://news.ycombinator.com/item?id=49885274", "by": "prometheus1992", "date": "2026-09-28", "version": null, "stance": "recommend", "quote": "we used various bert variants to satisfy our usecases and still in uses", "excerpt": "nothing in utility. we used various bert variants to satisfy our usecases and still in uses. free and they run locally.", "context": "", "use_case": "text classification", "switched_from": "", "story": "Jeff \u2013 Jev-compatible 0.8B decision models, trained at home, ~30 ms"}, {"id": 796703874, "item_id": 49793992, "url": "https://news.ycombinator.com/item?id=49793992", "by": "npn", "date": "2026-09-21", "version": null, "stance": "warn", "quote": "", "excerpt": "> And even if BERT wasn't woefully underintelligent for the task... have 100+ instances of BERT running locally faster than Jev API response times? Sweet rig you must have... why the heck do you need 100+ instances of bert. do you even attempt to research about this before? the laya paper show that you can do the similar stuff with jev using modern bert only: https://laya.convaiinnovations.com/ and even without the n", "context": "", "use_case": "", "switched_from": "", "story": "Kev: Tiny Jev-like family of decision models built on top of Qwen3.5"}, {"id": 796283664, "item_id": 49767729, "url": "https://news.ycombinator.com/item?id=49767729", "by": "evrydayhustling", "date": "2026-09-19", "version": null, "stance": "neutral", "quote": "We used to use BERT-based embeddings + semantic distance for classification / decision problems in new domains.", "excerpt": "We used to use BERT-based embeddings + semantic distance for classification / decision problems in new domains. There was a lot of interest at the time in these kinds of pre-generative but portable models -- Meta's Prophet was another example that came up a lot.", "context": "", "use_case": "classification embeddings", "switched_from": "", "story": "I built non-autoregressive decision models with RL a year ago"}, {"id": 796051938, "item_id": 49753246, "url": "https://news.ycombinator.com/item?id=49753246", "by": "Topfi", "date": "2026-09-18", "version": null, "stance": "neutral", "quote": "Something like BERT could be and has been tuned", "excerpt": "Jev is, as far as I understand, essentially very optimised for zero shot classification [0]. Something like BERT could be and has been tuned to provide similar \"decision making\" at a similar latency and cost advantage quite some time back. Advantage over full on LLMs is mainly the efficiency and of something like Jev over e.g. the encoder/decoder based classifier I had in front of an LLM to route to different prompts", "context": "", "use_case": "", "switched_from": "", "story": "OpenJev"}, {"id": 787986546, "item_id": 49249159, "url": "https://news.ycombinator.com/item?id=49249159", "by": "canyon289", "date": "2026-08-10", "version": null, "stance": "recommend", "quote": "it's still worthwhile to review these models and codebases", "excerpt": "Disclaimer, I work on Gemma and open models at Deepmind and the opinions here are my own There were open models from EleutherAI (GPT-Neo), Google Brain (T5X, Bert), and HuggingFace was promoting open models (and others doing open work I haven't listed here) all prior to 2023 and the big Chatgpt moment. https://github.com/EleutherAI/gpt-neo/releases https://github.com/google-research/bert https://github.com/google-res", "context": "", "use_case": "", "switched_from": "", "story": "Mark Zuckerberg attacks 'closed' AI rivals as Meta returns to open models"}, {"id": 785483536, "item_id": 49092721, "url": "https://news.ycombinator.com/item?id=49092721", "by": "vkaku", "date": "2026-07-28", "version": null, "stance": "neutral", "quote": "run this with a BERT model", "excerpt": "I did some research about scaling laws and what's unsolved there. This project was built on structural sparsity, do away with any requirements to train models on largely outdated corpus of data, and only train it on reasoning logic. Because if models can use tools well and retrieve everything else, it greatly would reduce the need for large language models. Further, when generating traces, I ran a benchmark that show", "context": "", "use_case": "", "switched_from": "", "story": "A $500 RL fine-tune of a 9B open model beat frontier models on catalog review"}, {"id": 766514737, "item_id": 47907171, "url": "https://news.ycombinator.com/item?id=47907171", "by": "nl", "date": "2026-04-25", "version": null, "stance": "neutral", "quote": "uses a BERT based named-entity-recognition model running in your browser", "excerpt": "I'm no where near as smart as OpenAI of course, but I did build https://tools.nicklothian.com/webner/index.html that uses a BERT based named-entity-recognition model running in your browser to do a subset of PII redaction. It works pretty well for the use cases I was playing with. The OpenAI model is small enough that I might enhance my tool to use it.", "context": "", "use_case": "Named entity recognition", "switched_from": "", "story": "OpenAI model for masking personally identifiable information (PII) in text"}, {"id": 755297392, "item_id": 47206087, "url": "https://news.ycombinator.com/item?id=47206087", "by": "joefourier", "date": "2026-03-01", "version": null, "stance": "neutral", "quote": "You can actually generate surprisingly coherent text with minimal finetuning of BERT", "excerpt": "You can actually generate surprisingly coherent text with minimal finetuning of BERT, by reinterpreting it as a diffusion model: https://nathan.rs/posts/roberta-diffusion/ I don\u2019t see a useful definition of LLM that doesn\u2019t include BERT, especially given its historical importance. 340M parameters is only \u201csmall\u201d in the sense that a baby whale is small.", "context": "", "use_case": "Text generation via diffusion", "switched_from": "", "story": "Microgpt"}, {"id": 743827986, "item_id": 46489249, "url": "https://news.ycombinator.com/item?id=46489249", "by": "PaulHoule", "date": "2026-01-04", "version": null, "stance": "warn", "quote": "don't screw around with fine-tuned BERT", "excerpt": "It's pretty easy. SBERT + classical classifiers from scikit-learn, don't forget the probability calibration. I get diversity by clustering on k-Means and taking the best N/k from k=20 clusters and I also blend in about 30% random items to keep the system honest. It's on my agenda to make a general-purpose text classifier with a \"better\" model (better sensitivity to word order) but I don't think a better AUC-ROC would", "context": "", "use_case": "", "switched_from": "", "story": "Nike's Crisis and the Economics of Brand Decay"}, {"id": 735943456, "item_id": 45996466, "url": "https://news.ycombinator.com/item?id=45996466", "by": "thatjoeoverthr", "date": "2025-11-20", "version": null, "stance": "neutral", "quote": "using a trained BERT model as a reranker for Markov chain emmisions", "excerpt": "Others have mentioned the large context window. This matters. But also important is embeddings. Tokens in a classic Markov chain are discrete surrogate keys. \u201cLove\u201d, for example, and \u201clove\u201d are two different tokens. As are \u201crage\u201d and \u201cfury\u201d. In a modern model, we start with an embedding model, and build a LUT mapping token identities to vectors. This does two things for you. First, it solves the above problem, which ", "context": "", "use_case": "Reranking Markov chain output", "switched_from": "", "story": "Ask HN: How are Markov chains so different from tiny LLMs?"}]}