Key takeaways
- A GPT-2 grade chatbot for about $48 of rented GPU time, by the project's own figures. The 2019 cost was about $43,000.
- You get one readable codebase covering the whole path from tokeniser to chat, so you can see how a model is built.
- One setting, the number of layers, scales the model, and the rest is worked out automatically.
- Catch: the result is small, and the README's own cost and time figures differ. Treat them as estimates.
What it is
nanochat is an open-source harness for training small ChatGPT-style language models, written by Andrej Karpathy and found on GitHub. It describes itself as the simplest experimental harness for training LLMs, designed for a single GPU node.
The problem it solves
Training a language model usually looks out of reach: expensive hardware, many moving parts, and codebases too large to read. The README says GPT-2 capability cost about $43,000 to train in 2019. nanochat aims to let students and researchers work on small models end to end for under $1000, in code that is short enough to understand.
What you get
- The whole pipeline in one place: tokenisation, pretraining, finetuning, evaluation and chat, so nothing is hidden.
- One dial for model size. The number of layers sets the rest, so you do not tune dozens of settings.
- A public leaderboard for time to a GPT-2 grade model, which gives researchers a clear target.
- A deliberately small baseline rather than a configurable framework, which makes it easy to fork and change.
How it compares
The README also says a spot instance could bring the total closer to about $15, and elsewhere says the run takes about 1.5 hours. Its own figures therefore differ, so treat all of them as estimates that depend on GPU prices.
The catches
- The model is small. The README compares talking to it with talking to a kindergartener, and it can hallucinate.
- You need rented GPU time. Less than 80GB of GPU memory means tuning, and a single GPU takes about eight times longer.
- The author has not tried every hardware path, so expect sharp edges. CPU runs give weak results.
- It is an MIT-licensed research and learning tool, not a product.
What we did not test
We did not train a model or run any scripts. Each claim is tied to a quote from the project's README or the GitHub API, and the cost and time figures are the project's, not ours.
Verdict
Editors' view: worth attention from students, researchers and engineers who want to understand how a chat model is built, under the MIT licence. Anyone hoping for a ready-to-use assistant should look elsewhere.
Sources
Need help adopting this, or have a software project in mind?
Tell us what you are building. The initial consultation is free. Email [email protected] or follow @zolnaltd.
Start a conversation →