Key takeaways

  • A GPT-2 grade chatbot for about $48 of rented GPU time, by the project's own figures. The 2019 cost was about $43,000.
  • You get one readable codebase covering the whole path from tokeniser to chat, so you can see how a model is built.
  • One setting, the number of layers, scales the model, and the rest is worked out automatically.
  • Catch: the result is small, and the README's own cost and time figures differ. Treat them as estimates.

What it is

nanochat is an open-source harness for training small ChatGPT-style language models, written by Andrej Karpathy and found on GitHub. It describes itself as the simplest experimental harness for training LLMs, designed for a single GPU node.

The problem it solves

Training a language model usually looks out of reach: expensive hardware, many moving parts, and codebases too large to read. The README says GPT-2 capability cost about $43,000 to train in 2019. nanochat aims to let students and researchers work on small models end to end for under $1000, in code that is short enough to understand.

What you get

  • The whole pipeline in one place: tokenisation, pretraining, finetuning, evaluation and chat, so nothing is hidden.
  • One dial for model size. The number of layers sets the rest, so you do not tune dozens of settings.
  • A public leaderboard for time to a GPT-2 grade model, which gives researchers a clear target.
  • A deliberately small baseline rather than a configurable framework, which makes it easy to fork and change.

How it compares

about $48GPT-2 capability now, about 2 hours of an 8XH100 node (project's own figure)
about $43,000The same capability in 2019 (project's own figure)

The README also says a spot instance could bring the total closer to about $15, and elsewhere says the run takes about 1.5 hours. Its own figures therefore differ, so treat all of them as estimates that depend on GPU prices.

The catches

  • The model is small. The README compares talking to it with talking to a kindergartener, and it can hallucinate.
  • You need rented GPU time. Less than 80GB of GPU memory means tuning, and a single GPU takes about eight times longer.
  • The author has not tried every hardware path, so expect sharp edges. CPU runs give weak results.
  • It is an MIT-licensed research and learning tool, not a product.

What we did not test

We did not train a model or run any scripts. Each claim is tied to a quote from the project's README or the GitHub API, and the cost and time figures are the project's, not ours.

Verdict

Editors' view: worth attention from students, researchers and engineers who want to understand how a chat model is built, under the MIT licence. Anyone hoping for a ready-to-use assistant should look elsewhere.

Repository
karpathy/nanochat
Python · MIT
View on GitHub ↗

Sources

Need help adopting this, or have a software project in mind?

Tell us what you are building. The initial consultation is free. Email [email protected] or follow @zolnaltd.

Start a conversation →