Key takeaways

  • Run very large open AI models on the computer you already own: colibri keeps the active part in RAM and reads the rest from disk.
  • No GPU needed: every engine runs on the CPU, and any GPU with a Vulkan driver can speed things up.
  • A setup step looks at your machine and recommends a model that fits it.
  • The project's own figures for its 744B-parameter model: 0.05-0.1 tok/s cold on a 25 GB laptop, 1.83 on a 128 GB Ryzen AI Max+ 395.
  • The catches: on big models the disk sets the speed, a small GPU can slow small models, and some models carry their own licences.

What it is

colibri is a free, open-source engine written in pure C, with the tagline "tiny engine, immense model". It runs very large AI models on ordinary machines, and the code is on GitHub at JustVugg/colibri under the Apache licence.

The problem it solves

People who want to run big open models locally are usually stopped by memory. The README points out that these mixture-of-experts models use only a small part of themselves for each token. Its example is a model with 744B parameters, about 40B used for each token, and only about 11 GB of those changing from one token to the next.

So colibri keeps the part that is always needed in RAM and reads the rest, the experts, from disk when the model asks for them.

What you get

  • No special hardware: every engine runs on the CPU with nothing else installed.
  • Optional speed: any GPU with a Vulkan driver can be used.
  • Less guesswork: the setup recommends a model that fits your machine.
  • Choice: thirteen engines run today, covering many model families.
  • A small codebase: one file per model family, in pure C.

How it compares

These are the project's own measurements, not ours. For that 744B-parameter model the README lists 0.05-0.1 tok/s cold on a 25 GB laptop, against 1.83 on a 128 GB Ryzen AI Max+ 395. That gap shows the trade: for the largest models the disk sets the speed.

The catches

  • Speed: on the largest models the disk sets the speed, and a cold laptop run is a fraction of a token per second.
  • GPUs are not automatic wins: a small model whose experts already sit in RAM can lose speed on an integrated GPU.
  • Exactness: on a GPU a long answer can drift from the CPU's by a word.
  • Gaps: some table entries are blank because nobody has measured them yet, and one GPU path has not been measured on the cards it is meant for.
  • Licences: the engine is Apache-licensed, but individual models have their own terms, and one image model is non-commercial.

What we did not test

We did not install colibri, download any model or time any run. Every claim here comes from a literal quote in the project's README or the GitHub API, and the speed figures are the project's own.

Verdict

Editors' view: this suits tinkerers and researchers who want to try very large open models on hardware they already have, and who accept slow answers in return. If you need fast, dependable responses from the biggest models, a machine with plenty of memory or a server is still the safer choice.

Repository
JustVugg/colibri
C · Apache-2.0
View on GitHub ↗

Sources

Need help adopting this, or have a software project in mind?

Tell us what you are building. The initial consultation is free. Email [email protected] or follow @zolnaltd.

Start a conversation →