Key takeaways
- A machine learning library in plain C and C++ with no dependencies, so there is little to set up around it.
- It targets many processor families and backends, including CPU, GPU, NPU and the browser.
- Low-bit integer quantisation formats are built in, which is how large models are made smaller.
- The README promises zero memory allocations at runtime, a claim we could not test.
- The catches: it is a library, not an app, and the README is short on detail.
What it is
ggml describes itself as a tensor library for machine learning. It is a plain C and C++ implementation without any dependencies, released under the MIT licence, and the code is on GitHub at ggml-org/ggml.
The problem it solves
Machine learning software has to run on very different hardware, and heavy setup is a barrier for the developers who build it. The project's stated goal is to be simple, portable and efficient, with minimal setup.
It is aimed at developers who write or embed machine learning code, not at people looking for a finished application.
What you get
- Less to install: the README says it has no dependencies.
- Reach: it lists x86, ARM, RISC-V, LoongArch, PowerPC, s390x and WebAssembly as supported platforms.
- Choice of hardware: backends cover CPU, GPU, NPU and the browser.
- Smaller models: it supports 2- to 8-bit integer quantisation, plus MXFP4 and NVFP4 microscaling formats.
- Predictable memory: the README says it makes zero memory allocations during runtime, and it links documentation for the GGUF file format and a commented matrix multiplication example.
The catches
- It is a library, so you need to write code against it. There is no ready-made interface.
- The README is short. It gives no benchmarks, no comparisons and no statement about maturity or stability.
- Contributions to the core library are routed through the llama.cpp repository, so the project's centre of activity may not be where you expect.
- Performance, hardware support and memory behaviour are all the project's own description.
What we did not test
We did not build or run ggml, try any backend, measure speed or memory, or check the quantisation formats. Every claim here comes from a literal quote in the project's README or the GitHub API, and the feature descriptions are the project's own. We have not reviewed the code or its security.
Verdict
Editors' view: this is for developers who build or embed machine learning software and want a small, portable foundation. If you only want to use models through an existing app, you will not need to touch it directly. If you are evaluating it, begin with the commented example and check the platform you care about first.
Sources
Need help adopting this, or have a software project in mind?
Tell us what you are building. The initial consultation is free. Email [email protected] or follow @zolnaltd.
Start a conversation →