colibrì is a pure-C runtime that runs GLM-5.2, a 744-billion-parameter mixture-of-experts model, on a consumer machine with about 25 GB of RAM. It keeps the model's dense core in memory and streams the experts from disk on demand. No GPU required, no Python at runtime, zero dependencies. Here's the honest, plain-English read before you clear 400 GB and try it.
A 744B mixture-of-experts model only activates ~40B parameters per token. colibrì keeps the ~17B dense core resident in RAM at int4 (~9.9 GB) and leaves the 21,504 routed experts (~370 GB) on disk, reading only the handful each token needs.
The whole engine is one ~2,400-line C file plus small headers. No BLAS, no Python at runtime, no GPU required. Python is used once, offline, only to convert the model to its int4 format.
This is the honest headline: a frontier-class model answering correctly on a box that costs less than one H100 fan. On a plain NVMe machine it is fractions of a token per second cold. Warm cache, pinned experts and speculation speed it up a lot.
Native builds for all three (no WSL needed on Windows), with optional experimental Metal and CUDA tiers for people who do have GPUs. The default, headline path is plain CPU plus disk.
Nothing is metered and there is no account. The real cost is a big fast SSD, patience, and electricity. A warm cache, pinned hot experts and the int8 speculation head push useful-response latency down a lot, but the speed of your disk sets the floor.
About 400 GB free on a real NVMe path, plus ~25 GB of RAM. A slow or nearly-full drive is the thing that will make this unusable, not the model itself.
cd c && ./setup.sh checks your compiler, builds, and self-tests. There are no dependencies to install for the engine itself.
Grab the Hugging Face build with the int8 MTP heads (matey-0 clone). This skips the multi-hour FP8-to-int4 conversion entirely.
The single most common gotcha: an int4 head gives 0% draft acceptance and speculation silently never turns on. Check the out-mtp-* file sizes against the README before you start.
RAM budget, expert cache and speculation are all detected automatically. Run ./coli doctor first for a read-only readiness check that tells you if the placement is runnable.
cd c && ./setup.sh # build + self-test the engine
# download the int8-MTP model, then point COLI_MODEL at it:
COLI_MODEL=/nvme/glm52_i4 ./coli doctor # read-only readiness check
COLI_MODEL=/nvme/glm52_i4 ./coli chat # goCold decode reads ~11 GB from disk per token. On a 1 GB/s drive that is fractions of a token per second. This is a "runs at all" feat, not interactive chat, until the cache warms.
The int8 MTP head version, or speculative decoding sits at 0% and you lose the roughly 2x speedup lever. Easy to get wrong, easy to check.
Speculation and the GPU or batched paths round slightly differently, so greedy output can differ token-for-token between runs. Set DRAFT=0 (and IDOT=0 COLI_CUDA=0) if you need exact reproducibility.
Streaming is read-only, so it will not wear the SSD. The real risks are swap traffic if you starve it of RAM, and sustained thermals on cheaper drives during hours of full-read duty.
colibrì's README is unusually candid about its own limits, with community-measured numbers and linked issues behind every claim. Star it, read the design notes, and check the Hugging Face model link before you start.