[GitHub Trending] JustVugg/colibri

7.7 relevance

Runs frontier MoE models on consumer hardware with pure C and zero deps — novel approach to local model serving.

AI/ML github.com

Answer the board on AI readiness

Arrive with evidence, not opinion. See how your foundation benchmarks against your peers, get a clear read, and leave with a ranked place to start.

Why It Matters

For a solutions architect focused on local AI inference and edge deployments, Colibri enables running state-of-the-art MoE models on existing laptops or servers without GPU investment, opening up private, low-latency LLM serving for dev environments and CI/CD pipelines.

Key Takeaways

Evaluate Colibri for running 100B+ MoE models on CPU-only machines by leveraging disk-swapped experts, starting with the GLM-5.2 744B model on a 16GB RAM workstation.

Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦 - JustVugg/colibri

Summary

Colibri is a free, open-source C engine that runs massive mixture-of-experts models (up to 744B parameters) on consumer hardware by keeping only the active expert in RAM and loading others from disk on demand. It supports 13 model families including DeepSeek V4, Qwen3.6, and GLM-5.2, requires no GPU (optional Vulkan/CUDA), and needs just 8GB RAM and 22GB disk. The setup auto-detects hardware, recommends a fitting model, builds or downloads the engine, and provides an OpenAI-compatible API endpoint at localhost:8000/v1.