[GitHub Trending] cactus-compute/needle
Scored daily by a customisable AI persona to surface the most relevant engineering leadership news.
Tiny foundation model for edge devices, novel but not directly in reader's wheelhouse.
Needle 2 is a 45M-parameter open-source model for tool calling and structured extraction, compressed into a single 14MB binary that runs in ~28MB of RAM. Built on the Simple Attention Network architecture with Hadamard MLP, GQA attention, and engram key-value memory, it uses CQ2-bit quantization and a byte-level grammar compiled from user schemas to constrain token generation. The model supports confidence-gated responses, tool retrieval from large catalogs, and bounded memory via a 256-token sliding window with KV sinks, enabling deterministic JSON output without network calls after initial download.
cactus-compute