Skip to content

[GitHub Trending] cactus-compute/needle

6.7 relevance
Score Breakdown
technical depth
7
novelty
8
actionability
6
community
6
strategic
6
personal
6

Scored daily by a customisable AI persona to surface the most relevant engineering leadership news.

Tiny foundation model for edge devices is novel and relevant to IoT/embedded trends.

AI/ML github.com
14MB foundation model for tiny devices; phones, wearables, smart home, and robots. - cactus-compute/needle
Summary

Cactus Compute released Needle 2, a 45M-parameter open model for tool calling and structured extraction that runs entirely in a 14MB binary using ~28MB of RAM. Built on Simple Attention Network research with Hadamard MLP, GQA attention, and engram key-value memory, it compresses weights to CQ2-bit using Cactus Quants and delivers confidence-gated JSON responses with a byte-level grammar compiled from user schemas. The Python package (pip install cactus-needle) offers inference, LoRA fine-tuning, and export, with a 256-token sliding window and built-in tool retrieval that selects the top five tools per turn, keeping memory constant regardless of conversation length.

Author

cactus-compute