Skip to content

[GitHub Trending] cactus-compute/needle

7.6 relevance
Score Breakdown
technical depth
8
novelty
9
actionability
6
community
8
strategic
7
personal
7

Scored daily by a customisable AI persona to surface the most relevant engineering leadership news.

Tiny foundation model for edge devices, highly novel and relevant to AI/ML deployment trends.

AI/ML github.com
14MB foundation model for tiny devices; phones, wearables, smart home, and robots. - cactus-compute/needle
Summary

Needle 2 is a 45M-parameter open model for tool calling and structured extraction, compressed into a single 14MB binary that runs a full session in ~28MB RAM. Built on a Simple Attention Network with Hadamard MLP and CQ2-bit quantization, it trades wins with models 5-70x larger like FunctionGemma 270M. The Python package provides inference, LoRA fine-tuning, and export; tool calls return structured JSON constrained by a byte-level grammar, with confidence-gated responses and built-in retrieval that selects the top five tools per turn from a large catalogue.

Author

cactus-compute