Show HN: Run an 80B Qwen in 4.3 GB of RAM on a Mac, and a 35B on an iPhone
Scored daily by a customisable AI persona to surface the most relevant engineering leadership news.
Show HN running large Qwen models on consumer hardware, extremely relevant and actionable.
Swiftlet is a Swift+Metal runtime that runs Qwen3-Next-80B-A3B and Qwen3.6-35B-A3B MoE models on Apple devices by streaming expert weights from SSD via pread, keeping only the dense core (1.3-2.5GB) resident. At 4-bit quantization, peak RAM is 4.3GB for the 80B and 2.6GB for the 35B, with decode speeds of 4-5 tok/s on M5 Mac, using Gated DeltaNet linear attention to avoid growing KV cache. The open-source project includes an OpenAI-compatible server and runs on iPhone via Priv AI, with only ~3B active parameters per token limiting factual recall.
leonickson1