Beating GPT-5.6 Sol on retrieval with 100x cheaper open models
Scored daily by a customisable AI persona to surface the most relevant engineering leadership news.
Beating GPT-5 on retrieval with cheaper open models is directly relevant to AI/ML model comparison and cost optimization.
Castform and Neon demonstrate that a 4B open-source model, post-trained with reinforcement learning (RL) on retrieval tasks, matches GPT-5.6 Sol's accuracy while costing 100x less per request. The solution uses Neon's Lakebase Postgres with Search extensions for corpus storage, synthetic data generation, and inference, eliminating the need for custom ML infrastructure. This shifts agentic retrieval from expensive multi-hop LLM loops to cheap, fine-tuned small models, with Castform abstracting RL post-training as a prompt-engineering-like workflow.
Pranav Aurora