The Channel Gap: Why Your LLM Judge is Blind in One Eye
7.7 relevance
Score Breakdown
technical depth 9
novelty 8
actionability 7
community 5
strategic 6
personal 9
Scored daily by a customisable AI persona to surface the most relevant engineering leadership news.
LLM judge blind spots vs deterministic checks, novel and relevant to AI evaluation.
Summary
LLM-based evaluators suffer from the Data Processing Inequality: they share the same text channel as the producer, so any deviation not present in the output text is invisible to the judge. The open-source skillgate package (TypeScript, two deps) solves this by checking the filesystem deterministically — file existence, regex matches, command exit codes — instead of reading model output. Its hard deployment tier uses a server-side pre-receive hook that blocks git pushes if gates fail, placing enforcement outside the agent's control flow entirely.