Skip to content

Four Signals

Agentic insights for modern tech teams

AI/ML / thenewstack.io

Claude Opus 5 scored 30% on ARC-AGI-3. Wrapped in Nvidia’s AVO, it hit 100%.

Nvidia's Agentic Variation Operators (AVO) system elevated Claude Opus 5 from a 30.2% baseline on the ARC-AGI-3 benchmark to a perfect 100% RHAE score across all 25 environments and 183 levels. The AVO agent replaces the predefined variation step of evolutionary search with an autonomous decision-making process that inspects, edits, tests, and commits code while maintaining state across long-horizon tasks. This result demonstrates that system architecture—not model capability alone—can unlock frontier-level performance, as the same underlying computational pattern applies to both GPU-kernel optimization and abstract reasoning benchmarks.

Why it matters

For engineers building autonomous agent systems, this validates that investing in agent orchestration infrastructure (state management, context window handling, iterative self-correction) can yield dramatic performance gains beyond what model improvements alone provide.

AI Code Review at Scale: LinkedIn's Multi-Agent Approach
AI/ML / infoq.com

AI Code Review at Scale: LinkedIn's Multi-Agent Approach

LinkedIn engineers built a multi-agent AI code review platform to overcome the limitations of single-model reviewers, which suffer from blind spots, insufficient customization, and lack of operational control. The platform uses multiple independent AI agents with distinct models and reasoning approaches, cross-validating findings to boost signal and reduce hallucinations. Deployed on Kubernetes with an event-driven pipeline, it achieved a 63.9% overall suggestion acceptance rate across 5,230 sampled comments, with 80% acceptance for logic errors and 100% for concurrency bugs.

AWS Releases Aws-Bench to Evaluate Agents on Cloud Tasks
AI/ML / infoq.com

AWS Releases Aws-Bench to Evaluate Agents on Cloud Tasks

AWS released aws-bench, an open-source benchmark that evaluates AI agents on real cloud tasks like diagnosing misconfigurations and provisioning infrastructure using disposable AWS accounts. Built on the Harbor framework, it deploys CDK-defined scenarios in isolated accounts, scores agents via LLM judges or programmatic checks, and supports agents like Claude Code and Gemini CLI. The launch lacks baseline metrics or a leaderboard, and its reliance on LLM judges raises concerns about benchmark gaming, as highlighted by UC Berkeley researchers who demonstrated near-perfect scores on other benchmarks without solving tasks.

Cloudflare Cuts Astro GitHub Issues by 85% with AI Agents
AI/ML / infoq.com

Cloudflare Cuts Astro GitHub Issues by 85% with AI Agents

Cloudflare reduced Astro's open GitHub issues from over 200 to roughly 30 (an 85% drop) by deploying isolated AI agents within GitHub Actions that reproduce bugs, diagnose root causes, verify fixes, and generate preview releases for reporter validation. The workflow is orchestrated as a state machine driven by issue labels, with each stage—reproduction, diagnosis, verification, and fix—running as a separate subagent that passes context via a report.md file. Failed agent runs are treated as codebase maintainability signals; for example, adding a descriptive code comment prevented an agent from repeatedly introducing regressions in a Hot Module Replacement case. The system has been open-sourced as triagebot-action and its orchestration model evolved into Flue, a declarative framework for durable agent workflows with append-only event logs for resume capability.

Spline rebuilt its entire 3D editor. Then it handed the keys to Claude Code.
AI/ML / thenewstack.io

Spline rebuilt its entire 3D editor. Then it handed the keys to Claude Code.

Spline V2 is a complete rebuild of its 3D editor that ships a bundled MCP server, letting Claude Code, Cursor, Codex, and other coding agents directly manipulate live, editable scenes. The agent sees the current scene state before making changes, routes 3D edits through the editor and interface changes through Spline's Hana framework, and all modifications remain fully editable in the project's undo history. Spline's own AI agent follows the same MCP-based approach, treating each change as a normal editor operation that syncs across collaborators.

Most coding agent benchmarks skip large-scale refactoring. Not this one.
AI/ML / thenewstack.io

Most coding agent benchmarks skip large-scale refactoring. Not this one.

A new benchmark, SWE-Bench ProMax, targets large-scale code refactoring across seven languages (Python, Java, TypeScript, Go, C, C++, Rust) with 170 curated instances from real commits, finding the best AI agent achieves only a 41.2% resolve rate. Researchers at Shanghai Jiao Tong University, Peking University, and Douyin Group designed it to address quality flaws in existing benchmarks like SWE-Bench Verified, where nearly 60% of unsolved instances had flawed tests. Experts note LLMs lack the structural understanding needed for zero-tolerance refactoring, as token proximity does not guarantee comprehension of large codebases.

General / matduggan.com

OTel isn’t going well

OpenTelemetry's core maintainers are stuck in a three-way deadlock: a binary stability gate, a tiny maintainer bench, and a massive scope across dozens of languages and frameworks. This creates perverse incentives to endlessly debate features before marking them stable, while vendor SDKs remain idiot-proof and far ahead in polish. The result is a project that works magically for auto-instrumentation but drops teams off a steep cliff when they need manual instrumentation, especially for languages outside Go and .NET.

Pi Agent vs OpenCode after 100+ Hours of Real Use ✌️
AI/ML / dev.to

Pi Agent vs OpenCode after 100+ Hours of Real Use ✌️

Pi Agent and OpenCode are the leading open-source coding agents, each with distinct philosophies. Pi is a minimal, hackable platform (4 tools, <1k token prompt) that excels in efficiency and raw task performance (21/30 tasks at $0.078 each), while OpenCode is a feature-rich product (plan mode, MCP, subagents) with broader surface support and lower median latency (280.6s). The comparison reveals a dead tie on features, with Pi winning on cost and results, and OpenCode winning on out-of-the-box usability and provider support.

Grok, Claude, and Hermes agents get job titles — and persistent permissions
AI/ML / thenewstack.io

Grok, Claude, and Hermes agents get job titles — and persistent permissions

xAI's Grok Bot, Anthropic's Claude Tag, and Nous Research's Hermes are converging on a model where persistent agent identities—with memory, permissions, and job titles—replace ephemeral chat sessions as the unit of reuse. Hermes open-sources this via Agent profiles (config, keys, SOUL file, cron state), Grok Bot uses a shared account-scoped computer, and Claude Tag provisions a service identity in Slack. The critical insight: persistence is currently a configuration and state-separation concern, not a security boundary, and none of the vendors have demonstrated long-term agent coherence.

I Told My LLM Critic to Be Adversarial. It Started Blocking Plans for Being 'Not Thorough Enough.'
AI/ML / dev.to

I Told My LLM Critic to Be Adversarial. It Started Blocking Plans for Being 'Not Thorough Enough.'

A developer building an open-source PlannerCritic engine discovered that prompting an LLM critic to be 'adversarial' caused it to block plans for completeness concerns rather than concrete safety defects, escalating 16 strict goals incorrectly. The fix combined a prompt change with explicit severity rules and a deterministic code guardrail using a frozenset of allowed blocker families (unsafe_sequencing, weak_rollback, unverified_dependencies, feasibility), which downgraded any blocker outside those families to a warning. After the fix, zero advisory findings appeared as blockers across 92 runs, with all 132 blockers tied to real structural defects, proving that prompt engineering alone was insufficient—LLMs lack a stable concept of severity, so code-level enforcement is necessary.