Skip to main content

Posts

Showing posts from July, 2026

Don't Ship AI Agent Skills Without Evals

At the AI Engineer World's Fair on July 1, 2026, Philipp Schmid, a Staff Engineer on the Gemini API and agents team at Google DeepMind, asked a room full of engineers a simple question: who uses skills with their coding agents? Every hand went up. Who has evals for those skills? Almost none. That gap is the entire talk, which Schmid also wrote up on his own blog, philschmid.de/testing-skills . His team indexed the skills ecosystem through SkillsBench and found the same pattern everywhere: people ship a SKILL.md file after two manual test runs and move on. It looks fine in a demo. It quietly corrupts outputs in production, because bad skills don't crash. They just make the agent confidently wrong. If you've already read our breakdown of Matt Pocock's Claude Code skills library , think of this as the other half of that story: what happens once you've installed a skill and it's actually running against real prompts. Key Takeaways SkillsBench indexed 47,000+ un...

Why Domain-Specific AI Agents Beat One Big Agent

Everyone is building agents right now. Real estate firms. Independent insurance brokers. Fortune 500 companies with budgets big enough to hire an army of consultants. Ask around and you'll hear the same story everywhere: "we're building our own agent." And yet almost nobody is asking the obvious question: why does the default approach keep failing? One large, general-purpose agent gets wired up to every tool the business owns. It impresses in the demo. Then it quietly stalls before production. There's a gap between what businesses want and what they're actually getting. They want AI woven into their data, their workflows, their day-to-day operations. What they get instead is one oversized agent trying to be a sales rep, a compliance officer, and a customer support line, all at once. That gap is an architecture problem, not a model problem. Key Takeaways The default "one big agent" pattern breaks down on context bloat, cost, fragility, and portability...