#testing
3 posts
In a previous post I listed eleven ways an agent eval can lie, and one of them was trusting an uncalibrated LLM judge. This post covers what can go wrong with LLM-as-a-judge…
A SQL agent is asked the simplest question finance can ask: what was our net revenue last month? It answers 320.00. The right answer is 210.00. Nothing errors, the query looks…
That Test isn't Flaky. It's Broken. - In the era of AI agents, basic software engineering matters more, not less. Will Keleher argues for zero tolerance for intermittently failing…