#evals
2 posts
In a previous post I listed eleven ways an agent eval can lie, and one of them was trusting an uncalibrated LLM judge. This post covers what can go wrong with LLM-as-a-judge…
A SQL agent is asked the simplest question finance can ask: what was our net revenue last month? It answers 320.00. The right answer is 210.00. Nothing errors, the query looks…