1 article.
The most-cited study of AI grading put 80 multi-turn questions to GPT-4 and 58 expert annotators. GPT-4 matched human preference 85% of the time, above the 81% two humans managed. But when researchers took an answer that was already correct, added a redundant restatement of it at the top, and re-judged, Claude and GPT-3.5 were fooled 91% of the time, every judge favoured whichever answer appeared first, and on maths a reference answer cut the judge's failure rate from 70% to 15% while plain step-by-step reasoning left 6 failures in 20.