5 articles.
Apple researchers rebuilt 100 grade-school math problems into templates and generated 5,000 variants, then ran 25 state-of-the-art models against them. The problems needed identical reasoning, yet accuracy swung by more than 12 points between easy and hard versions of the same question, and adding one sentence that changes nothing cost some models over 65% of their accuracy. Few-shot examples did not fix it.
MMLU and similar multiple-choice benchmarks no longer discriminate between strong models. Task-based, real-world evaluations like SWE-bench give a more honest picture of capability.
The LMSYS Chatbot Arena ranks models by human preference in blind pairwise comparisons. Open models have closed much of the gap, but proprietary models still lead on complex tasks. The metric measures preference, not accuracy.
The OWASP Top 10 for LLM applications lists the most common security risks when deploying AI features. Here is what each means in business terms and which to prioritise first.
Two 2026 results about AI-written tests look contradictory and are not. Given a test prefix and the methods called, models reach 43% mutation score against 45% for human-written oracles on a contamination-free dataset. Asked to write the whole test for a real project, the same models cover about a third of the lines. Here is where the pipeline breaks, and why coverage is the wrong metric to check.