The most cited field experiment on generative AI and knowledge work came out of an academic setting: Noy and Zhang gave professional writing tasks to hundreds of college-educated workers, half with access to ChatGPT.
What they found
| Measure | Result |
|---|---|
| Time to complete | About 40% less for the AI group |
| Output quality (blind ratings) | Higher for the AI group |
| Quality gap between workers | Compressed |
| Effect by initial skill | Larger relative gain for lower-rated writers |
The “compression” result matters: AI helped the weaker writers most, echoing the customer-support finding that gains concentrate among less experienced workers. The average number hides a distribution.
Caveats to carry into a business case
- The tasks were mid-level, well-specified writing, not open-ended strategy.
- Time savings did not automatically become cost savings; the study measured task time, not headcount.
- Quality was rated on the rubric of the study, not on long-run business outcomes such as customer retention.
- Relying on averages can overstate the gain for your stronger staff.
VERDICT
Expect real speed and quality gains on well-specified writing, concentrated among less experienced staff. Do not extrapolate the 40% into a company-wide number without measuring your own tasks.