Can an AI-generated audit fairly evaluate academic economics research?
An AI-generated 'audit' of economists Emmanuel Saez and Gabriel Zucman's research on wealth and taxation, published by Tyler Cowen, circulated days before a vote on a proposed California billionaire wealth tax, sparking debate over whether AI audits of research are reliable.
Side A
AI audits can surface real flaws
This camp argues that AI audits of research have real value. Crémieux says the audit contains some errors but is still substantially correct. Bryan Caplan says he has not reviewed this particular audit, but a Claude audit he did review mostly held up, and that dismissing AI audits without reading them is a mistake. Steve Hsu says frontier models are much better than human reviewers and that he trusts the audit's methodology more than journal review.
Evidence posts
Overlap
Where they overlap
- Both sides accept that research can be questioned and improved, and that some specific criticisms may deserve examination.
Side B
The AI audit is unreliable
This camp argues the audit does not fairly evaluate the research. Gabriel Zucman says the AI-generated audit has no merit, that an AI prompted to find errors will oblige, and that it labels methodological choices discussed in the literature as flaws. Economist Antoine Levy, who says his views are opposed to Saez and Zucman's, says the audit seems mostly wrong on papers he knows well. Paul Novosad says AI results shared without prompts and with an agenda should not be trusted at face value.
Who took this side2
Evidence posts
Reader vote
Be the first to weigh inWhere do you land? Anonymous, one vote per reader. Sign in with X to be counted as a verified reader.
Free newsletter
Both Sides Daily
One email a day with the day's debates: what each side says, where they meet, and how readers voted. No spam, unsubscribe with one click.
This page was compiled by AI and published after automated checks. Something wrong?Report a correction