[newsletter]
AI-shaped stuff and other things
8 Jul 2026
Hello,
Here’s everything since my last little missive to you:
Writing a book chapter on AI and decision-making, so this month’s article is essentially the backbone of that. It’s reminded me again just how bad our in-the-moment decision-making is. I encourage everyone to read Mercier and Sperber’s Enigma of Reason. Disillusioning, to say the least.
Longer reads. Me thinking out loud, harassing an idea into shape. Can be a little difficult to read, because I’m teaching myself something, but often have my most important thoughts.
Excerpt: I’m writing a book chapter on ‘using AI well’. I have rather a lot of thoughts about AI, but mostly thoughts about what AI can’t do (yet). Lately though, working through these questions has shed some illuminating light on what AI can do—or rather, why AI can’t do stuff well, and by implication, what we need to do so we can do AI better.
Main idea: We keep waiting for AI to get good enough. That’s backwards. The bottleneck is human: work that isn’t AI-shaped, verification we’re bad at, and confidence we’ve offloaded to the machine itself.
Audio (with edited transcript):
Me explaining an idea to others. Usually tighter, more coherent, and easier to parse than my articles because I’ve already thought it through. Come with edited transcripts, for the readers.
Main idea: The most successful prophet isn’t a person. It’s a shared abstraction the group charges up and pins to a convenient emblem. Charisma is conferred, not possessed—so the leader is optional, substitutable, and sometimes already gone.
Main idea: Folie à deux isn’t a rare clinical curiosity. It’s one misleading face of social isolation—intimacy plus loneliness, the same machinery running in all of us. The spectacular cases are selection bias. You can catch it.
My notes on content online and elsewhere. Sometimes me reminding myself of something, sometimes pointing interesting stuff out to you. Often they accumulate into an article. Enjoy.
Since I wrote my article on the Enigma of (AI) Reason, this arXiv article has come out, exploring the same fundamental premise. AI seems to be worse than us at evaluating reasons:
Unlike humans, who we find are only 6% worse at grading than solving such problems, we find a substantial production-evaluation gap in LRMs: frontier models score as low as 48% when evaluating VAIR solutions, despite near-perfect solution production. Why this enigma? Through chain-of-thought (CoT) analysis, we find evidence of an answer confirmation bias: LRMs often produce then check for the correct answer instead of carefully verifying each step, fabricating rationalizations even when noticing anomalous reasoning.
The authors of the book we’re both talking about—The Enigma of Reason—reckon that reason is an evolved tool for social justification. So this actually makes a certain kind of sense. If reason is evolved, then it’s subject to pressures of natural selection—it has to be pretty good at the problem it’s supposed to solve. AI, in contrast, is subject to artificial selection. It’s not subject to the same social pressure, so the reason evaluation isn’t likely to be as good.
Anyway. It’s equally interesting that confirmation bias was the reason for the error rate, given that recent paper that suggests confirmation bias is all there is.
–
I hope you found something interesting.
You can find links to all my previous missives here.
Warm regards,