[marginalium]
Dunning-Kruger is about noise
3 Jul 2025
The Dunning-Kruger effect is the idea that:
People base their perceptions of performance, in part, on their preconceived notions about their skills. Because these notions often do not correlate with objective performance, they can lead people to make judgments about their performance that have little to do with actual accomplishment.
It has had trouble surviving the replication crisis. This is basically because of noise. People who are bad at things will score badly, and so will people who have bad luck. People who are good at things will score well, and so will people with good luck. If they rate their ability to perform, unlucky people will over-estimate and lucky people will under-estimate.
Most damningly, if you do random-number simulations, you will get a similar pattern.
Except this isn’t actually damning, it’s just confusing. If you sort high and low scorers, lucky and unlucky scorers, and random data into a line from high-to-low, you will get a line from high-to-low. It would be weird if you didn’t.
It would also be really if people who were bad at things didn’t over-estimate their performance. Probably, it’s that experts are just more ‘lucky’—they make fewer errors, and poor performers are less ‘lucky’—they make more errors.
Anyway, people are fighting. but what’s interesting to me is that it seems to imply, it may not so much be ignorance that makes us overconfident as the contextual noise. An error in conclusion I’ve made myself
Anthologies: Betterment, On Being Fruitful, Bias vs Noise, Karstica