[marginalium]
AGI is far away
10 Jun 2025
AGI is far away. I only really skimmed this—it’s about the slow productivity gains from what seems to be enormous bursts of growth in AI capability.
Some of this is people hiding the fact that AI is doing their work for them. You could tell your boss that you finished all your work early because of AI and you need more work, or you could do other more fun things instead.
Some of this is because even though the AI can produce surprising results very quickly, it still takes human triage time to account for errors and hallucinations and whatnot, and so the time spent is just traded from the work, to supervising the work.
A lot of this is because a lot of knowledge-work requires context, tacit knowledge, or interpersonal judgment that you need to feed the models. So you either spend time feeding them, or you just do the stuff yourself.
The author says something very interesting about this last point:
the fundamental problem is that LLMs don’t get better over time the way a human would. The lack of continual learning is a huge huge problem. The LLM baseline at many tasks might be higher than an average human’s. But there’s no way to give a model high level feedback. You’re stuck with the abilities you get out of the box. You can keep messing around with the system prompt. In practice this just doesn’t produce anything even close to the kind of learning and improvement that human employees experience.
The reason humans are so useful is not mainly their raw intelligence. It’s their ability to build up context, interrogate their own failures, and pick up small improvements and efficiencies as they practice a task.
Feels truthy.
Anthologies: Gratification, Wealth Architecture, Digital Architecture, On Being Fruitful