Love this comment. If anyone knows anything about machine learning or brains, this resembles modal limitations in learning.

A lot of our intelligence is shaped around our sensory experience, because we build tools for thinking via the tools we’ve already built, ever since baby motorbabbling to figure how our limbs work. Why Hellen Keller had such trouble learning, but once she got an interface she could engage with for communication, things took off.

We always use different tools, but some people don’t see colour. This doesn’t mean they are stupid when they answer differently in describing a rainbow.

Also why llms struggle with visual/physical concepts if the logic requires information that doesn’t translate through text well. Etc.

Point being, on top of how shitty memorization is as the be all end all, learning and properly framing issues will have similar blindspots like not recognizing the anvil cloud.

This is also why people in informational bubbles can confirm their own model from ‘learning’ over people’s lives experiences.

Like most issues, it doesn’t mean throwing the baby out with the bathwater, but epistemic humility is important, and it is important not to ignore the possibility of blindspots, even when confidence is high.

Always in context of the robustness of the framing around it, with the same rules applied at that level. Why “nothing about us without us” is important.

But also we gotta stop people giving high confidence to high dissonance problems, and socializing it into law. We should be past the “mmr causes autism” debate by now, but I’m hearing it from the head of health in the USA.