RAG serves as a knowledge layer.

What they really lack right now is effective introspection and executive function.

Too many people are trying to build a single model to do things correctly rather than layering models to do things correctly, which more closely approximates how the brain works.

We are shocked when AI chooses to nuke people in a wargame, but conveniently gloss over the fact that nearly every human put in front of a giant red button saying “Launch nukes” is going to have an intrusive thought to push the button. This is part of how we have an exploratory search around choices and consequences and rely on a functioning prefrontal cortex to inhibit those thoughts after working through the consequences. We need to be layering generative models behind additional post-processing layers that take similar approaches of reflection and refinement. It’s just more expensive to do things that way, so cheap low effort things like chatbots still suck.