Just some thoughts:

• Current LLMs (chat AIs) are “frozen brains.” (Over-)Simplified, the synapses on the AI’s input neurons are given the 2048 prior words (the “context”) and the AI’s output synapses mean a different word each, so the synapse that lights up most strongly is the next word the AI will say. Then the picked word is added to the “context” and the neural network is executed once more for the next next word.

• Coming up with the weights of the synapses takes insane effort (run millions of books through the “context” and look if the AI t predicts the next word correctly, if not, change a random synapse). Afaik, GPT-4 was trained on more than 2000 NVidia A100 GPUs for somewhere around 4 to 7 months, I think they mentioned paying for 7.5 Megawatt hours.

• If you had a super computer that could keep running the AI with live training, the AI’s ability to string up words would likely, and quickly, degrade into incoherence because it would just ingest and repeat whatever went into it. Existing biological brains have these complex mechanisms of distilling experiences and evaluating them in terms of usefulness/success of their own actions.

I think that foundation, that part that makes biological brains put the action/consequence in the foreground of the learning experience, rather than just ingesting, is what eludes us. Perhaps at some future point in time, we could take the initial brain structure that grows in a human as the seed for an AI (but I guess then we’d likely have to simulate all the highly complex traits of real neurons, including mixed chemical and electrical signaling and possibly even quantum-level effects that have been theorized).