Nice post. A while back I read something on reddit about a theory for technological advances always being used for the worst possible nightmare scenario. But I can’t find it now. Fundamentally I’m a technological optimist but I can’t even yet fully imagine the subtle systemic issues this will cause. Except the rather obvious one:

Algorithms on social media decide what people see and that shapes their perception of the world. It is relatively easy to manipulate in subtle ways. If AI can learn to categorize the psychology of users and then “blindly anticipate” what and how they will respond to stimuli (memes / news / framing) then that will in theory allow for a total control by means of psychological manipulation. I’m not sure how close we are to this, the counter argument would be that AI or LLMs currently don’t understand at all what is going on, but public relations / advertising / propaganda works on emotional levels and doesn’t “have to make sense”. The emotional logic is much easier to categorize and generate. So even if there is no blatant evil master plan just optimizing for max profit, max engagement, could make the AI pursue (dumbly) a broad strategy that is more evil than that.