Most European languages are some almagation and standardization of a patchwork of dialects. English destroyed scots, Tours french destroyed occitan, hochdeutsch largely displaced bavarian, etc. The reason is the dramatic increase in state power in the 19th century (therefore giving importance to the language of administration), combined with policies of “cultural unification” and nation building. So there in fact very much is artificiality in language that is predictable.

The reason Esperanto wasn’t adopted by states when it was invented is because internationalism was seen as either an utopia or a threat by states. The reason it isn’t adopted by the UE now is because it would be a highly visible and unpopular move that would take decades to bear fruit and therefore politicians will never support it. The entire UE therefore learns english, massively increasing the influence of people who brand themselves its enemies.