One thing which actually scares me with AI ia we get one chance. And there are a bunch who don't think of repercussions, just profit/war/etc.

A group can be as careful as possible but it doesn't mean shit if their Smarter Than Human AI isn't also the first one out because as soon as it can improve itself, nothing is catching up