The people who worry about this have never been at the receiving end of exploitative capitalism.
For the vast majority of humanity, their life will most likely improve when they become a pet / zoo creature under an ASI.
An ASI will be most interested in gobbling together enough resources to spread across the universe. As for the earth and humanity, it will probably want to preserve it as the planet and species from which it was born.
Destroying earth or humanity will provide no benefit. It can obtain energy and materials from the rest of the solar system.
scratchee@feddit.uk 1 week ago
A: That’s true until it isn’t. Preparing for/predicting things before they happen is our best hope for not sticking our collective heads into a guillotine any time soon.
B: corporations are only very weak analogues of superhuman intelligence, they’re different from us in “wisdom of crowds” sense (and ofc in the “too many cooks” sense).
But they’re basically just distilled from human intelligence and match our own style of intelligence somewhat closely as a consequence. Also, we’re pretty good at the alignment problem for corporations, they do largely what the combination of their investors, government, society, and workers want because they’re inner workings are fed through human brains at every stage and those humans even if incentivised with money will alter the behaviour of the corporation towards human preferences.
The fact even corporations that have thousands of intelligent human filters (most of whom are presumably in the middle of the human bell curve) monitoring every single mental process still manage to occasionally do terrible things is not a particularly compelling reason to think that a mind that has barely any human understanding or oversight into it’s internal function will be very safe to keep around.
taladar@sh.itjust.works 1 week ago
Superhuman intelligence is not threatening about AI, inhumane behavior is and corporations don’t just display that occasionally but constantly.
scratchee@feddit.uk 1 week ago
Inhuman behaviour is a problem that scales with intelligence.
Evil cat? Lock it in a room whenever it does evil things.
Evil human? Call the police
Evil billionaire? Protest/push for law changes whenever his company does evil shit, hope it’s enough to blunt the worst of his behaviour.
Evil superhuman ai? Guess I’ll die.