Comment on Google's Agentic AI wipes user's entire HDD without permission in catastrophic failure
87Six@lemmy.zip 11 hours ago
Kinda wrong to say “without permission”. The user can choose whether the AI can run commands on its own or ask first.
Still, REALLY BAD, but the title doesn’t need to make it worse. It’s already horrible.
A big problem in computer security these days is all-or-nothing security: either you can’t do anything, or you can do everything.
I have no interest in agentic AI, but if I did, I would want it to have very clearly specified permission to certain folders, processes and APIs. So maybe it could wipe the project directory (which would have backup of course), but not a complete harddisk.
And honestly, I want that level of granularity for everything.
The user can choose whether the AI can run commands on its own or ask first.
That implies the user understands every single code with every single parameters. That’s impossible even for experience programmers, here is an example :
rm *filename
versus
rm * filename
where a single character makes the entire difference between deleting all files ending up with filename rather than all files in the current directory and also the file named filename.
Of course here you will spot it because you’ve been primed for it. In a normal workflow then it’s totally difference.
Also IMHO more importantly if you watch the video ~7min the clarified the expected the “agent” to stick to the project directory, not to be able to go “out” of it.
That implies the user understands every single code with every single parameters. That’s impossible even for experience programmers
I wouldn’t say impossible but I would say it completely defeats the purpose of these agentic AIs
Either I know and understand these commands so well I can safely evaluate them, therefore I really do not need the AI… or, I don’t really know them well and therefore I shouldn’t use the AI
Yep. That’s exactly why I tend to never discuss “AI” with people who don’t have to actually have a PhD in the domain, or at least a degree in CS. It’s nothing against them specifically, it’s only that they are dangerously repeating what they heard during marketing presentations with no ability to criticize it and, in such cases, it can be quite dangerous.
TL;DR: people who could benefit from it don’t need it, people who would shouldn’t.
100% agree with that tldr
This is EXACTLY the YouTube woodworkers dilemma…
TONs of YT channels to show people how to do woodwork would normally showcase $50K worth of equipment to show how to make a cutting board.
The thing is, people with access to such equipment, already know how to make a cutting board and are learning nothing from you… on the other hand, newbies who what to know what is this “sanding” thing they have heard, will not benefit from the vid since they do not have those tools, they’d have crappy manual tools at most.
Therefore, those videos are completely useless for learning… at best, they made for good background noise while people eat their lunches in their cubicles
Jhex@lemmy.world 8 hours ago
hmmm when I let a plumber into my house to fix my leaky tub, I didn’t imply he had permission to sleep with my wife who also lives in the house I let the plumber into
The difference you try to make is precisely what these agentic AIs should know to respect… which they won’t because they are not actually aware of what they are doing… they are like a dog that “does math” simply by barking until the master signals them to stop
Hawanja@lemmy.world 31 minutes ago
I mean, it’s not even that. Your dog at least can learn and has limited reasoning capabilities. Your dog will know when it fucks up. AI doesn’t do any of that because it’s not really “intelligent.”
87Six@lemmy.zip 8 hours ago
I agree with you, but still, the AI doesn’t do this by default which is a shitty defense, but it’s fact
Jhex@lemmy.world 7 hours ago
Absolutely… this just illustrates that these AI tools are, at best, some assistance that need to be kept on a very short leash… which can only be properly done by people who already know how to do the work the AI is supposed to assist with.
But that is NOT what the AI bubblers are peddling
PmMeFrogMemes@lemmy.world 7 hours ago
in your example tho it would be like the plumber asked you specifically if he could bone, and you were like “sure dawg sounds good”
Jhex@lemmy.world 7 hours ago
No, not at all
I get what you are saying but any reasonable entity would understand that telling someone at the door “come in”, does not mean “come in my wife’s ass”
Specifically the “without permission” in the title, relates to the fact the AI did not ask about it… it simply took a previously granted right to run commands and ran any/all commands without warning.
If you and I were working on a project together and nothing is working right, I could say “hmm let’s start over” and you would know it means “let’s start the project from scratch”, not “let’s wipe the data centre”
PumaStoleMyBluff@lemmy.world 4 hours ago
Inviting an agentic AI isn’t really asking them to do one task, though.
It’s more like offering a plumber a room in your house to stay in 24/7 so they can be on-call when you need them. And telling them they can use your food, dishes, clothes, and living room while they’re there and you’re at work.
Which makes it much less surprising when they get bored and bone your wife.
wooffersyt@lemmings.world 8 hours ago
🥱