Outside of law enforcement, this is certainly how shitty customer service policies get enforced. In other words, “Computer says no”.
"A computer can never be held accountable. Therefore computers are a valuable tool for law enforcement and the ruling class."
Submitted 23 hours ago by TheReturnOfPEB@reddthat.com to showerthoughts@lemmy.world
Comments
ch00f@lemmy.world 23 hours ago
Semi_Hemi_Demigod@lemmy.world 23 hours ago
The British Post office rolled out a hugely buggy piece of software that bankrupted small business owners, got some sentenced to years in prison, and caused thirteen people to commit suicide because “computers can’t be wrong”
ch00f@lemmy.world 22 hours ago
Oh yeah! I forgot about that!
dogfoodeater@lemmy.blahaj.zone 19 hours ago
Here in Australia they rolled out an automated system to calculate welfare overpayments and issue debts. It didn’t quite work of course and hundreds of thousands of the poorest Australian were issued with false debts, some of whom died or committed suicide before they could be repaid. People still keep floating the idea of automation and AI in our welfare systems…
Xaphanos@lemmy.world 22 hours ago
Not to be that guy but… Link?::: spoiler Title :::
Kyrgizion@lemmy.world 22 hours ago
Not only UK’s fuckup, also Fujitsu’s.
over_clox@lemmy.world 23 hours ago
Now where does this thought come from?
Do you not know what a computer is? It’s literally a digital logical accountant! Yeah yeah, we should probably blame the programmers and engineers instead when shit goes sideways, but now I think we need to also hold CEOs accountable when they decide to inject faulty AI into mission critical systems…
henfredemars@infosec.pub 23 hours ago
There’s a reason why license agreements often stay there there are no warranties express or implied, no guarantees, and no fitness for any particular purpose.
Semi_Hemi_Demigod@lemmy.world 23 hours ago
“This software is useless and should not be used by anyone for any purpose” is my favorite part of license agreements.
mogranja@lemmy.world 21 hours ago
If a building collapses. You blame the people who built the walls and poured the concrete, or the ones who chose the materials and approved the project?
In any case, often programmers and engineers retain no rights to the software they worked on. So whoever profits from the software should also shoulder the blame.
over_clox@lemmy.world 20 hours ago
Those in charge, that approve the continued use of proven faulty software, should take all the blame, after significant faults have been proven anyways.
I mean you have a point, but still, 1+2+3≠15, and a bag of Doritos is not a gun. When AI fucks up this badly, the real guilty parties
(my AI keyboard wanted to replace guilty with gullible BTW, and I’m using the FUTO keyboard no less),
the real guilty parties are the ones in charge that allow such proven faulty systems to continue running for mission critical systems.
Like fuck, a bag of Doritos is not a fucking gun!
Zorsith@lemmy.blahaj.zone 18 hours ago
IT disagrees. Misbehaving hardware can be taken out back and shot.
faythofdragons@slrpnk.net 18 hours ago
We all remember what happened to the printer in Office Space
popekingjoe@lemmy.world 17 hours ago
Damn it feels good to be a gangsta.
bryndos@fedia.io 9 hours ago
Law enforcement will seize and use computers and the data they hold as evidence to convict criminals, just like any other tool that they might be warranted to seize.
Courts will examine the evidence of what it did to determine what role it played in the offence and whether it supports the allegation.
Likewise police complaints authorities could do the same in principle against the police; if someone were to give them a warrant and the power to execute it.
If a thing happens in public that was unwarranted and can be traced back to a police force or how they deployed any equipment, they can be judicially reviewed* for any decision to deploy that bit of kit. It's more a matter of will they actually be JR'd and will that be review be just and timely. * - in my country.
I don't think it's much different from how they deploy other tech like clubs and pepper spray, tear gas, tazers or firearms. If they have no fear of acting outwith their authority , that's a problem.
In some ways it might be easier to have an 'our word' vs 'their word' defense when they shoot someone, compared to a computer program that might literally document the abuse of power in its code or log files.
"Oops i dropped my notebook", is maybe easier than, "oops i accidentally deleted my local file and then sent a request to IT - that was approved by my manager - asking them to delete instead of restore any onsite or offsite backups".
ChonkyOwlbear@lemmy.world 22 hours ago
That’s why cops love using dogs too. Courts have ruled that dogs can’t lie. That means if a dog indicates you have contraband, then a search is warranted, even if nothing is found. This of course ignores that it is entirely possible the dog indicated contraband because the cop trained it to do so on command.
affenlehrer@feddit.org 20 hours ago
You could hold developers of algorithms, logic and even symbolic AI accountable.
However, it’s a completely different story for AI based on deep neutral networks. After training they’re just a bunch of weights and parameters without individual meaning and it’s not a few, it’s billions or trillions of them. And almost none of them were individually set, they’re often randomly initialized and then automatically tuned by deep learning algorithms during training until the behavior / predictions of the neural net are “good enough”.
It’s practically impossible to review the network and when you test it you just get the result for the concrete test cases, you can’t interpolate or assume even slightly different cases will behave similarly. You also can’t fix an individual bug. You can just train again or more and this effort might fix the problem but it could also destroy something that worked before (catastrophic forgetting).
QuantumTickle@lemmy.zip 23 hours ago
We don’t jail the gun for murder.
PeriodicallyPedantic@lemmy.ca 21 hours ago
I feel like you’re missing the point.
They’re not saying to jail computers, they’re saying be ware of political leaders using computers to abdicate responsibility.snooggums@piefed.world 21 hours ago
We also don't give the murderer a free pass because they used a gun.
A tool is a tool, and the person who designed it or used it is responsible depending on why it caused a negative outcome.
kibiz0r@midwest.social 19 hours ago
We shut down companies for it though, and what AI vendors are doing is basically selling the ability to turn job roles into “accountability sinks”, where your true value is in taking the fall for AI when it gets it wrong (…enough that someone successfully sues).
If you want to put it in gun terms: The AI vendors are selling a gun that automatically shoots at some targets but not others. The targets it recommends are almost always profitable in the short term, but not always legal. You must hire a person to sit next to the gun and stop it from shooting illegal targets. It can shoot 1000 targets per minute.
JoeBigelow@lemmy.ca 19 hours ago
Sounds like a fun job if the acceptable failure rate is like, 50%
over_clox@lemmy.world 23 hours ago
The gun isn’t running software in the background when humans are away either. See my other comment, when shit goes sideways, blame the programmers, engineers, and now the CEOs that decided to jam screwy AI up our collective asses…
QuantumTickle@lemmy.zip 23 hours ago
We don’t jail gun manufacturers either.
When a tool is used to kill a human, the user of the tool is guilty.
mogranja@lemmy.world 21 hours ago
Blame the programmers? Yeah, no. The software is owned by the company, blame them.
individual@toast.ooo 20 hours ago
this is straight out of the book ‘Do Android Dream Of Electric Sheep’ later turned into the movie ‘iRobot’
infinitesunrise@slrpnk.net 19 hours ago
All technology has the potential to be both liberatory and oppressive, all that ever matters is who wields it and to what end.
Kyrgizion@lemmy.world 22 hours ago
This is unironically one of the main drivers of AI. As soon as all crucial social systems are inundated with AI, the built-in bias will be excused as “minor glitches” of the system, but the real reason was always a total lack of accountability.
Damage@slrpnk.net 22 hours ago
Yeah, not like people in power, who are held accountable all the time!
KingThrillgore@lemmy.ml 22 hours ago
“While computers are mechanical, the processes must be dictated and implemented by a human. Therefore, the only person culpable when a computer does something, is the human who wrote the instruction set, code, or algorithm.”
If we interpreted the problem with human culpability and consequences, a whole lot of bugs would stop disappearing from software right quick.
shalafi@lemmy.world 22 hours ago
If a mistake in my code exposes me to criminal investigation, I’m getting a new job.
JeSuisUnHombre@lemmy.zip 22 hours ago
Same argument for cars
stepan@lemmy.cafe 23 hours ago
the AI decided ¯_(ツ)_/¯
PillowTalk420@lemmy.world 22 hours ago
Excuse me, you dropped this: \
Cort@lemmy.world 22 hours ago
No, I’m sorry, the AI has decided you don’t need that forearm, but you’re welcome to keep the hand.