No ethical AI without explainable AI
How AI works is often a mystery — that's a problem
Submitted 10 months ago by boem@lemmy.world to technology@lemmy.world
https://www.nature.com/articles/d41586-023-04154-4
Comments
match@pawb.social 10 months ago
bh11235@infosec.pub 10 months ago
no ethical people without explainable people
logicbomb@lemmy.world 10 months ago
People are able to explain themselves, and some AI also can, with similar poor results.
I’m reminded of one of Azimov’s stories about a robot whose job was to aim an energy beam at a collector on Earth.
Upon talking to the robot, they realized that it was less of a job to the robot and more of a religion.
The inspector freaked out because this meant that the robot wasn’t performing to specs.
Spoilers: Eventually they realized that the robot was doing the job either way, and they just let it do it for whatever reason.
snooggums@kbin.social 10 months ago
technically correct
yamanii@lemmy.world 10 months ago
Yeah, it’s fascinating technology but also too many smokes and mirrors to trust any of the AI salesman since they can’t explain themselves exactly how it makes decisions.
bh11235@infosec.pub 10 months ago
This is an issue that has plagued the machine learning field since long before this latest generative AI craze. Decision trees you can understand, SVMs and Naive Bayes too, but the moment you get into automatic feature extraction and RBF kernels and stuff like that, it becomes difficult to understand how the verdicts issued by the model relate to the real world. But I’m sure it’s fair to say GPTs are even more inscrutable and made the problem worse.
btaf45@lemmy.world 10 months ago
This may be a dumb question, but why can’t you set the debugger on and step thru the program to see why it branches the way it does?
msage@programming.dev 10 months ago
Because it doesn’t have branches, it has neurons - and A LOT of them.
Each of them is tuned by the input data, which is a long and expensive process.
At the end, you hope your model has noticed patterns and not doing stuff at random.
But all you see is just weights on countless neurons.
Not sure I’m describing it correctly though.
bh11235@infosec.pub 10 months ago
I do exactly this kind of thing for my day job. In short: reading a syntactic description of an algorithm written in assembly language is not the equivalent of understanding what you’ve just read, which in turn is not the equivalent of having a concise and comprehensible logical representation of what you’ve just read, which in turn is not the equivalent of understanding the principles according to which the logical system thus described will behave when given various kinds of input.
General_Effort@lemmy.world 10 months ago
An Artificial Neural Network isn’t exactly an algorithm. There are algorithms to “run” ANNs, but the ANN itself is really a big bundle of equations.
An ANN has an input layer of neurons and an output layer. Between them are one or more hidden layers. Each neuron in one layer is connected to each neuron in the next layer. Let’s do without hidden layers for a start. Let’s say we are interested in handwriting. We take a little grayscale image of a letter (say, 16*16 pixels) and want to determine if it shows an upper case “A”.
Your input layer would have 16*16= 256 neurons and your output layer just 1. Each input value is a single number representing how bright that pixel is. You take these 256 numbers, multiply each one by another number, representing the strength of the connection between each of the input neurons and the single output neuron. Then you add them up and that value represents the likelihood of the image showing an “A”.
I think that wouldn’t work well (or at all) without a hidden layer but IDK.
The numbers representing the strength of the connections, are the parameters of the model, aka the weights. In this extremely simple case, they can be interpreted easily. If a parameter is large, then that pixel being dark makes it more likely that we have an “A”. If it’s negative, then it’s less likely. Finding these numbers/parameters/weights is what training a model means.
When you add a hidden layer, things get murky. You have an intermediate result and don’t know what it represents.
The impressive AI models take much more input, produce much more diverse output and have many hidden layers. The small ones, you can run on a gaming PC, have several billion parameters. The big ones, like ChatGPT, have several 100 billion. Each of these numbers is potentially involved in creating the output.
1stTime4MeInMCU@mander.xyz 10 months ago
An oversimplification but Imagine you have an algebraic math function where every word in English can be assigned a number.
x+y+z=n where x y z are the three words in a sentence. N is the next predicted word based on the coefficients of the previous 3.
Now imagine you have 10 trillion coefficients instead of 3. That’s an LLM, more or less.
ViscloReader@lemmy.world 10 months ago
Because in the case of the aforementioned
automatic feature extraction. In this case, the algorithm chooses itself what feature is relevant when making decisions. The problem is that those features are almost impossible to decript since they are often list of numbers.Socsa@sh.itjust.works 10 months ago
It’s still inscrutable, but it makes more sense if you think of all these these as arbitrary function approximation on higher dimension manifolds. The reason we can’t generate traditional numerical somvers for these problems is because the underlying analytical models fall apart when you over-parameterize them. Backprop is very robust at extreme parameter counts, and comes with much weaker assumptions that thing like series decomposition, so it really just looks like a generic numerical method which can scale to absurd levels.