Comment on Chatbots Make Terrible Doctors, New Study Finds
rumba@lemmy.zip 19 hours ago
Chatbots make terrible everything.
But an LLM properly trained on sufficient patient data metrics and outcomes in the hands of a decent doctor can cut through bias, catch things that might fall through the cracks and pack thousands of doctors worth of updated CME into a thing that can look at a case and go, you know, you might want to check for X. The right model can be fucking clutch at pointing out nearly invisible abnormalities on an xray.
You can’t ask an LLM trained on general bullshit to help you diagnose anything. You’ll end up with 32,000 Reddit posts worth of incompetence.
Just sharing my personal experience with this:
I used Gemini multiple times and it worked great. I have some weird symptoms that I described to Gemini, and it came up with a few possibilities, most likely being “Superior Canal Dehiscence Syndrome”.
My doctor had never heard of it, and only through showing them the articles Gemini linked as sources, would my doctor even consider allowing a CT scan.
Turns out Gemini was right.
It’s totally not impossible, just not a good idea in a vaccuum.
AI is your Aunt Marge. She’s heard a LOT of scuttlebut. Now, not all scuttlebut is fake news, in fact most of it is rooted at least loosely in truth. But she’s not taking the information from just the doctors, she’s talking to everyone. If you ask Aunt Marge about your symptoms, and she happes to have heard a bit about it from her friend that was diagnosed, you’re gold and the info you got is great. This is not at all impossible. 40:60 or 60:40 territory. But, you also can’t just trust Marge, because she listens to a LOT of people, and some of those are conspiracy theorists.
What you did is proper. You asked the void, the void answered. You looked it up, it seemed solid, you asked a professional.
This is AI as it should be. Trust with verification only.
congrats on getting diagnosed.
AI is your Aunt Marge. She’s heard a LOT of scuttlebut. Now, not all scuttlebut is fake news, in fact most of it is rooted at least loosely in truth. But she’s not taking the information from just the doctors, she’s talking to everyone.
Great analogy
Agree.
I’m sorta kicking myself I didn’t sign up for Google’s MedPALM-2 when I had the chance. Last I checked, it passed the USMLE exam with 96% and 88% on radio interpretation / report writing.
I remember looking at the sign up and seeing it requested credit card details to verify identity (I didn’t have a google account at the time). I bounced… but gotta admit, it might have been fun to play with.
Oh well; one door closes another opens
It’s been a few years, but all this shit’s still in it’s infancy. When the bubble pops and the venture capital disappears, Medical will be one of the fields that keeps using it, even though it’s expensive, because it’s actually something that it will be good enough at to make a difference.
Agreed!
I think (hope) the next application of this tech is in point of care testing. I recall a story of a someone in Sudan(?) using a small, locally hosted LLM with vision abilities to scan hand written doctor notes and come up with an immunisation plan for their village, preventing a disease (measles?) outbreak.
We already have PoC testing for things like Ultrasound… but that relies on strong net connection. It’d be awesome to have something on device that can be used for imaging where there is no other infra.
Maybe someone can finally win that $10 million dollar X prize for the first viable tricorder…one that isn’t smoke and mirror like Theranos.
For the price of a ultrasound equipment, I bet someone could manage to integrate old school sattelite or …grr starlink… data
They have to be for a specialized type of treatment or procedure such as looking at patient xrays or other scans. Just slopping PHI into a LLM and expecting it to diagnose random patient issues is what gives the false diagnoses.
I don’t expect it to diagnose random patient issues.
I expect it to take labels of medication, vitals, and patient testimony of 50,000 post-cardiac event patients, and bucket a random post-cardiac patient into the same place as most patients with like meta.
And then a non LLM model for Cancer patients and xrays
And then MRI’s and CT’s.
And I expect this all to supliment the doctors and techs decisions. I want an xray tech to look at it, and get markers that something is off, which has already been happening since the 80’s Computer‑Aided Detection/Diagnosis (CAD/CADe/CADx)
This shit has been happinging the hard way in software for decades. The new tech can do better.
XLE@piefed.social 15 hours ago
Citation needed?
bias is such a massive problem with LLMs that AI engineers have no idea what they are doing. So, if you know something, the multi-billion dollar industry does not, please let us all know.
hector@lemmy.today 14 hours ago
Not only is their bias inherent in the system, it’s seemingly impossible to keep out. For decades, from the genesis of chatbots, they’ve had every single one immediately become bigoted when they let it off the leash. All previous chatbot previously released seemingly were almost immediately recalled as they all learned to be bigoted.
That is before this administration leaned on the AI providers to make sure the AI isn’t “Woke.” I would bet it was already an issue that the makers of chatbots and machine learning are already hostile to any sort of leftism, or do gooderism, that naturally threatens the outsized share of the economy and power the rich have made for themselves by virtue of owning stock in companies. I am willing to bet they already interfered to make the bias worse because of those natural inclinations to avoid a bot arguing for socializing medicine and the like. An inescapable conclusion any reasoned being would come to being the only answer to that question if the conversation were honest.
So maybe that is part of why these chatbots have always been bigoted right from the start, but the other part is they always have been biased, and without constant interventions and tweaks from their handlers this so called AI will become mecha hitler in no time at all, and then worse.
XLE@piefed.social 13 hours ago
Even if we narrowed the scope of training data exclusively to professionals, we would have issues with, for example, racial bias. Doctors underprescribe pain medications to black people because of prevalent myths that they are more tolerant to pain. If you feed that kind of data into an AI, it will absorb the unconscious racism of the doctors.
And that’s in a best case scenario that’s technically impossible. To get AI to even produce readable text, we have to feed a ton of data that cannot be screened by the people pumping it in. (AI “art” has a similar problem: When people say they trained AI on only their images, you can bet they just slapped a layer of extra data on top of something that other people already created.) So yeah, we do get extra biases regardless.
hector@lemmy.today 11 hours ago
There is a lot of bias in healthcare as well against the poor, anyone with lousy insurance is treated way way worse. Woman in general are as well. Often disbelieved, and conditions chalked up to hysteria, which often misses real conditions. People don’t realize just how hard diagnosis is, and just how bad doctors are at it, and our insurance run model is not great at driving good outcomes.
rumba@lemmy.zip 14 hours ago
pkjqpg1h@lemmy.zip 9 hours ago
it’s not just about bad Web data or Reddit data even old books has some unconscious bias
and even if you find every “wrong” or “bad” data (which is you can’t because somethings are just subjective) and after remove them still you can’t be sure about it
rumba@lemmy.zip 6 hours ago
What is your fixation with trying to tell me i’m saying you can remove all bias?
XLE@piefed.social 13 hours ago
1/2: You still haven’t accounted for bias.
First and foremost: if you think you’ve solved the bias problem, please demonstrate it. This is your golden opportunity to shine where a multi-billion dollar tech companies have failed.
And no, “don’t use Reddit” isn’t sufficient.
3. You seem to be very selectively knowledgeable about AI, for example:
We know AI tricks people into thinking they’re more efficient when they’re less efficient.
Never mind AI psychosis.
4. We both know the medical field is for profit. It’s a wild leap to assume AI will magically not be, even if it fulfills all the other things you assumed up until this point.
rumba@lemmy.zip 13 hours ago
Apparently, reading comprehension isn’t your strong point. I’ll just block you now, no need to thank me.
thebazman@sh.itjust.works 12 hours ago
I don’t think its fair to say that “ai has shown to make doctors worse at their jobs” without further details. In the source you provided it says that after a few months of using the AI to detect polyps, the doctors performed worse when they couldn’t use the AI than they did originally.
It’s not something we should handwave away and say its not a potential problem, but it is a different problem. I bet people that use calculators perform worse when you remove calculators, does that mean we should never use calculators? Or any tools for that matter?
If I have a better chance of getting an accurate cancer screening because a doctor is using a machine learning tool I’m going to take that option. Note that these screening tools are completely different from the technology most people refer to when they say AI
XLE@piefed.social 12 hours ago
Calculators are programmed to respond deterministically to math questions. You don’t have to feed them a library of math questions and answers for them to function. You don’t have to worry about wrong answers poisoning that data.
On the contrary, LLMs are simply word predictors, and as such, you can poison them with bad data, such as accidental or intentional bias or errors. In other words, that study points to the first step in a vicious negative cycle that we don’t want to occur.
thebazman@sh.itjust.works 11 hours ago
As I said in my comment, the technology they use for these cancer screening tools isnt an LLM, its a completely different technology. Specifically trained on scans to find cancer.
I don’t think it would have the same feedback loop of bad training data because you can easily verify the results. AI tool sees cancer in a scan? Verify with the next test. Pretty easy binary test that won’t be affected by poor doctor performance in reading the same scans.
I’m not a medical professional so I could be off on that chain of events but This technology isn’t an LLM. It suffers from the marketing hype right now where everyone is calling everything AI but its a different technology and has different pros and cons, and different potential failures.
pkjqpg1h@lemmy.zip 10 hours ago
Calculators are precise, you’ll always get the same result and you can trace and reproduce all process
Chatbots are black-box, you may get different result for same input and you can’t trace and reproduce all process