Comment on Chatbots Make Terrible Doctors, New Study Finds

theunknownmuncher@lemmy.world ⁨1⁩ ⁨month⁩ ago

• scarabic@lemmy.world ⁨1⁩ ⁨month⁩ ago

  It’s actually interesting. They found the LLMs gave the correct diagnosis Hugh-90-something percent of the time if they had access to the notes doctors wrote about their symptoms. But when thrust into the room, cold, with patients, the LLMs couldn’t gather that symptom info themselves.

  source

  • Hacksaw@lemmy.ca ⁨1⁩ ⁨month⁩ ago

    LLM gives correct answer when doctor writes it down first… Wowoweewow very nice!

    source

    • tyler@programming.dev ⁨1⁩ ⁨month⁩ ago

      You have misunderstood what they said.

      source

      • Hacksaw@lemmy.ca ⁨1⁩ ⁨month⁩ ago

        If you seriously think the doctor’s notes about the patient’s symptoms don’t include the doctor’s diagnostic instincts then I can’t help you.

        The symptom questions ARE the diagnostic work. Your doctor doesn’t ask you every possible question. You show up and you say “my stomach hurts”. The Doctor asks questions to rule things out until there is only one likely diagnosis then they stop and prescribe you a solution if available. They don’t just ask a random set of questions. If you give the AI the notes JUST BEFORE the diagnosis and treatment it’s completely trivial to diagnose because the diagnostic work is already complete.

        God you AI people literally don’t even understand what skill, craft, trade, and art are and you think you can emulate them with a text predictor.

        source

        • -> View More Comments
    • scarabic@lemmy.world ⁨1⁩ ⁨month⁩ ago

      If you think there’s no work between symptoms and diagnosis, you’re dumber than you think LLMs are.

      source
  • SuspciousCarrot78@lemmy.world ⁨1⁩ ⁨month⁩ ago

    Funny how the hivemind over looks that bit enroute to stunt on LLMs.

    If anything, that 90% result supports the idea that Garbage In = Garbage Out. I imagine a properly used expert system (Med-PALM-2) is even better than 90% accurate in differentials.

    source