Comment on Chatbots Make Terrible Doctors, New Study Finds

WorldsDumbestMan@lemmy.today ⁨2⁩ ⁨months⁩ ago

• XLE@piefed.social ⁨2⁩ ⁨months⁩ ago

  You can use zero randomization to get the same answer for the same input every time, but at that point you’re sort of playing cat and mouse with a black box that’s still giving you randomized answers. Even if you found a false positive or false negative, you can’t really debug it out…

  source

  • Buddahriffic@lemmy.world ⁨2⁩ ⁨months⁩ ago

    Yeah, if you turn off randomization based on the same prompts, you can still end up with variation based on differences in the prompt wording. And who knows what false correlations it overfitted to in the training data. Like one wording might bias it towards picking medhealth data while another wording might make it more likely to use 4chan data. Not sure if these models are trained on general internet data, but even if it’s just trained on medical encyclopedias, wording might bias it towards or away from cancers, or how severe it estimates it to be.

    source

    • WorldsDumbestMan@lemmy.today ⁨2⁩ ⁨months⁩ ago

      I see it like programming randomly, until you get something that is accidentally right, then you rate it, and it now shows up every time. I think that’s how it roughly works. True about the prompt wording, that can be somewhat limited too, thanks to the army of idiots beta testers that will make every kind of prompt.

      Having said that uh…it’s not much better than just straight up programming the thing yourself. It’s like, programming, but extra lazy, right?

      source