Comment

Comment on A weird phrase is plaguing scientific papers – and we traced it back to a glitch in AI training data

The lede is buried deep in this one. Yeah, these dumb LLMs got bad training data that persists to this day, but more concerning is the fact that some scientists are relying upon LLMs to write their papers. This is literally the way scientists communicate their findings to other scientists, lawmakers, and the public, and they’re using fucking predictive text like it has cognition and knows anything.

Sure, most (all?) of those papers got retracted, but those are just the ones that got caught. How many more are lurking out there with garbage claims fabricated by a chatbot?

Thankfully, science will inevitably sus those papers out eventually, as it always does, but it’s shameful that any scientist would be so fatuous to put out a paper written by a dumb bot. You’re the experts. Write your own goddamn papers.

source

Sort:hotnew top

adespoton@lemmy.ca ⁨3⁩ ⁨days⁩ ago
In some cases, it’s people who’ve done the research and written the paper who then use an LLM to give it a final polish. Often, it’s people who are writing in a non-native language.

Doesn’t make it good or right, but adds some context.

source
- Telorand@reddthat.com ⁨3⁩ ⁨days⁩ ago
  Sure, and I’m sympathetic to the baffling difficulties of English, but use Google Translate and ask someone who’s more fluent for help with the final polish (as a single suggestion). Trusting your work, trusting science to an LLM is lunacy.
  
  source
  - Saleh@feddit.org ⁨3⁩ ⁨days⁩ ago
    Google translate is an LLM.
    
    en.wikipedia.org/wiki/Google_Translate
    en.wikipedia.org/wiki/Neural_machine_translation
    
    source
    wewbull@feddit.uk ⁨3⁩ ⁨days⁩ ago
    Check the results though. Google translate is far far better at translation than a generic LLM.
    
    source
  - Squirrelsdrivemenuts@lemmy.world ⁨3⁩ ⁨days⁩ ago
    It might be hard for them to find someone who is both fluent in english AND knows the field well enough to know vegetative electron microscopy is not a thing. Most universities have one general translation help service and science has a lot of field-specific weird terms.
    
    source
    moakley@lemmy.world ⁨3⁩ ⁨days⁩ ago
    That’s why he said start with Google Translate. Because Google Translate isn’t giving gibberish like vegetative electron microscopy.
    
    source
- wewbull@feddit.uk ⁨3⁩ ⁨days⁩ ago
  Adding extra polish like nonsense phrases. Nobody is supervising it then.
  
  source
BussyCat@lemmy.world ⁨3⁩ ⁨days⁩ ago
They were translating them not actually writing them like obviously it should have been caught by reviewers but that’s not nearly as bad

source
- wewbull@feddit.uk ⁨3⁩ ⁨days⁩ ago
  Translating them…otherwise know as rewriting the whole paper.
  
  source
  - BussyCat@lemmy.world ⁨2⁩ ⁨days⁩ ago
    There is a huge difference between asking a LLM “ translate the quick brown fox jumped over the lazy dog” and “ write a sentence about a fox and a dog” when you ask it to translate you can get weird translation issues like we saw here but you also get those sometimes with google translate but it shouldn’t change the actual content of the paper
    
    source
    wewbull@feddit.uk ⁨2⁩ ⁨days⁩ ago
    Have you asked an LLM to translate anything bigger than a few sentences? It doesn’t have enough contextual storage to keep a whole paper “in mind” and soon wanders off into nonsense.
    
    Google translate is a different beast.
    
    source
dgriffith@aussie.zone ⁨3⁩ ⁨days⁩ ago

Thankfully, science will inevitably sus those papers out eventually, as it always does,

In the future, all search engines will have an option to ignore any results from 2022-20xx, the era of AI slop.

source
unexposedhazard@discuss.tchncs.de ⁨3⁩ ⁨days⁩ ago
Its the immediate takeaway i made from the headline, so i dont feel like its buried deep

source
- Telorand@reddthat.com ⁨3⁩ ⁨days⁩ ago
  It’s not mentioned at all in the article, so what you inferred from the headline is not what the author conveyed.
  
  source
  - unexposedhazard@discuss.tchncs.de ⁨3⁩ ⁨days⁩ ago
    Ah, i admit i didnt read it, because the headline and the implication of AI being an issue in academia wasnt exactly news to me.
    
    source
Ledericas@lemm.ee ⁨2⁩ ⁨days⁩ ago
oh yea,not to mention alot of papers tend to be low quality before the AI was used, ive been hearing people are writing dozens of papers just to fluff up thier resume/cv. it was quanitity over quality. i was in an presentation where the guy presenting thier research wrote 40+ papers just to get hired a university somewhere.

source