Comment on AI-generated code contains more bugs and errors than human output

MyMindIsLikeAnOcean@piefed.world ⁨3⁩ ⁨weeks⁩ ago

• Kolanaki@pawb.social ⁨3⁩ ⁨weeks⁩ ago

  It can’t even copy and paste a Hello World example properly.

  source

  • user224@lemmy.sdf.org ⁨3⁩ ⁨weeks⁩ ago

    It works well for recalling something you already know, whether it be computer or human language. What’s a word for… what’s a command/function that does…

    source

    • frongt@lemmy.zip ⁨3⁩ ⁨weeks⁩ ago

      For words, it’s pretty good. For code, it often invents a reasonable-sounding function or model name that doesn’t exist.

      source

      • Xenny@lemmy.world ⁨2⁩ ⁨weeks⁩ ago

        It’s not even good for words. AI just writes the same stories over and over and over and over and over and over. I’d argue the best and only real use for an llm is to help be a rough draft editor and correct punctuation and grammar. We’ve gone way way way too far with the scope of what it’s actually capable of

        source

        • -> View More Comments
    • ThirdConsul@lemmy.ml ⁨2⁩ ⁨weeks⁩ ago

      According to OpenAis internal test suite and system card, hallucination rate is about 50% and the newer the model the worse it gets.

      And that fact remains unchanged on other LLM models.

      source
  • ptu@sopuli.xyz ⁨3⁩ ⁨weeks⁩ ago

    I use it for things that are simple and monotonous to write. This way I’m able to deliver results to tasks I couldn’t have been arsed to do. I’m a data analyst and mostly use mysql and power query

    source
  • dogdeanafternoon@lemmy.ca ⁨3⁩ ⁨weeks⁩ ago

    What’s your preferred Hello world language? I’m gunna test this out. The more complex the code you need, the more they suck, but I’ll be amazed if it doesn’t work first try to simply print hello world.

    source

    • xthexder@l.sw0.com ⁨3⁩ ⁨weeks⁩ ago

      Malbolge is a fun one

      source

      • dogdeanafternoon@lemmy.ca ⁨3⁩ ⁨weeks⁩ ago

        Cheeky, I love it.

        Got correct code first try. Failed creating working docker first try. Second try worked.

        tmp="$(mktemp)"; cat >"$tmp" <<'MBEOF'
        ('&%:9]!~}|z2Vxwv-,POqponl$Hjig%eB@@>}=<M:9wv6WsU2T|nm-,jcL(I&%$#"
        `CB]V?Tx<uVtT`Rpo3NlF.Jh++FdbCBA@?]!~|4XzyTT43Qsqq(Lnmkj"Fhg${z@>
        MBEOF
        docker run --rm -v "$tmp":/code/hello.mb:ro esolang/malbolge malbolge /code/hello.mb; rm "$tmp"
        

        Output: Hello World!

        source

        • -> View More Comments
  • Serinus@lemmy.world ⁨2⁩ ⁨weeks⁩ ago

    It works well when you use it for small (or repetitive) and explicit tasks. That you can easily check.

    source
• morto@piefed.social ⁨2⁩ ⁨weeks⁩ ago

  In a postgraduate class, everyone was praising ai, calling it nicknames and even their friend (yes, friend), and one day, the professor and a colleague were discussing some code when I approached, and they started their routine bullying on me for being dumb and not using ai. Then I looked at his code and asked to test his core algorithm that he converted from a fortran code and “enhanced” it. I ran it with some test data and compared to the original code and the result was different! They blindly trusted some ai code that deviated from their theoretical methodology, and are publishing papers with those results!

  Even after showing the different result, they didn’t convince themselves of anything and still bully me for not using ai. Seriously, this shit became some sort of cult at this point. People are becoming irrational. If people in other universities are behaving the same and publishing like this, I’m seriously concerned for the future of science and humanity itself. Maybe we should archive everything published up to 2022, to leave as a base for the survivors from our downfall.

  source

  • MyMindIsLikeAnOcean@piefed.world ⁨2⁩ ⁨weeks⁩ ago

    The way it was described to me by some academics is that it’s useful…but only as a “research assistant” to bounce ideas off of and bring in arcane or tertiary concepts you might not have considered (after you vet them thoroughly, of course).

    The danger, as described by the same academics, is that it can act as a “buddy” who confirms you biases. It can generate truly plausible bullshit to support deeply flawed hypotheses, for example. Their main concern is it “learning” to stroke the egos of the people using it so it creates a feedback loop and it’s own bubbles of bullshit.

    source

    • tym@lemmy.world ⁨2⁩ ⁨weeks⁩ ago

      So, linkedin?

      source
  • Xenny@lemmy.world ⁨2⁩ ⁨weeks⁩ ago

    That’s not a bad idea. I’m already downloading lots of human knowledge and media that I want backed up because I can’t trust humanity anymore to have it available anymore

    source