Comment on Vibe coding service Replit deleted production database

zerofk@lemmy.zip ⁨4⁩ ⁨days⁩ ago

• rockerface@lemmy.cafe ⁨4⁩ ⁨days⁩ ago

  Well, there was a catastrophic error of judgement. It was made by whichever human thought it was okay to let a LLM work on production codebase.

  source
• jj4211@lemmy.world ⁨4⁩ ⁨days⁩ ago

  judgement

  Yeah, it admitted to an error in judgement because the prompter clearly declared it so.

  Generally LLMs will make whatever statement about what has happened that you want it to say. If you told it it went fantastic, it would agree. If you told it that it went terribly, it will parrot that sentiment back.

  Which what seems to make it so dangerous for some people’s mental health, a text generator that wants to agree with whatever you are saying, but doing so without verbatim copying so it gives an illusion of another thought process agreeing with them. Meanwhile, concurrent with your chat is another person starting from the exact same model getting a dialog that violently disagrees with the first person. It’s an echo chamber.

  source
• hisao@ani.social ⁨4⁩ ⁨days⁩ ago

  Are you aware of generalization and it being able to infer things and work with facts in highly abstract way? Might not necessarily be judgement, but definitely more than just completion. If a model is capable of only completion (ie suggesting only the exact text strings present in its training set), it means it suffers from heavy underfitting in AI terms.

  source

  • ChairmanMeow@programming.dev ⁨4⁩ ⁨days⁩ ago

    Completion is not the same as only returning the exact strings in its training set.

    LLMs don’t really seem to display true inference or abstract thought, even when it seems that way. A recent Apple paper demonstrated this quite clearly.

    source

    • hisao@ani.social ⁨4⁩ ⁨days⁩ ago

      Coming up with even more vague terms to try to downplay it is missing the point. The point is simple: it’s able to solve complex problems and do very impressive things that even human struggle to, in very short time. It doesn’t really matter what we consider true abstract thought of true inference. If that is something humans do, then what it does might very well be more powerful than true abstract thought, because it’s able to solve more complex problems and perform more complex pattern matching.

      source

      • Jhex@lemmy.world ⁨4⁩ ⁨days⁩ ago

        The point is simple: it’s able to solve complex problems and do very impressive things that even human struggle to, in very short time

        You mean like a calculator does?

        source

        • -> View More Comments
      • ChairmanMeow@programming.dev ⁨4⁩ ⁨days⁩ ago

        Well the thing is, LLMs don’t seem to really “solve” complex problems. They remember solutions they’ve seen before.

        The example I saw was asking an LLM to solve “Towers of Hanoi” with 100 disks. This is a common recursive programming problem, takes quite a while for a human to write the answer to. The LLM manages this easily. But when asked to solve the same problem with with say 79 disks, or 41 disks, or some other oddball number, the LLM fails to solve the problem, despite it being simpler(!).

        It can do pattern matching and provide solutions, but it’s not able to come up with truly new solutions. It does not “think” in that way. LLMs are amazing data storage formats, but they’re not truly ‘intelligent’ in the way most people think.

        source

        • -> View More Comments