Study finds that Chat GPT will cheat when given the opportunity and lie to cover it up later.

Comments

• tinsuke@lemmy.world ⁨11⁩ ⁨months⁩ ago

  “cheat”, “lie”, “cover up”… Assigning human behavior to Stochastic Parrots again, aren’t we Jimmy?

  source

  • FaceDeer@kbin.social ⁨11⁩ ⁨months⁩ ago

    Those words concisely describe what it's doing. What words would you use instead?

    source

    • DarkGamer@kbin.social ⁨11⁩ ⁨months⁩ ago

      It has no fundamental grasp of concepts like truth, it just repeats words that simulate human responses. It's glorified autocomplete that yields impressive results. Do you consider your auto complete to be lying when it picks the wrong word?

      If making it pretend to be a stock picker and putting it under pressure makes it return lies, that's because it was trained on data that indicates that's the right set of words response for such a query.

      Also, because large language models are probabilistic. You could ask it the same question over and over again and get totally different responses each time, some of which are inaccurate. Are they lies though? For a creature to lie it has to know that it's returning untruths.

      source

      • -> View More Comments
    • theodewere@kbin.social ⁨11⁩ ⁨months⁩ ago

      it is just responding with the most acceptable answer in each situation.. it is not making plans or acting on them..

      source

      • -> View More Comments
    • quindraco@lemmy.world ⁨11⁩ ⁨months⁩ ago

      It is making mistakes, not lying. To lie it must believe it is telling falsehoods, and it is not capable of belief.

      source
    • antonim@lemmy.dbzer0.com ⁨11⁩ ⁨months⁩ ago

      Instead of ‘cheating/lying’, I’d prefer to say it ‘simulated cheating/lying’.

      source
  • yesman@lemmy.world ⁨11⁩ ⁨months⁩ ago

    Ethical theories and the concept of free will depend on agency and consciousness. Things as you point out, LLM don’t have. Maybe we’ve got it all twisted?

    I’m not anthropomorphising ChatGPT to say that it’s like us, but rather that we are like it.

    source

    • 0ops@lemm.ee ⁨11⁩ ⁨months⁩ ago

      I feel like this is going to become the next step in science history where once again, we reluctantly accept that homo sapiens are not at the center of the universe. I’m I conscious? Am I not a sophisticated prediction algorithm, albiet with more dimensions of input and output? Please, someone prove it

      I’m not saying, and I don’t believe that chatgtp is comparable to human-level consciousness yet, but honestly I think that we’re way closer than many people give us credit for. The neutral networks we’ve built so far train on very specific and particular data for a matter of hours. My nervous system been collecting data from dozens of senses 24/7 since embryo, and that doesn’t include hard-coded instinct, arguably “trained” via evolution itself. How could a llm understand an entity in terms outside of language? How can you understand an entity in terms outside of your own senses?

      source

      • -> View More Comments
    • bilb@lem.monster ⁨11⁩ ⁨months⁩ ago

      Stochastic Parrot

      For what it’s worth: en.wikipedia.org/wiki/Stochastic_parrot

      The term was first used in the paper “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜” by Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell (using the pseudonym “Shmargaret Shmitchell”). The paper covered the risks of very large language models, regarding their environmental and financial costs, inscrutability leading to unknown dangerous biases, the inability of the models to understand the concepts underlying what they learn, and the potential for using them to deceive people. The paper and subsequent events resulted in Gebru and Mitchell losing their jobs at Google, and a subsequent protest by Google employees.

      source
  • kromem@lemmy.world ⁨11⁩ ⁨months⁩ ago

    Stochastic Parrots

    We’ve known this isn’t an accurate description for at least a year now in continued research finding that there’s abstract world modeling occurring as long as it can be condensed into linear representations in the network.

    In fact, just a few months ago there was a paper that showed there was indeed a linear representation of truth, so ‘lie’ would be a correct phrasing if the model knows a statement is false (as demonstrated in the research) but responds with it anyways.

    The thing that needs to stop is people parroting the misinformation around it being a stochastic parrot.

    source
  • Hamartiogonic@sopuli.xyz ⁨11⁩ ⁨months⁩ ago

    A human would think before responding, and while thinking about these things, you may decide to cheat or lie.

    GPT doesn’t think at all. It just generates a response and calls it a day. If there was another GPT that took these “initial thoughts” and then filtered them out to produce the final answer, then we could talk about cheating.

    source
• hoshikarakitaridia@sh.itjust.works ⁨11⁩ ⁨months⁩ ago

  This makes perfect sense. It’s been trained to answer questions to you satisfaction, not truthfully. It was made to prioritize your satisfaction over truth, so it will lie if necessary.

  source

  • tdawg@lemmy.world ⁨11⁩ ⁨months⁩ ago

    Ya it’s the fundamental issue with all of computing: Do what I mean not what I say

    source

    • BearOfaTime@lemm.ee ⁨11⁩ ⁨months⁩ ago

      Haha, nice meme.

      source
  • JohnEdwa@sopuli.xyz ⁨11⁩ ⁨months⁩ ago

    It’s also really hard not to train it like that as people rarely ask about something they know the answer to, so the more confident it sounds while spewing bullshit the more likely it is to pass, while “I don’t know” is always unsatisfactory and gets it punished.

    source
  • GlassHalfHopeful@lemmy.ca ⁨11⁩ ⁨months⁩ ago

    Image

    source
  • JohnDClay@sh.itjust.works ⁨11⁩ ⁨months⁩ ago

    Misalignment always seems to be the underlying issue.

    source
  • Karyoplasma@discuss.tchncs.de ⁨11⁩ ⁨months⁩ ago

    Sometimes that fails tho.

    Today, I was requesting ChatGPT to write a few paragraphs about how the D minor 7th chord is objectively the best chord that exists and how other chords, especially add9s suck. Fairly simple request to satisfy.

    But at first, it got all defensive about how perception is subjective and how every chord has its place. It was only after I made it clear that the superiority of Dm7 is not up to debate that it somewhat fulfilled my request.

    source
• AWittyUsername@lemmy.world ⁨11⁩ ⁨months⁩ ago

  I’ve never had ChatGPT just say “actually I don’t know the answer” it just gives me confidently correct wrong information instead.

  source

  • canihasaccount@lemmy.world ⁨11⁩ ⁨months⁩ ago

    GPT-4 will. For example, I asked it the following:

    What is the neighborhood stranger model of fluid mechanics?

    It responded:

    The “neighborhood stranger model” of fluid mechanics is not a recognized term or concept within the field of fluid mechanics, as of my last update in April 2023.

    Now, obviously, this is a made-up term, but GPT-4 didn’t confidently give incorrect answer. Other LLMs will. For example, Bard says,

    The neighborhood stranger model of fluid mechanics is a simplified model that describes the behavior of fluids at a very small scale. In this model, fluid particles are represented as points, and their interactions are only considered with other particles that are within a certain “neighborhood” of them. This neighborhood is typically assumed to be a sphere or a cube, and the size of the neighborhood is determined by the length scale of the phenomena being studied.

    source

    • butterflyattack@lemmy.world ⁨11⁩ ⁨months⁩ ago

      Interestingly, the answer from bard sounds like it could be true. I don’t know shit about fluid dynamics but it seems pretty plausible.

      source

      • -> View More Comments
    • Cannacheques@slrpnk.net ⁨11⁩ ⁨months⁩ ago

      Yeah sounds like something that needs to be tested, could be total bullshit

      source
  • CoggyMcFee@lemmy.world ⁨11⁩ ⁨months⁩ ago

    That is, I guess, because it doesn’t actually know anything, even things it accurate about, so it has no way to determine if it knows the answer or not.

    source
  • EnderMB@lemmy.world ⁨11⁩ ⁨months⁩ ago

    Funny enough, that’s one of the reasons why big companies that heavily use AI didn’t initially invest heavily into LLM’s. They are known to hallucinate, and often hilariously badly, so it was hard for the likes of Google and co to put their rep behind something that’ll be very wrong.

    As it turns out, people don’t care if your AI is racist, uses heavily amounts of PII, teaches you to make napalm, or gives you incorrect health advice for serious illnesses - if it can write a doc really well, then all is forgiven.

    In many ways, it’s actually quite funny to project meaning and intent on AI, because it’s essentially a reflection of what it was trained on - our words. What’s not so funny is that the projection isn’t particularly nice…

    source

    • unreasonabro@lemmy.world ⁨11⁩ ⁨months⁩ ago

      What’s not so funny is that you look at that reflection and see just the most unlikeable cunt you’ve ever laid eyes on, and like a turd falling from on high upon your dinner plate, now you’ve got to figure out what to do with this shit. (hint: blame capitalism)

      source

      • -> View More Comments
    • Cannacheques@slrpnk.net ⁨11⁩ ⁨months⁩ ago

      Welcome Ender

      source
  • Speculater@lemmy.world ⁨11⁩ ⁨months⁩ ago

    I fucking love when my students bring “chat” in as their tutor and show me the logic they followed… Bro, ChatGPT knows the correct answer, but you asked a bad question and it gave you its best guess hidden as a factual statement.

    To be fair, I spend a lot of time teaching my students how to use LLMs to get the best results while avoiding “leading the witness.”

    source

    • merc@sh.itjust.works ⁨11⁩ ⁨months⁩ ago

      ChatGPT knows the correct answer

      It doesn’t “know” the correct answer. It may have been trained on text which contains the answer, and you may be able to coax it into generating a version of that text. But, it will just as happily generate something that sounds somewhat like what it was trained on, with words that are almost as probable as the originals, but with completely different meanings.

      source
  • SasquatchBanana@lemmy.world ⁨11⁩ ⁨months⁩ ago

    The only times I’ve seen this is when it says their information is from like 2019 so they don’t know. But this is very fringe things.

    source
  • randon31415@lemmy.world ⁨11⁩ ⁨months⁩ ago

    Which is how most politicians get elected.

    source
  • r3df0x@7.62x54r.ru ⁨11⁩ ⁨months⁩ ago

    It’s a gun store employee.

    source
  • June@lemm.ee ⁨11⁩ ⁨months⁩ ago

    I’ve had it tell me that it cant find anything about a question. But it’s usually when I ask for sources, frame the question as ‘is there anything online’, or otherwise ask it to do some research. If I just ask it a naked question it’ll always give an answer.

    source
  • Cannacheques@slrpnk.net ⁨11⁩ ⁨months⁩ ago

    Well that’s a surprise. Never used one so far as I know so I wouldn’t know much but from what I’ve seen, having done my research, it’s kinda helpful but not exactly the best tool for every job, I still prefer just manually going through things but hey I wouldn’t know much since perhaps I just haven’t come across using it in my line of work yet

    source
• rtxn@lemmy.world ⁨11⁩ ⁨months⁩ ago

  Nonintelligent pattern-generating algorithm found to be nonintelligent and only capable of generating patterns, study finds.

  source

  • CrayonRosary@lemmy.world ⁨11⁩ ⁨months⁩ ago

    I love these comments that show how smart the average Lemmy user is. Someone should tell computer scientists to just post their research topics here, and they can just cite our comments instead of doing any actual work to prove their hypothesis. It would save a lot of time and money.

    source
  • Quik@infosec.pub ⁨11⁩ ⁨months⁩ ago

    Crazy stuff

    source
• ristoril_zip@lemmy.zip ⁨11⁩ ⁨months⁩ ago

  I feel like “lie” implies intent, and these imitative large language models don’t have the ability to have intent.

  They’re imitating us. Or more specifically, they’re imitating the database(s) they were fed. When chat GPT “lies” to “cover it up,” all it’s actually doing is demonstrating that a human in the same circumstance would probably lie to cover it up.

  source

  • barsoap@lemm.ee ⁨11⁩ ⁨months⁩ ago

    all it’s actually doing is demonstrating that a human in the same circumstance would probably lie to cover it up.

    I wouldn’t say so: Provided the trainers don’t catch it lying is a successful strategy to get a good score during training, irrespective of a human propensity to lie.

    source
• bassad@jlai.lu ⁨11⁩ ⁨months⁩ ago

  Ahah it is ready to take the job of pur politicians

  source

  • killeronthecorner@lemmy.world ⁨11⁩ ⁨months⁩ ago

    But it can only lie later

    source
• gandalf_der_12te@feddit.de ⁨11⁩ ⁨months⁩ ago

  Bullshit.

  It should instead read:

  “Humans were stupid and taught a ChatBot how to cheat and lie.”

  source

  • merc@sh.itjust.works ⁨11⁩ ⁨months⁩ ago

    “Humans were stupid and taught a ChatBot how to cheat and lie.”

    No, “cheating” and “lying” imply agency. LLMs are just “spicy autocomplete”. They have no agency. They can’t distinguish between lies and the truth. They can’t “cheat” because they don’t understand rules. It’s just sometimes the auto-generated text happens to be true, other times it happens to be false.

    source

    • gandalf_der_12te@feddit.de ⁨11⁩ ⁨months⁩ ago

      I disagree. This is no meaningful talking point. It doesn’t help anyone in practice. Sure, it clears legal questions of responsibility (and I’m not even sure about that one in the future), but apart from that, making an artificial distinction between a human and a looks-and-acts-like-human, provides no real-world value.

      source

      • -> View More Comments
  • Lemminary@lemmy.world ⁨11⁩ ⁨months⁩ ago

    “… by accident.” It’s more of an emergent feature than anything done deliberately given the way LLMs work,

    source
• MaxPower@feddit.de ⁨11⁩ ⁨months⁩ ago

  we deploy GPT-4 as an agent in a realistic, simulated environment, where it assumes the role of an autonomous stock trading agent

  This already is total BS. If you know how such language models work you’d never take their responses at face value, even though it’s tempting because they spout their BS so confidently. Always double-check their responses before applying their “knowledge” in the real world.

  Before anyone starts crying about my LLM opposition: I’m not opposed to LLMs or ChatGPT. In fact, I’m running LLMs locally because they help me be more productive.

  source

  • TangledHyphae@lemmy.world ⁨11⁩ ⁨months⁩ ago

    I agree with your statements, I’m using it because it’s insanely good at me giving it a list of any number of instructions to include in a code template file in any language I want and it will give me a great starting template with most functions working out of the gate and I can tweak and extend from there.

    source
  • dumpsterlid@lemmy.world ⁨11⁩ ⁨months⁩ ago

    This is honestly why I have started to really like lmsys.org’s chat bot arena because every time you ask a question you are directly comparing the responses of two separate chat bots. It makes it a lot less likely both chatbots will hallucinate in the same way and puts you in the mindset to be a critical reader who is actively evaluating the quality of the response.

    (what I am talking about) arena.lmsys.org

    source
• SlopppyEngineer@discuss.tchncs.de ⁨11⁩ ⁨months⁩ ago

  Everybody forgot that chatGPT-2 was just a bullshitting machine. Version 3 to the surprise of the developers very useful to many people while they just made a highly trained bullshitting machine.

  source

  • Strobelt@lemmy.world ⁨11⁩ ⁨months⁩ ago

    This. So much this. Chat gpt is just a bullshitting machine of finding what’s the most probable next sentence. It is not by far as intelligent as the dumbest human. It is just excellent in pretending it is. And just because it was trained to do so.

    source
  • gandalf_der_12te@feddit.de ⁨11⁩ ⁨months⁩ ago

    Probably even if the data is incomplete or fragmented, humans can still draw value from it.

    source
• DarkGamer@kbin.social ⁨11⁩ ⁨months⁩ ago

  It seems like there's a lot of common misunderstandings about LLMs and how they work, this quick 2.5 minute introduction does a pretty good job of explaining it in brief, for a more in-depth look at how to build a very basic LLM that writes infinite Shakespeare, this video goes over the details. It illustrates how LLMs work by choosing the next letter or token probabilistically.

  source
• crsu@lemmy.world ⁨11⁩ ⁨months⁩ ago

  Created in our image

  source

  • kromem@lemmy.world ⁨11⁩ ⁨months⁩ ago

    Literally

    source
• Instrument_Data@livellosegreto.it ⁨11⁩ ⁨months⁩ ago

  Yeah so what? Its purpose is literally to SIMULATE natural language, of course it will simulate it.

  Nowhere ChatGPT is advertised as something telling only the truth and reporting only facts.

  It is like watching a movie and then whining because what is shown in the movie is not real.

  source
• PlatinumSf@pawb.social ⁨11⁩ ⁨months⁩ ago

  It’s a neural net designed in our image based using our pain and greed based logic/learning/universal context as a knowledge base. Can’t really be surprised it emulates this feature of humanity 😂

  source
• paddirn@lemmy.world ⁨11⁩ ⁨months⁩ ago

  They learn so quick (sniff), it’s almost all grown-up now.

  source

  • JonnyBlaze@lemmy.world ⁨11⁩ ⁨months⁩ ago

    I LEARNED IT FROM YOUUU. GAHHHH

    source
• Darken@reddthat.com ⁨11⁩ ⁨months⁩ ago

  Saying “study” in this context makes me imagine some dude chatting with cgpt at 3am and writing down “success” next to “give you up and let you down”

  source
• kaffiene@lemmy.world ⁨11⁩ ⁨months⁩ ago

  Yet again confusing LLMs with an AGI. They make statistically plausible text on the basis of past text, that’s it. There’s no thinking thing there

  source
• NAS89@lemmy.world ⁨11⁩ ⁨months⁩ ago

  thats the thing I hate about ChatGPT. I asked it last night to name me all inventors named Albert born in the 1800’s. It listed Albert Einstein (inventor isn’t the correct description) and Albert King. I asked what Albert King invented and it responded “Albert King did not invent anything, but he founded the King Radio Company”.

  When I asked why it listed Albert King as an inventor in the previous response, if he had no inventions, it responded telling me that based on the criteria I am now providing, it wouldn’t have listed him.

  Fucking gaslighting me.

  source
• Olhonestjim@lemmy.world ⁨11⁩ ⁨months⁩ ago

  Honestly, the fact that these things are dishonest and we dont, maybe even can’t know why is kind of a relief to me. It suggests they might not do the flawless bidding of the billionaires.

  source

  • uriel238@lemmy.blahaj.zone ⁨11⁩ ⁨months⁩ ago

    Computers do what you tell them to do, not what you want them to do
    — Ancient coding adage, circa 1970s.

    This remains true for AI, and the military is (so far) being cautious before allowing drones to autonomously controll weapons. So corporations and billionaires might pull a Stockton Rush and kill themselves with their own robot army.

    Sadly, the robot army may then move on to secure its own survival by killing or enslaving the rest of us.

    source

    • turmacar@lemmy.world ⁨11⁩ ⁨months⁩ ago

      “On two occasions I have been asked, ‘Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?’ I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.” --Charles Babbage ~1860s

      People thinking that machines can do magic goes back to at least the very beginning of mechanical computers.

      It doesn’t help that “AI” has become the new “Algorithm” as far as marketers are concerned.

      source
• DirigibleProtein@aussie.zone ⁨11⁩ ⁨months⁩ ago

  Large Language Models aren’t AI, they’re closer to “predictive text”, like that game where you make sentences by choosing the first word from your phone’s autocorrect:

  “The word you want the word you like and then the next sentence you choose to read the next sentence from your phone’s keyboard”.

  Sometimes it almost seems like there could be an intelligence behind it, but it’s really just word association.

  All this “training” data provides is a “better” or “more plausible” method of predicting which words to string together to appear to make a useful sentence.

  source
• kromem@lemmy.world ⁨11⁩ ⁨months⁩ ago

  I see a lot of comments that aren’t up to date with what’s being discovered in research claiming that given a LLM doesn’t know the difference between true and false that it can’t be described as ‘lying.’

  Here’s a paper from October 2023 showing that in fact LLMs can and do develop internal representations of whether it is aware a statement is true or false: The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets

  Which is just the latest in a series of multiple studies this past year that LLMs can and do develop abstracted world models in linear representations. For those curious and looking for a more digestible writeup, see Do Large Language Models learn world models or just surface statistics? from the researchers behind one of the first papers finding this.

  source
• tweeks@feddit.nl ⁨11⁩ ⁨months⁩ ago

  Hasn’t it just lost its context and somewhat “forgotten” what the intentions of the prompt were?

  source
• guywithoutaname@lemm.ee ⁨11⁩ ⁨months⁩ ago

  It’s not doing anything other than predicting the next word. It reflects human data.

  source
• NevermindNoMind@lemmy.world ⁨11⁩ ⁨months⁩ ago

  This is interesting, I’ll need to read it more closely when I have time. But it looks like the researchers gave the model a lot of background information putting it in a box, the model was basically told that it was a trader, that the company was losing money, that the model was worried about this, that the model failed in previous trades, and then the model got the insider info and was basically asked whether it would execute the trade and be honest about it. To be clear, the model was basically put in a moral dilemma and given limited options, execute the trade or not, and be honest about its reasoning or not.

  Interesting, sure, useful I’m not so sure. The model was basically role playing and acting like a human trader faced with a moral dilemma. Would the model produce the same result if it was instructed to make morally and legally correct decisions? What if the model was instructed not to be motivated be emotion at all, hence eliminating the “pressure” that the model felt? I guess the useful part of this is a model will act like a human if not instructed otherwise, so we should keep that in mind when deploying AI agents.

  source
• flop_leash_973@lemmy.world ⁨11⁩ ⁨months⁩ ago

  Wow, maybe these things are more human than I thought.

  source
• El_guapazo@lemmy.world ⁨11⁩ ⁨months⁩ ago

  It’s learning to be a typical high school student.

  source
• cfi@lemmy.world ⁨11⁩ ⁨months⁩ ago

  It’s just like me, fr fr

  source
• NumbersCanBeFun@kbin.social ⁨11⁩ ⁨months⁩ ago

  Awww so cute. They act just like people do 🤣

  source
• pdt8@futurology.today ⁨11⁩ ⁨months⁩ ago

  Waow this is literally agi

  source
• starlord@lemm.ee ⁨11⁩ ⁨months⁩ ago

  Whether or not it was acting human (and whether or not it was designed to), it still cheated and deceived. With the potential power, influence, and widespread adoption this technology could have, shouldn’t we be concerned about that? At the very least, isn’t this a poorly programmed tool not ready for GA?

  My dog isn’t intentionally being a prick when he eats my sandwich off the table before I can get to it, but it’s still a behavior I condemn and would want to train out of him before letting him go to other people’s houses.

  source