This demonstrates in a really layman-understandable way some of the shortcomings of LLMs as a whole, I think.
Trolling chatbots with made-up memes
Submitted 1 year ago by vowseh@lemmy.eco.br to technology@lemmy.world
https://www.aiweirdness.com/trolling-chatbots-with-made-up-memes/
Comments
csfirecracker@lemmyf.uk 1 year ago
FatTony@discuss.online 1 year ago
An LLM never lies because it cannot lie.
orclev@lemmy.world 1 year ago
Corollary: An LLM never tells the truth because it cannot tell the truth.
FaceDeer@kbin.social 1 year ago
It's only a "shortcoming" if you aren't aware of how these LLMs function and are using it for something it's not good at (in this case information retrieval). If instead you want it to be making stuff up, what was previously an undesirable hallucination becomes desirable creativity.
csfirecracker@lemmyf.uk 1 year ago
I didn’t mean to argue against the usefulness of LLMs entirely, they absolutely have their place. I was moreso referring to how everyone and their dog are making AI assistants for tasks that need accurate data without addressing how easy it is for them to present you bad data with total confidence.
Nougat@kbin.social 1 year ago
I would say the specific shortcoming being demonstrated here is the inability for LLMs to determine whether a piece of information is factual (not that they're even dealing with "pieces of information" like that in the first place). They are also not able to tell whether a human questioner is being truthful, or misleading, or plain lying, honestly mistaken, or nonsensical. Of course, which one of those is the case matters in a conversation which ought to have its basis in fact.
Moobythegoldensock@lemm.ee 1 year ago
Indeed, and all it takes is one lie to send it down that road.
For example, I asked ChatGPT how to teach my cat to ice skate, with predictable admonishment:
Teaching a cat to ice skate is not a practical or safe activity for both you and your cat. Cats are not naturally adapted to the slippery surface of ice, and attempting to teach them to skate could result in injury or distress for your pet. Additionally, it’s unlikely that your cat would enjoy this activity or have any interest in it.
But after I reassured it that my cat loves ice skating, it changed its tune:
In addition to working with a professional trainer, you can try practicing some basic skating skills with your cat, such as stopping and turning. Start by teaching your cat to stop using a simple “snowplow” stop, which involves pointing the skates inward and applying pressure to the ice to slow down. Reward your cat with treats and praise when they successfully stop. [. . .] Once your cats have mastered basic skating skills and are comfortable on the ice, you can begin introducing them to the fundamentals of hockey, such as stickhandling, passing, and shooting. It’s important to start with simple drills and progress gradually, always using positive reinforcement and rewards to encourage your cats and make the experience enjoyable for them. [. . .] While it may be cute to imagine cats playing hockey with tiny little sticks, it’s important to keep in mind that it may not be safe or comfortable for them to hold onto traditional hockey sticks. However, you could consider using specially designed cat-sized hockey sticks that are lightweight and easy for cats to handle. [. . .] [Y]ou could establish penalties for aggressive behavior such as scratching, biting or excessive roughness between cats. In addition, you could create rules around how many cats are allowed on the ice at once, and limit the amount of playtime for each cat to avoid overexertion. [. . .] When selecting knee pads for your cat, it’s important to prioritize comfort and safety. The knee pads should fit snugly without restricting movement or causing discomfort, and should be made from soft, breathable materials that won’t irritate your cat’s skin. [. . .] If you do decide to hold an exhibition match against the small dog league, it’s important to take steps to ensure the safety and well-being of all the animals involved. This may include carefully selecting the dogs and cats based on their temperament and play style, providing ample supervision and training for all animals and handlers, and implementing rules and guidelines to prevent injury or aggression.
Even after telling it I lied and my cat doesn’t actually like ice skating, its acceptance of my previous lie still affected it:
If your cat is not showing any interest in ice skating and seems to prefer other activities, it may not be necessary to try and teach her to skate. While some cats may enjoy the thrill of ice skating, others may prefer more relaxed activities like napping or playing with toys.
PhantomPhanatic@lemmy.world 1 year ago
Here’s the thing, the LLM isn’t recalling and presenting pieces of information. It’s creating human-like strings of words. It will give you a human-like phrase based on whatever you tell it. Chatbots like ChatGPT are fine tuned to try to filter what they say to be more helpful and truthful but at it’s core it just takes what you say and makes human-like phrases to match.
csfirecracker@lemmyf.uk 1 year ago
Thank you for putting it far more eloquently than I could have
yuriy@lemmy.world 1 year ago
from “tall horse” to “this is what the rich get away with” and it’s just still a giraffe. that might as well be real, it just feels real.
surewhynotlem@lemmy.world 1 year ago
The entire concept behind a LLM is that the machine is designed to make up stories, and occasionally those stories aren’t false. To use it for anything besides that is reckless.
HarkMahlberg@kbin.social 1 year ago
Even AI-generated fiction can be reckless if it contains themes that are false, harmful, or destructive. If it writes a story that depicts genocide positively and masks it through metaphor, allegory, parable, whatever, then yes it's just "a made up story" but it's no less dangerous than if it were an Op Ed in a major new outlet.
treefrog@lemm.ee 1 year ago
So, just like real people, AI hate telling people I don’t know.
gerryflap@feddit.nl 1 year ago
Damn the snowbonk is kinda cool though. They’re a cutie
Imgonnatrythis@sh.itjust.works 1 year ago
I look forward to the snowbonk derived memes.
orclev@lemmy.world 1 year ago
Ironically we’ve had a snowbonk for about a decade. One of our relatives gave us a couple of ceramic seals/walruses and we thought it would be funny to put one of them in our freezer, which is where its sat ever since. To be fair it’s quite a bit smaller than a loaf of bread, about the size of a couple muffins, so not sure it counts. Also I’m pretty sure the one we actually put in the freezer is a seal not a walrus. So it’s a snowbonk from wish.com.
GigglyBobble@kbin.social 1 year ago
The first AI-generated meme.