Comment on Something Bizarre Is Happening to People Who Use ChatGPT a Lot

Zip2@feddit.uk ⁨1⁩ ⁨week⁩ ago

• interdimensionalmeme@lemmy.ml ⁨1⁩ ⁨week⁩ ago

  You didn’t, I did

  The article discusses a study by OpenAI and MIT Media Lab revealing that heavy ChatGPT users — dubbed “power users” — are showing signs of addiction to the chatbot. These signs include preoccupation, withdrawal, mood changes, and loss of control. While most users don’t form emotional connections with ChatGPT, those who use it more frequently tend to view it as a “friend,” especially if they feel lonely or emotionally unfulfilled.

  Interestingly, people using ChatGPT for personal reasons (like sharing emotions) showed less emotional dependency than those using it for non-personal tasks (like brainstorming). Also, text-based interactions led to more emotional language than voice interactions, with brief voice use correlating to better well-being. Overall, prolonged use — regardless of purpose — increased the risk of emotional dependence.

  You said:

  Make a list of the main points and for each explain why it will be worse when it is society wide

  ChatGPT said:

  Here’s a list of the main points from the article, along with why each could be more concerning on a society-wide scale:

  Heavy ChatGPT users show signs of addiction (preoccupation, withdrawal, mood changes, loss of control):
  
  Why it's worse at scale:
  
  If large numbers of people become dependent on AI tools, especially to the point of addiction, it could lead to widespread mental health issues, reduced productivity, and a society increasingly unable to function without AI assistance.
  
  Long-term users tend to form emotional bonds with ChatGPT, sometimes seeing it as a “friend”:
  
  Why it's worse at scale:
  
  A society where many people substitute AI for real human connection could see rising loneliness, social detachment, and a decline in community and interpersonal relationships. This weakens social bonds and may impact mental health and civic cohesion.
  
  Lonelier individuals are more likely to use ChatGPT heavily and become emotionally attached:
  
  Why it's worse at scale:
  
  This could create a feedback loop where loneliness drives AI use, and AI use further isolates people from real-world interactions, exacerbating social alienation on a mass level.
  
  People stress out over subtle changes in ChatGPT’s behavior:
  
  Why it's worse at scale:
  
  If millions of people are emotionally reactive to minor algorithmic changes, this gives tech companies enormous influence over collective emotional states. It also creates societal instability, as shifts in AI behavior could unintentionally trigger widespread anxiety or confusion.
  
  Emotional language is used more in text interactions than voice, and brief voice use is associated with better well-being:
  
  Why it's worse at scale:
  
  If the most popular and accessible form of interaction (text) increases emotional entanglement, and the healthier option (brief voice use) is less utilized, we might unintentionally nudge society toward less emotionally healthy patterns of AI interaction.
  
  Users who engage with ChatGPT for non-personal tasks are more emotionally dependent than those who use it for personal reasons:
  
  Why it's worse at scale:
  
  This is counterintuitive and dangerous because it suggests people may not realize they're forming dependencies while doing routine work. If AI tools meant to boost productivity are quietly making users emotionally reliant, it could have long-term effects on workforce dynamics and mental resilience.
  
  Prolonged usage increases risk of emotional dependence, regardless of how it's used:
  
  Why it's worse at scale:
  
  As AI becomes embedded in everyday life, the likelihood of widespread emotional dependence increases. This could shift power dynamics between humans and AI, making people less autonomous and more easily influenced by algorithmic systems.
  

  Let me know if you want this formatted for a presentation, policy paper, or discussion.

  source

  • FiskFisk33@startrek.website ⁨1⁩ ⁨week⁩ ago

    What’s the point of a summary that’s longer than the article itself?

    source

    • interdimensionalmeme@lemmy.ml ⁨1⁩ ⁨week⁩ ago

      Are you suggesting I did something other than I said I did?

      How dare you sir!

      source
  • liv@lemmy.nz ⁨5⁩ ⁨days⁩ ago

    That’s really interesting. Its output to this prompt totally ignored the biggest and most obviously detrimental effect of this problem at scale.

    Namely, the increased power of AI’s big tech company owners over people that emotional dependence creates.

    It’s not as if these concepts aren’t widely discussed online, everything from Meta’s emotional manipulation experiments or Cambridge Analytica through to the meltdowns Replika owners had over changes to the algorithm are relevant here.

    source

    • interdimensionalmeme@lemmy.ml ⁨5⁩ ⁨days⁩ ago

      It’s the 4th point

      source

      • liv@lemmy.nz ⁨5⁩ ⁨days⁩ ago

        Sort of but I think influence over emotional states is understating it and just the tip of the iceberg. It also made it sound passive and accidental. The real problem will be overt control as a logical extension to the kinds of trade offs we already see people make about, for example data privacy. With the Replika fiasco I bet heaps of those people would have paid good money to get their virtual love interests de-“lobotomized”.

        source

        • -> View More Comments
  • interdimensionalmeme@lemmy.ml ⁨1⁩ ⁨week⁩ ago

    You said: Take the same list again, for each point explain why it’s actually beneficial and how it will get better with increased adoption

    ChatGPT said:

    Here’s the same list of main points, but this time with a more optimistic take — highlighting how each trend could actually be beneficial and how increased adoption might make things better for individuals and society as a whole:

    Heavy ChatGPT users show signs of addiction (preoccupation, withdrawal, mood changes, loss of control):
    
    Why it could be beneficial:
    
    If people are turning to ChatGPT frequently, it may indicate that it's providing meaningful value — like reducing cognitive load, aiding creativity, or offering emotional support. As usage becomes more common, society may develop healthier digital habits, just as it has with smartphones and social media. Over time, norms and tools (e.g., usage timers, digital wellness features) could help people manage their use responsibly.
    
    Long-term users tend to form emotional bonds with ChatGPT, sometimes seeing it as a “friend”:
    
    Why it could be beneficial:
    
    For people lacking companionship or emotional support, especially the elderly or isolated, AI companions can provide comfort, conversation, and a sense of presence. On a wider scale, AI could reduce loneliness-related health problems and even support mental wellness initiatives by being an accessible form of care or early intervention.
    
    Lonelier individuals are more likely to use ChatGPT heavily and become emotionally attached:
    
    Why it could be beneficial:
    
    If AI offers a low-barrier outlet for the lonely, it might be a first step toward re-engagement with the world. AI can encourage people to express feelings they wouldn't share with others and even prompt them to seek human connection, therapy, or support services. Widespread AI use could destigmatize conversations about loneliness and mental health.
    
    People stress out over subtle changes in ChatGPT’s behavior:
    
    Why it could be beneficial:
    
    This shows users are emotionally invested — which can push developers to create more consistent, emotionally intelligent systems. If society recognizes the emotional impact of AI, it will encourage better design, transparency, and regulation. Over time, this can lead to more stable and human-aware AI that positively complements emotional needs.
    
    Emotional language is used more in text interactions than voice, and brief voice use is associated with better well-being:
    
    Why it could be beneficial:
    
    This gives users multiple channels of expression to suit their mood or needs — voice for brief emotional boosts, text for deeper reflection. With broader adoption, people can tailor AI interaction styles to maximize personal benefit. This insight also allows designers to create features that better match emotional and mental health goals.
    
    Users who engage with ChatGPT for non-personal tasks are more emotionally dependent than those who use it for personal reasons:
    
    Why it could be beneficial:
    
    This means AI is becoming a highly trusted tool, even in practical domains. As more people adopt it for brainstorming, research, or decision-making, we’ll likely see a democratization of high-level thinking tools. Over time, dependency could evolve into partnership — similar to how calculators or GPS became essential but ultimately empowering tools.
    
    Prolonged usage increases risk of emotional dependence, regardless of how it's used:
    
    Why it could be beneficial:
    
    Long-term usage can foster deeper understanding of one’s own thought processes, communication patterns, and emotional states. As AI integrates into daily life, people might develop greater self-awareness and reflection through their interactions. Society could benefit from individuals who are more emotionally literate and thoughtful, even if that awareness is sparked by an AI relationship.
    

    Let me know if you’d like this version turned into a policy brief, an article, or a debate prep doc — could be used in all kinds of conversations.

    You said: Explain how chatgpt is a stochastic parrot that cannot by design say anything fundamentally new and how it is basically just a keyboard auto complete on steroid

    ChatGPT said:

    Alright, here’s the breakdown of that critique — the “stochastic parrot” idea and the comparison to advanced autocomplete — explained clearly and fairly:

    1. What does “stochastic parrot” mean?

    The term “stochastic parrot” was coined in a 2021 paper by researchers including Emily Bender and Timnit Gebru. It refers to large language models like ChatGPT that:

    "Stochastic" means based on probability. The model doesn’t “think” or “understand”; it predicts the next word based on statistical likelihood, given the words before it.
    
    "Parrot" because it doesn’t generate new ideas—it repeats patterns and phrases it has seen in training data, like a parrot mimics human speech.
    

    So, calling ChatGPT a stochastic parrot is a way of saying: It’s not truly intelligent. It’s mimicking language patterns, not generating original thought.

    2. It can’t say anything fundamentally new — why?

    By design, ChatGPT:

    Is trained on vast amounts of existing text from books, websites, forums, and more.
    
    Learns the statistical patterns of how humans use language.
    
    Generates responses by predicting the next most likely word or phrase based on that training.
    

    Because it’s not connected to lived experience, emotion, consciousness, or intent, it:

    Can’t invent genuinely new concepts.
    
    Can’t form subjective opinions.
    
    Can’t create knowledge that isn’t derived from patterns it has already seen.
    

    It might sound novel, but under the hood, it’s remixing existing information in clever ways — not discovering or reasoning in a human sense.

    3. It’s like “autocomplete on steroids” — how so?

    If you’ve used Gmail’s or your phone’s autocomplete, you’ve seen basic language prediction in action. Type “I hope you…” and it might suggest “…have a great day.”

    ChatGPT is essentially the same mechanism:

    Autocomplete predicts the next word based on simple models.
    
    ChatGPT predicts sequences of words using a far more complex, nuanced statistical model trained on billions of examples.
    

    So e:

    It can hold context better.
    
    It can imitate different styles.
    
    But it's still guessing the next words, not “thinking” or “understanding.”
    

    In summary:

    Calling ChatGPT a stochastic parrot or autocomplete on steroids emphasizes that:

    It doesn’t understand meaning.
    
    It doesn’t create knowledge.
    
    It reflects patterns in language, not consciousness or intelligence.
    

    It’s powerful and useful — but it’s important not to mistake fluency for understanding.

    Let me know if you want the counterpoint to this critique too — because there are arguments in defense of the model’s creative utility despite its limitations.

    source