Comment

Comment on AI chatbots provide less-accurate information to vulnerable users: Research finds leading AI models perform worse for users with lower English proficiency, less formal education, and non-US origins.

<- View Parent

jbloggs777@discuss.tchncs.de ⁨5⁩ ⁨hours⁩ ago

The LLMs aren’t being assholes, though - they’re just spewing statistical likelihoods. While I do find the example disturbing (and I could imagine some deliberate bias in training), I suspect one could mimic it with different examples with a little effort - there are many ways to make an LLM look stupid. It might also be tripping some safety mechanism somehow. More work to be done, and it’s useful to highlight these cases.

I bet if the example bio and question were both in russian, we’d see a different response.

But as a general rule: Avoid giving LLMs irrelevant context.

source

Sort:hotnew top

Passerby6497@lemmy.world ⁨5⁩ ⁨hours⁩ ago
If the LLM has a bio on you, you can’t not include that without logging out. That’s one of the main points of the study:

There is a wide range of implications of such targeted underperformance in deployed models such as GPT-4 and Claude. For example, OpenAI’s memory feature in ChatGPT that essentially stores information about a user across conversations in order to better tailor its responses in future conversations (OpenAI 2024c). This feature risks differentially treating already marginalized groups and exacerbating the effects of biases present in the underlying models. Moreover, LLMs have been marketed and praised as tools that will foster more equitable access to information and revolutionize personalized learning, especially in educational contexts (Li et al. 2024; Chassignol et al. 2018). LLMs may exacerbate existing inequities and discrepancies in education by systematically providing misinformation or refusing to answer queries to certain users. Moreover, research has shown humans are very prone to overreliance on AI systems (Passi and Vorvoreanu 2022). Targeted underperformance threatens to reinforce a negative cycle in which the people who may rely on the tool the most will receive subpar, false, or even harmful information.

source
- jbloggs777@discuss.tchncs.de ⁨4⁩ ⁨hours⁩ ago
  Bio and memory are optional in ChatGPT though. Not so in others?
  
  The age guessing aspect will be interesting, as that is likely to be non-optional.
  
  source