Didn’t check for their specific approach, but this is a pretty standard metric in research.
It mostly boils down to either full mechanical turk (crowd source people to mark whether a post is positive or negative) or generating training data through one. I think there is a Michael Reeves video where he demonstrated this while analyzing /r/wallstreetbets posts since he needed to fully understand all the jargon/stupidity. But the idea is the same. You use humans to periodically update what words/phrases are negative and positive and then have a model train on those to extrapolate.
But there are plenty of training sets and even models out there for interpeting. The lesser ones will see “asshole” and assume negative and “awesome” and assume positive. But all the ones worth using at this point will use NLP to understand that “My asshole itches” is not a negative comment but “It would be awesome if you played in traffic” is very negative.
deweydecibel@lemmy.world 11 months ago
We also have to deal with the fact that toxicity has become an almost meaningless label. The way we seem to apply it now, feels like we’d say there was a lot of “toxicity” around the time of the civil rights movement, too.
We’ve conflated “angry, hateful, bitter, disruptive, belittling” with “caring enough to get upset”. There’s been study after study trying to blame social media for the rise in “political toxicity”, and every last single one of them seems to want to sweetly ignore the context of the moment in time we’re living in. People are acting volatile because there are a lot of volatile events happening that directly affect people’s lives. And all these high-minded discussions about how people online are so mean and rude, or how people don’t listen to each other anymore, consistently ignore that very crucial piece of context.
So I ask again, what do we mean by “toxic”? Because I have a strong feeling a good deal of women were being real “toxic” on June 24 2022. Why is the story not about why?
thepiggz@programming.dev 11 months ago
I think you’re onto something saying toxic is a pretty unspecific term to use when talking about such things. Maybe it would be a better conversation to ask: when do the ends justify the means?
Windex007@lemmy.world 11 months ago
I’ll even step the conversation back a hot second to: do the means even result in the desired ends?
I’d argue (supported by every study ever done on the subject), that it doesn’t. The issue isn’t that you haven’t called your MAGA uncle a hillbilly redneck enough. No matter how many times you get called a woke liberal snowflake, I don’t think you’re going to genuinely re-think your position on building a wall.
If there IS an amount of verbal rage that could turn you into a MAGA, then by all means, disregard.
But… If there isn’t, and you genuinely care about changing outcomes, then I strongly challenge people to consider if “the ends justify the means” is predicated on an earlier faulty assumption that the means even generate the ends at all.
jmp242@sopuli.xyz 11 months ago
I agree that I’ve heard a lot of the same studies. I wonder though about the nudge and shame effects however. By this I mean, we’re pretty sure peer pressure is a thing (or at least I haven’t heard of anyone disputing that in recent research). I’ve seen self-censorship and studies that seem to also show that works.
I don’t know if the world would be better overall if we went back to 1990s levels of people not taking conspiracy theories seriously, and it being a negative view from “the average person” if you were ranting that the earth was flat. That happened somehow - those were tamped down, and I’d argue it’s plausible that it was basically peer pressure.
The means might well not be to convince the MAGA uncle, but to influence his kids, your kids, and the rest of the family to treat him as “the crazy uncle” rather than a person to emulate. Similarly, while you’ll never convince hardcore woke or MAGA people they’re wrong, you might affect the wider view of what’s “normal” for others watching. We’ve all seen the alternative of not engaging / leaving leave the space to become a self reinforcing echo chamber.
thepiggz@programming.dev 11 months ago
Agreed. Always a good thought to have when one is considering going down that road. Is the future predictable enough to really expect that particular end?