Robot moderation could be coming to your town

Submitted ⁨⁨5⁩ ⁨weeks⁩ ago⁩ by ⁨auk@slrpnk.net⁩ to ⁨fediverse@lemmy.world⁩

I made a robot moderator. It models trust flow through a network that’s made of voting patterns, and detects people and posts/comments that are accumulating a large amount of “negative trust,” so to speak.

In its current form, it is supposed to run autonomously. In practice, I have to step in and fix some of its boo-boos when it makes them, which happens sometimes but not very often.

I think it’s working well enough at this point that I’d like to experiment with a mode where it can form an assistant to an existing moderation team, instead of taking its own actions. I’m thinking about making it auto-report suspect comments, instead of autonomously deleting them. There are other modes that might be useful, but that might be a good place to start out. Is anyone interested in trying the experiment in one of your communities? I’m pretty confident that at this point it can ease moderation load without causing many problems.

!santabot@slrpnk.net

source

Comments

Sort:hotnew top

alcoholicorn@lemmy.ml ⁨5⁩ ⁨weeks⁩ ago
So it would delete people’s posts if they get downvoted a lot or if the poster tends to upvote heavily downvoted posts?

You’ve automated the suppression of dissenting voices.

source
- auk@slrpnk.net ⁨5⁩ ⁨weeks⁩ ago
  
  So it would delete people’s posts if they get downvoted a lot
  
  No.
  
  or if the poster tends to upvote heavily downvoted posts?
  
  No.
  
  You’ve automated the suppression of dissenting voices.
  
  Am not.
  
  It’s a perfectly fair concern. I’m trying to be careful to make sure I’m not doing that. There’s quite a lot of explanation in the FAQ, and some conversations you can look back over with people who were concerned, because they’ve had experience with exactly that happening to them.
  
  At one point I tried to illustrate with data just how big a jerk you have to be before it starts banning you. If you’re interested, I can start doing that again. Being a dissenting voice on its own is nowhere near enough to anger the bot. You can look over !pleasantpolitics@slrpnk.net and see quite a few dissenting voices. I’ve also offered to delve, for any user who feels that this has happened to them, into the breakdown of why they’re being ranked down, which almost always is because they’re being a jerk about their “dissenting” opinion, and not the opinion itself.
  
  Also, I think it’s hilarious that someone coming from lemmy.ml is accusing me of trying to suppress dissenting voices. Lemmy.ml has been suppressing dissenting voices since its inception. The degree to which I’m bending over backwards not to suppress dissenting voices is something I think you should absorb and carry over to the lemmy.ml moderators as a good replacement for their current banhammer circus.
  
  source
  - hendrik@palaver.p3x.de ⁨5⁩ ⁨weeks⁩ ago
    Hehe, good roast on lemmy.ml
    
    source
    -> View More Comments
  - alcoholicorn@lemmy.ml ⁨5⁩ ⁨weeks⁩ ago
    
    At one point I tried to illustrate with data just how big a jerk you have to be before it starts banning you. If you’re interested, I can start doing that again.
    
    Would you?
    
    My understanding is that downvotes reflect whether or not someone agrees with a post or comment much more than whether the user is making a constructive comment or not so they can only be used to infer how agreeable the comment is.
    
    I’ve also offered to delve, for any user who feels that this has happened to them, into the breakdown of why they’re being ranked down, which almost always is because they’re being a jerk about their “dissenting” opinion, and not the opinion itself.
    
    Use me as an example, I regularly get dozens of downvotes for such hot takes as “supporting genocide is hurts the dems chances of getting elected, we need them to stop that if we want them to win.”.
    
    source
    -> View More Comments
Danterious@lemmy.dbzer0.com ⁨5⁩ ⁨weeks⁩ ago
Is there a way of tailoring the moderation to a communities needs? One problem that I can see arising is that it could lead to a mono culture of moderation practices. If there is a way of making the auto reports relative that would be interesting.

~Anti~ ~Commercial-AI~ ~license~ ~(CC~ ~BY-NC-SA~ ~4.0)~

source
- auk@slrpnk.net ⁨5⁩ ⁨weeks⁩ ago
  I tried that early on. It does have a “perspective,” in terms of what communities are the trusted ones. What I found was that more data is simply better. It’s able to sort out for itself who the jerks are, and who are the widely trusted social networks, when it looks at a global picture. Trying to tell it to interpret the data a certain configured way or curtail things, when I tried it, only increased the chance of error without making it any better-tuned to the specific community it’s looking at.
  
  I think giving people some insight into how it works, and ability to play with the settings, so to speak, so they feel confident that it’s on their side instead of being a black box, is a really good idea. I tried some things along those lines, but I didn’t get very far along.
  
  Maybe it’d be nice to set it up so it’s more transparent. Instead of auto-banning, it can send auto-reports to the moderators with comments which it considers to be bad, and an indication of how bad or why. And then, once a week, it can publish a report of what it’s done and why, some justification for anyone who it took action against, so that everyone in the community can see it, so there aren’t surprises or secrets.
  
  I thought about some other ideas, such as opening up an “appeal” community where someone can come in and talk with people and agree not to be a jerk, and get unbanned as long as they aren’t toxic going forward. That, coupled with the idea that if you come in for your appeal and yell at everyone that you are right and everyone else is wrong and this is unfair, your ban stays, could I think be a good thing. Maybe it would just be a magnet for toxicity. But in general, one reason I really like the idea is that it’s getting away from one individual making decisions about what is and isn’t toxic and outsourcing it more to the community at large and how they feel about it, which feels more fair.
  
  source
  - Danterious@lemmy.dbzer0.com ⁨5⁩ ⁨weeks⁩ ago
    
    But in general, one reason I really like the idea is that it’s getting away from one individual making decisions about what is and isn’t toxic and outsourcing it more to the community at large and how they feel about it, which feels more fair.
    
    Yeah that does sound useful it is just that there are some communities where it isn’t necessarily clear who is a jerk and who has a controversial minority opinion. For example how do you think the bot would’ve handled the vegan community debacle that happened. There were a lot of trusted users who were not necessarily on the side of vegans and it could’ve made those communities revert back to a norm of what users think to be good and bad.
    
    I think giving people some insight into how it works, and ability to play with the settings, so to speak, so they feel confident that it’s on their side instead of being a black box, is a really good idea. I tried some things along those lines, but I didn’t get very far along.
    
    If you’d want I can help with that. Like you said it sounds like a good way of decentralizing moderation so that we have less problems with power tripping moderators and more transparent decisions. I just want it so that communities can keep their specific values while easing their moderation burden.
    
    source
    -> View More Comments
toothbrush@lemmy.blahaj.zone ⁨5⁩ ⁨weeks⁩ ago
This is an extremely important tool for moderating large forums, thank you for working on this! This is one of those mod tools that some people may need to enable large scale moderated federated forums!

source
Blaze@feddit.org ⁨5⁩ ⁨weeks⁩ ago
Interesting concept

source
AlDente@sh.itjust.works ⁨5⁩ ⁨weeks⁩ ago
I must say I don’t like the idea of a social-credit-score bot.

Regarding your implementation, I saw the summary of your own comments elsewhere in this post and I noticed all the annotations were on upvoted/blue segments. Other summaries you posted focused more on negative/red segments. Would it be possible to enforce a minimum of 1 or 2 from both categories?

Also, would you be kind enough to read my tea leaves? Am I an acceptable citizen of the Lemmy community?

source
- Draconic_NEO@lemmy.world ⁨4⁩ ⁨weeks⁩ ago
  I’m in agreement. The social credit idea really doesn’t bode well. Karma resrictions on Reddit are one of the bad parts of Reddit and I for one am glad that it’s not a thing here.
  
  source
- auk@slrpnk.net ⁨4⁩ ⁨weeks⁩ ago
  You’re fine. Why would you not be? You left 15 comments in the last month, and they were all upvoted. It doesn’t even really have much to go on to rank you, but your rank is positive, nowhere near 0, much less far enough into the negative side that it would need to be to even be greylisted.
  
  99% of Lemmy is made of acceptable citizens. That’s a real number. Only 1% of the users that it evaluates, which is itself only a tiny fraction of the total Lemmy population, ever get blacklisted. You have to be very obnoxious before it starts targeting you. I can understand the worry that this is going to arbitrarily start attacking people because of some vague AI bot decision, but that’s not what is happening.
  
  The visualization of someone’s social credit score just picks the 5 most impactful posts, it doesn’t discriminate based on positive or negative. If you want to see what the red corresponds to on my graph, the most negative things I have done within the time window are:
  
  slrpnk.net/comment/11257933
  
  slrpnk.net/comment/11257387
  
  They both contributed some red to the graph, I think. The red at the far right end is comments within this post that people are taking exception to.
  
  source
Sundial@lemm.ee ⁨5⁩ ⁨weeks⁩ ago
What kind of sample size of comments or time length would the bot use to make a decision? Is it safe to assume that those new Troll accounts would fly under this bots radar?

source
- auk@slrpnk.net ⁨5⁩ ⁨weeks⁩ ago
  It’s the last 30 days of comments. That’s long enough to be robust, but short enough that someone can realistically rehabilitate their image with the bot by not being a jerk for 30 days, and restore their posting ability.
  
  I was hoping that it would be a good tool for self-reflection and fairness in moderation. In practice, the people who get banned for being jerks are totally uninterested in revising their commenting strategy, and choose instead just to yell at me that I’m awful and my bot is unfair and it should be their right to come in and be a jerk if they want to, and banning them means I am breaking Lemmy. Then they restart one of the arguments that got them banned in the first place. I don’t know what I was thinking, expecting anything different, but that’s what happened. You can see some of it happening in these comments.
  
  New accounts, or accounts that have been recently inactive, are a hard problem. I think I’ve got it mostly worked out now. If the bot has limited information, it won’t ban you, but it will be super-strict if you have a generally negative reception, and if its unclear impression of you is negative and you also make a comment that gets downvoted, it’ll delete the comment. I think it should work fairly well, but it’s still in development. It’s hard to test, because that situation only comes up a few times a month, so I basically just have to wait a while every time I do it.
  
  You can check a user by searching the modlog for their user, and santa@slrpnk.net as the moderator, and see what comes up. If you see that they’ve been banned at any point, then they are probably a reprobate of one sort or another.
  
  source
  - Sundial@lemm.ee ⁨5⁩ ⁨weeks⁩ ago
    Thanks for the information. I took a look at the bots community and for what it’s worth I appreciate the amount of effort you put into fine tuning it as well as being as transparent as possible.
    
    source
  - Backlog3231@reddthat.com ⁨5⁩ ⁨weeks⁩ ago
    How many new accounts do you want? I delete and recreate my account all the time. I’d be happy to make new accounts and put a bunch of comments in them for you. I could make one a week or something.
    
    source
    -> View More Comments
comfy@lemmy.ml ⁨4⁩ ⁨weeks⁩ ago
I’ve played (briefly) with automated moderation bots on forums, and the main thing stopping me from going much past known-bad profiles (e.g. visited the site from a literal spamlist) is not just false positives but malicious abuse. I wanted to add a feature which would censor an image immediately with a warning if it was reported for (say) porn, shock imagery or other extreme content, but if a user noticed this, they could falsely report content to censor it until a staff member dismisses the report.

Could an external brigade of trolls get legitimate users banned or their posts hidden just by gaming your bot? That’s a serious issue which could make real users have their work deleted, and in my experience, users can take that very personally.

source
- auk@slrpnk.net ⁨4⁩ ⁨weeks⁩ ago
  It’s possible. I think it’s more difficult than people think. You have to do it on a scale which is blatantly obvious to anyone who’s looking, so you’re just inviting a ban.
  
  One person swore to me that it would be really easy, so I invited them to try, and they made a gang of bots which farmed karma and then mass-downvoted me, trying to get me banned from my own place. If you look at my profile you’ll see some things which have -300 score because of it. I welcomed the effort, since I’m interested in how well it will resist that kind of attack. Their first effort did exactly nothing, because none of the downvote bots had any rank within the algorithm. I gave them some pointers on how they could improve for a second time around, and they went radio silent and I haven’t heard from them since then.
  
  source
  - comfy@lemmy.ml ⁨3⁩ ⁨weeks⁩ ago
    Haha they thought it was too easy and were proven wrong!
    
    Honestly, if a place is obscure enough, even smaller barriers of entry help, like forums that don’t let you post on important boards until you build a reputation. There’s only so much effort an adversary is willing to put in, and if there isn’t a financial incentive or huge political incentive, that barrier could be low.
    
    source
lambalicious@lemmy.sdf.org ⁨5⁩ ⁨weeks⁩ ago
Finally, what the Fediverse needed:

AIs.

Good to know they’ve caught up to us from the walled gardens. Welp, I guess that’s it folks. Let’s pack up and go back to Usenet.

source
- comfy@lemmy.ml ⁨4⁩ ⁨weeks⁩ ago
  One of the first few instances I heard of was botsin.space, which has been around since at least 2017. Bots aren’t new. (Not sure where you’re pulling “AI” from, this is old* tech, and I don’t mean that negatively)
  
  source
- auk@slrpnk.net ⁨4⁩ ⁨weeks⁩ ago
  The core algorithm, a robust way to determine global trust rankings based on a network of relative trust, was first codified and used for this purpose in the late 1800s. It would be difficult to come up with a way to attack this problem that is further separated from the era of AI algorithms. I think you would need help from a math historian.
  
  source

Robot moderation could be coming to your town

Comments

AIs.