How should Lemmy sort posts so small communities can compete?

Submitted ⁨⁨11⁩ ⁨hours⁩ ago⁩ by ⁨PumpkinDrama@reddthat.com⁩ to ⁨fediverse@lemmy.world⁩

cross-posted from: reddthat.com/post/52386265

Right now, big communities dominate the feed. I’m wondering what sort algorithm could level the field so niche or hobbyist communities have a fair chance to get seen.

There’s a good related post: Niche Communities won’t be able to reach their true potential until Lemmy adds a sort that takes engagement into account. It puts it well:

“If Lemmy is to truly start having active hobbyist communities instead of being 95% lefty US politics, Shitposts, and some tech stuff, it needs a sort that takes into account the user’s engagement.”

What do you think should be the default sort for a more balanced Lemmy?

source

Comments

Sort:hotnew top

PhilipTheBucket@piefed.social ⁨10⁩ ⁨hours⁩ ago
Try “Scaled,” it is better.

source
- antonim@lemmy.dbzer0.com ⁨10⁩ ⁨hours⁩ ago
  I’ve just tried it out on “All” and basically the entire first page is filled with one user’s posts to !visualarts@lemmy.dbzer0.com
  
  source
  - PhilipTheBucket@piefed.social ⁨10⁩ ⁨hours⁩ ago
    Hm
    
    https://lemmy.dbzer0.com/?dataType=Post&listingType=All&sort=Scaled
    
    Yeah, maybe so. I think you may have to do it to “Subscribed” only, and then sometimes cut out one specific community or other that it is over-promoting. I mean, it’s doing what it’s supposed to do, promoting small communities, but I think sometimes it maybe gets carried away.
    
    source
    -> View More Comments
  - MHLoppy@fedia.io ⁨10⁩ ⁨hours⁩ ago
    Imo that's as much a problem of the sorting algorithm as it is with a single community blasting out too many posts at once without any consideration for how current frontends are unable to usefully integrate that into people's feeds. A couple of years ago nanoUFO was being a bit (subjectively) overenthusiastic about posting - I counted and !games had 40 posts at once from them and it made the first couple pages of my feed basically unusable for a while.
    
    source
- avidamoeba@lemmy.ca ⁨7⁩ ⁨hours⁩ ago
  Scaled with Subscribed, then liberally subscribe to any and all communities I think I might be interested in. Check the Scaled + All firehose once in a while and subscribe to new communities if anything new shows up.
  
  source
cerement@slrpnk.net ⁨10⁩ ⁨hours⁩ ago
TIL: there are people who don’t sort by “New”

source
- schnurrito@discuss.tchncs.de ⁨10⁩ ⁨hours⁩ ago
  That would cause me to miss many interesting threads that were created at a time when I happened not to be looking at Lemmy.
  
  “New comments” it is for me, that causes threads to get bumped to the top as long as other people still find them interesting.
  
  source
- HappyFrog@lemmy.blahaj.zone ⁨9⁩ ⁨hours⁩ ago
  Yeah, I almost always sort by bew, then I check once a week using top to see if I missed anything.
  
  source
  - scrubbles@poptalk.scrubbles.tech ⁨8⁩ ⁨hours⁩ ago
    This is the way, and for comments added on posts
    
    source
OpenStars@piefed.social ⁨5⁩ ⁨hours⁩ ago
I tried and tried and tried and tried and tried and tried and tried and tried with Lemmy. The best I found that I could hope for was sorting by New, but mostly I just gave up hope for it.

Until I moved to PieFed, and now the issue has multiple solutions. For one, using the Topic/Feeds (which are user-customizeable and shareable) you really can have your cake and eat it too, e.g. you can unsubscribe from all politics communities so that those do not show up on your main homepage, but an entire new feed completely dedicated to News & Politics is just a click away. Or Memes. Or Hobbies. Or Movies & TV, or any of a thousand other things - again, you can build your own, or subscribe to one that someone else has made.

And for another, for sufficiently low-traffic communities you can click the bell icon (which you can do to pretty much anything - users, posts, comments, communities, etc. - plus you can even UNCLICK that to silence notifications from your own content!!), so that you get a notification for each and every single new post to it. But, if it ever does get to be too much, you can mark all as read and/or separate the different categories of notifications from one another - community posts by others vs. replies to your own content.

PieFed really is leaving Lemmy behind in the dust, as far as features are concerned.

source
Nemo@slrpnk.net ⁨10⁩ ⁨hours⁩ ago
It’s not a competition. Get rid of that mindset.

source
- Chozo@fedia.io ⁨7⁩ ⁨hours⁩ ago
  It's not necessarily about competition, it's about visibility. If I create something and I want to share it with people, that means I want people to see it. It doesn't necessarily mean "I want people to see this more than other posts", just "I'd rather not be posting into the void".
  
  For instance, I make YouTube Shorts for a game I play. I don't post them on Lemmy anymore, because the Lemmy community for the game only has 60 subscribers, most of whom aren't even active accounts anymore. The highest-upvoted thread in the community has 47 votes, the second-highest only has 9. This translates to effectively nobody on Lemmy seeing the videos I made, because this small, slow-paced community's posts get drowned out by everything else.
  
  source
  - Skavau@piefed.social ⁨7⁩ ⁨hours⁩ ago
    What’s the game? I feel like a broader community for “YouTube Shorts for a games people play” would have more successful than a community tailored to that specific game.
    
    source
- PumpkinDrama@reddthat.com ⁨9⁩ ⁨hours⁩ ago
  I think it is, and niche has lost to shitposting. Also, don’t tell me what to do.
  
  source
Inaminate_Carbon_Rod@lemmy.world ⁨3⁩ ⁨hours⁩ ago
I’d love to see more action in the Magic the Gathering communities.

source
PumpkinDrama@reddthat.com ⁨11⁩ ⁨hours⁩ ago
Compute raw post score (upvotes minus downvotes).

Normalize score by community size (e.g., divide by square root of monthly active users).

Calculate z-score relative to community mean and standard deviation.

Apply time decay to prioritize recent posts.

Sort posts by adjusted z-score.

Outcome: Posts that significantly outperform their community norm appear prominently, giving small and large communities equal visibility potential.

Enhancements: Minimum engagement thresholds, Bayesian shrinkage for small communities.
source
- trajekolus@piefed.social ⁨10⁩ ⁨hours⁩ ago
  I’m not sure how Scaled is determined, but as far as I know, it is made for what you want
  
  source
  - PumpkinDrama@reddthat.com ⁨10⁩ ⁨hours⁩ ago
    Nah it only rewards communities with few members, that means bot communities with lot of posts and almost no active users are always going to be top. That’s not the outliers sorting that I want.
    
    source
- rimu@piefed.social ⁨8⁩ ⁨hours⁩ ago
  That’s an interesting algorithm!
  
  IMO the Lemmy scaled sort would be ok if it wasn’t as powerful.
  
  Try the scaled sort on PieFed.social, the amplification isn’t as strong and I just tweaked it to filter out bot posts.
  
  https://piefed.social/home/scaled/all
  
  source
Jayjader@jlai.lu ⁨9⁩ ⁨hours⁩ ago
“Top over the last 6 hours” can be a decent middle ground between “new”, “scaled”, and “active”.

source
mrdown@lemmy.world ⁨10⁩ ⁨hours⁩ ago
I think crossposting from time to time may help

source
sanity_is_maddening@piefed.social ⁨8⁩ ⁨hours⁩ ago
Please. No algorithm, please.

And I don’t use the Hot or Popular filters because I think they do possess some of the same undertones of that perverse incentive that algorithmic social media has. It would be advisable to kill this before the lemmyverse grows too wide.

Just chronological in all options: Home. Local, and All. But allowing to filter any of them through themes or subject. Piefed is already toying really well with some of these concepts.

This alone allows a lot more of the other small instances and communities to be seen. And removes those perverse incentives to “climb” that corrupting ladder that makes some people post controversial and deranged posts that seek the reward of getting more attention. Algorithms rewarded this. The Hot and Popular dilter options of sifting through them rewards this too. And we all know it. And we are seeing its’ effect already. In an ideal world this wouldn’t be a problem. But we know these will keep being hijacked to that end.

And these don’t help the small and the thoughtful communities to arise as we want them to.

I personally set my home page to be the list of the communities I follow and decide what I’m in the mood for. The voyager app is also great for this. And all are set to chronological.

So summing up, my suggestions are:

-kill the Hot and Popular filters. Chronological feed only.

-Add subject filters instead.

-List of Communities followed as homepage. It let’s you know what you’re in the mood for.

These will help everything including the smaller instances and communities to be seen. This isn’t about my preferences. I do this to remove my own tendencies to fall prey to the rage bait catching my attention as a starting point. Because I’m no better. That’s the point. Media Experts as soon as 2008 said this ingrained reward loop is a problem because disgust and rage are the reactions that track the most immediately and lock on in the history of all media. If we want Lemmy to be better, we have to make it so. This already doesn’t have ads and algorithmic suggestions to nurture engagement, which is great. So this is the only thorn that remains from the dark legacy of social media. Nobody would be shutting up anybody. It’s just not rewarding shitposting. If I can disable it myself, the space at large is still falling prey to it as I would if I was using it.

PS: Also, Crossposting obviously helps smaller instances and communities to get noticed too.

source
- Skavau@piefed.social ⁨8⁩ ⁨hours⁩ ago
  You realise if Lemmy/Piefed even doubled in size, /new/ would just be useless. Just a wave of low quality posts, spam, and topically non-relevant posts to most people.
  
  It doesn’t scale.
  
  source
  - sanity_is_maddening@piefed.social ⁨7⁩ ⁨hours⁩ ago
    I know. But is scale all that matters? I know what you mean and I swipe through a lot of them.
    
    But that also happens with the Hot and Popular filters. Because if I just swipe through the All option across all the federated instances using the Hot or Popular filters it’s mostly memes, circlejerking and rageposting. Is this what is wanted? Because nobody needs Lemmy for that. Reddit and all of the other platforms are already serving that low level of engagement with slop to spare.
    
    This actively rewards it. Exactly the same way.
    
    I don’t remember who said it …"Scale is the death of virtue”… but it is definitely applicable here.
    
    I don’t know what you want out of lemmy, but if that outcome is what you intend to have, the end product will have virtually no distinction of its’ counterparts and people will see no reason to change. As there will be none outside of it’s not “corporate owned”. But if it’s still slop, people who want a change, will just skip the whole thing. As I did for a while. And if it wasn’t for Piefed, I don’t think I would be still hanging around here anymore either.
    
    Beware that a lot of the people who seek these alternatives do not want anything similar to what is huge out there. And you’ll essentially be killing the appeal to the larger target audience that will bother to “learn” federation and how instances operate in its context.
    
    This is a hard bargain. And I do understand what you mean, and I have to suffer through it. But if you don’t recognise what I’m speaking of, then I don’t think we’re having a honest conversation.
    
    I have not used those filters in a long time. But I’m betting that the enraged posting about what’s happening in the U.S. (and justifiably so), shitposting, circlejerking, memes and Tankies causing controversy again are taking the entire feed until one gets bored. If I’m wrong I bet is not by much. But hell, I could’ve stayed on reddit for that back when I was there which was a while ago.
    
    Meanwhile the communities with intellectually engaging posting, real propositions of solutions and thoughtful discussions get slided to nowhere like on every other platform. They’re here and they are incredible. But guess what, they’re also on reddit, youtube, instagram and so on. And they get even more traction than in here. And also no traction in comparison to everywhere else. Just like here. But they do still get more people than here in the end. So what is the appeal of changing?
    
    I know a lot of people who want to make ideas like Syntropy, Permacomputing, the circular economy and many other wonderful things known, and these communities do exist around here and are wonderful, but they’re even smaller than everywhere else. The proportions in ratio in everything remain the same here as in every other platform. “Hey, we don’t have far-right wing nuts and that many transphobes here!” - true. But it doesn’t seem like we actually use the space to do anything but complain about them and make the real solutions and discussions as invisible as everywhere else. It’s absolutely soulcrushing.
    
    The reward of rage and disgust still leaves on. And we get to nothing different with it. The loop that made our world even more deranged is not broken.
    
    Hey, if this is what everyone wants, I’ll just leave if I’m the only one bothered, no need to change anything on my account. But judging by the interactions I had over the two years since I’ve been around (first on lemm.ee and now on piefed), I can tell you a lot of people come here for the opposite of everywhere else. There’s a lot of older, reflective, literate individuals with a lot to share with others and eager to learn that come looking around these parts, and I stop seeing their names around because I think they eventually give up on what is essentially the same invisibility to high value conversations and the reward of rage and disgust as the media experts kept warning us about and nobody listened.
    
    If you read this far, I thank you for your attention.
    
    I hope you don’t think I don’t want the fediverse to succeed, I absolutely do. But not at any cost. And especially not at the cost of what can make it truly valuable as a difference and an actual change in direction.
    
    source
    -> View More Comments
- scrubbles@poptalk.scrubbles.tech ⁨8⁩ ⁨hours⁩ ago
  Algorithm is not a dirty word. Any sorting or filtering is an algorithm. Just because you don’t like something doesn’t mean we need to kill it off, just don’t use them.
  
  source
  - sanity_is_maddening@piefed.social ⁨6⁩ ⁨hours⁩ ago
    That was unnecessarily condescending…
    
    I know “algorithm” is not a dirty word. Algorithms are used almost everywhere for almost anything. But when we mention " the Algorithm” colloquially these days we know that we’re talking about algorithmic influence in engagement in digital platforms. And how it can have unintended consequences or dark incentives structured within them.
    
    Which was precisely why I was talking about these filters. Not from a place of like/dislike but because of its inherent reward system and how it rewards rage and disgust, exactly as media experts warned everyone it would with the algorithms at the core of the other platforms.
    
    And if you are denying and saying that the same is not already occurring here, you’re being disingenuous in this exchange. As one only needs to go through their feed with these filters enabled on navigating across all the federated instances to see the rageposting and disgust at the top. The few exceptions will only prove the rest is the rule.
    
    It’s frustrating, because people are not engaging with what I wrote, and are only replying deflections instead of addressing the direct criticism at the core of my argument.
    
    I’m not insulting anyone, I’m not throwing shade at people for enjoying memes, as I do to. I’m merely pointing out the perverse incentive lying at the core of this feature.
    
    And it doesn’t matter if I don’t like it, I can just not use it, because it’s the enabled option in everyone that uses that will be driving their behaviour and fulfilling the loop with its dark incentives.
    
    Why are people not engaging with what I’m saying? I even said “I’m no better” because I will fall prey to it if I leave it on, or even if I feel tempted to go back to it from time to time. That is why I don’t.
    
    But I still see the posts that get the most interactions and comments.
    
    This is not an armless feature. This is what broke the civic nature of discussion in society through social media. This loop of incentive will always lead to rage and disgust to raise above all else. And reward derangement in people that seek attention for attention sake.
    
    I don’t know what else to say…
    
    I’m sorry if I insulted anyone by pointing this out. But I don’t see how my comment can be misconstrued as a personal attack on anyone. As it is just an avaluation of a proposition through its inherent consequences. This is basic level of scientific method right here.
    
    This is not about liking/disliking. I really dislike coffee, and I have offered a coffee machine as a birthday gift to a friend. More than once, actually. I can have people enjoy things I dislike with no hint of an issue. It’s even baffling to me when others don’t know how to.
    
    This is about adressing an issue at its core. And out it influences behaviour and the user base at large.
    
    source
    -> View More Comments
- ShaggySnacks@lemmy.myserv.one ⁨8⁩ ⁨hours⁩ ago
  
  PS: Also, Crossposting obviously helps smaller instances and communities to get noticed too.
  
  Honsetly, crossposting is an amazing feature. The vast majority of my posts are just me crossposting posts to another community. It’s low effort-high value. It takes a few minutes to bring new content to smaller communities.
  
  source
BradleyUffner@lemmy.world ⁨7⁩ ⁨hours⁩ ago
It’s Lemmy. Their all small communities…

source
mesamunefire@piefed.social ⁨10⁩ ⁨hours⁩ ago
Maybe search by tags? I know pieces allows you to do that. And Mastodon.

Either that or make your own search via api and exclude any community with more than X subscribers. Probably would work well.

source
- PumpkinDrama@reddthat.com ⁨9⁩ ⁨hours⁩ ago
  I’m fine just using the hot sort or scaled sort in my subscribed communities. I don’t participate much since I mainly post in niche communities, and Lemmy tends to smother those, so I have to keep using Reddit.
  
  source
Tracaine@lemmy.world ⁨10⁩ ⁨hours⁩ ago
Compete with what? Maybe the small communities should just not suck.

source
- trajekolus@piefed.social ⁨10⁩ ⁨hours⁩ ago
  Seriously?
  A lot of small communities are small because they are niche or because they are new
  
  source
- Chozo@fedia.io ⁨7⁩ ⁨hours⁩ ago
  It's a signal-to-noise issue. There are some smaller communities I'd like to keep an eye on, but the posts from those communities get drowned out by the more active ones. I miss a lot of posts that I would have liked to have seen because of this.
  
  source