I agree that scaled sort is not working ideally right now. However the issues you linked were opened before the feature was released, so they are not relevant anymore. You are definitely welcome to open a new issue about improving the implementation. The scaled sort logic is here, if you have any concrete suggestions how to improve it thats even better. And I would rather improve this existing functionality to make it work as expected, before tacking on an entirely new feature which may or may not work.
[deleted]
Submitted 9 months ago by The_Lemmington_Post@discuss.online to fediverse@lemmy.world
Comments
nutomic@lemmy.ml 9 months ago
The_Lemmington_Post@discuss.online 9 months ago
[deleted]nutomic@lemmy.ml 9 months ago
There is already such an API endpoint which is available for mods and admins.
TropicalDingdong@lemmy.world 9 months ago
Emperor@feddit.uk 9 months ago
Personally, I believe the best way to address this issue is through the implementation of tags and custom feeds.
I expressed my enthusiasm for tags over in The Great Monkey Tagging Army discussion and have had a chance to ponder this some more - I still like the idea:
- It provides context and meaning to posts - just think of all the disambiguation pages on Wikipedia. Are you looking for Predator the film, the UAV, etc? Really only tags can help there - multitags or disambiguation? Or both? There are discussions to be had.
- It allows more depth to things like communities - I was just talking to someone who had been failing to find a place to talk about a specific British comic until they found !britishcomics@feddit.uk. With tags, I could have tagged the community with #dandy, #beano, #2000AD, #bunty, etc.
- It also provides a way for Lemmy to integrate better with the Fediverse as it doesn’t currently play well with services outside the Threadiverse.
My current thinking is leaning towards FediTags - a separate, federated plug-in service. So you could run it with Lemmy, Pixelfed, etc and it would bring together all the tagged content. You look up stonehenge + photographs and get pictures on Pixelfed, Mastodon and kbin. As a plug-in the changes in code needed to, for example, Lemmy would be minimal and, hopefully, a one-time thing. Development of FediTag could then take place completely separately.
nutomic@lemmy.ml 9 months ago
Tagging would make sense to categorize posts within a single community. But you seem to suggest tags which are shared across communities. I dont really see the point of this, as communities themselves are already used for a global “tagging” of posts. So it would only duplicate that functionality.
The_Lemmington_Post@discuss.online 9 months ago
[deleted]ada@lemmy.blahaj.zone 9 months ago
Reddit isn’t going to magically get better though. Reddit is dying and will only get worse with time, continually shedding users as time goes by.
So making noise about lemmy, and making suggestions on how to prove it is a good thing, because unlike reddit, lemmy might actually get better with time.
mesamunefire@lemmy.world 9 months ago
Reddit was like this when it started. I was there when Dig was a competitor. It took a decade of constant posts by people who really loved their communities to really take off. And reddit, when it started, didnt have subreddits, it was closer to hackernews.
Not saying your arguments are wrong per-say. But it will take some time to get started. The good news is federation means different instances will come and go but the protocol means we dont need to stay on any one kind of server. Kinda like email. I use mastodon to talk with peertube creators and use lemmy to talk to other lemmy instances all the time.
Blaze@dormi.zone 9 months ago
Mbin has that. They released a new version 2 weeks ago I’ll have a closer look soon, seems very promising
halm@leminal.space 9 months ago
I can see the draw of subscribing to and blocking different keywords — adding another taxonomy to the existing communities. Mastodon and similar microblog projects are ahead in that regard, and not only in terms of muting or following hashtags. I would definitely welcome more user settings on Lemmy so we could curate feeds granularly on an individual level.
The freedom from advanced algorithms really demands that users be the wardens of whatever other sorting comes next, rather than Lemmy devs leaving things be or Meta-like algorithms creep in because of familiarity.
OpenStars@startrek.website 9 months ago
Step 1: it would be nice if we could at least talk about this in a friendly & civilized manner. I have spent a portion of my day today trying to defend even so much as casually mentioning in passing - in a reply to a reply to a reply even, iirc, much less a full-on post - that I would like something similar to what you said. I give up whenever I detect that someone literally did not read what I already wrote, at which point I see that they just wanted to complain rather than add something of substance to the ongoing conversation. And even if we took it for granted that I was a dummy Mc-Stoopid-buttface, nobody bothered once to explain why I might be wrong.
i.e., there seems to be significant push-back to this approach. I have no idea why though - it seems entirely logical and do-able to me? Especially if it were purely optional, like a new sort option rather than taking over the existing Hot one? At a guess, it may just be a difficult task, so it awaits someone to be interested enough to actually implement it. Also, please remember that the entire Fediverse has and continues to be under perpetual attack (message from DMV.social closing down due to being spammed by illegal CP & CSAM amid concerns over the ethical considerations of being a server that allows posts from external users, i.e. the entire Federation model, quoting: “Quite frankly, this is disturbing and I just don’t want to deal with the possibility of this crap.”) - I do not know if it is Huffman, or Musk, or Zuckerberg, or whoever might not enjoy how this could potentially take away from their profits, but they are correct that if we continue to exist on our own, that we need to do something to protect ourselves against this type of thing. So… sorting is important yes, but I could see if it was not the HIGHEST priority, right now.
But moving on, one thought regarding it: allow each user their own customization filters for each “category” of posts, e.g. 1% politics, 2% sports, restrict news to 5% (though the latter requires significantly deeper thinking to implement - e.g. is an article in a Technology sub still “news”? tbh, “news” is probably not a real category then). Or, as you say, an algorithm that would just work mostly fine for most people. The problem with all of this being that tags would have to exist first, so someone would have to develop that before any of this could begin to be developed and tested.
Which brings us back to: it is really fun to talk about such matters, but ultimately it will take someone rolling up their sleeves first, maybe learning an entirely new language (or several - according to this GitHub page, Lemmy is: “Rust 76.4% PLpgSQL 16.4% TypeScript 5.5% Shell 1.5% Other 0.2%”), and just getting something done. Otherwise, beggars cannot also be choosers, if there is nothing else available to choose from.
The_Lemmington_Post@discuss.online 9 months ago
[deleted]Excrubulent@slrpnk.net 9 months ago
This is interesting. So sorting is done server-side, I assume this is for performance. In fact now that I think of it, for a client to implement a custom sort it would need to fetch basically every single post for the entire period, across the entire fediverse, which really doesn’t sound practical. At that point the client has effectively become its own local server.
So that means sorting algorithms need to be implemented server-wide.
I wonder if this could be done with plugins that servers could experiment with and modify more easily, rather than being baked into full version updates?
OpenStars@startrek.website 9 months ago
I feel the need to be pedantic here: that quote continues on to the very next (and final) sentence being:
Not something that can be done with a simple plugin.
However, anything that is logically possible, is doable, with enough effort & investment - e.g. that infamous quote:
All that quote means is that it would be most simple to do as a back-end task, not a simple front-end one (though even a front-end could, in theory, e.g. slurp up 1000 posts and then use some metric to figure out how to display the most proper subset of 20 from that superset).
But for instance, someone could spin up their own instance, and then add whatever sorting method they wanted to it. However, recall again what happened to DMV.social - anyone who opens up a Lemmy server will be spammed with CSAM, it would seem - so there are other more urgent matters to be attended to first, unless that someone used it purely as a testbed, and made all connections to it to be read-only, or else had a team of moderators willing to put in large amounts of time to fend off those attacks with both manual efforts and also developing automated tools at the same time - e.g. they would need to have technical skill even just to moderate, much less administer the machine (unless, like existing Lemmys, there is a whole team of admins doing the technical parts already). Anyway, I don’t suggest this lightly like it is trivially easy, just to say that it is possible.
It would be beneficial to talk more about these desirable features to ensure that when developers do invest time in them, we’ve already come up with a good and robust solution.
Sure, I am not trying to tell you what to do. Just stating that until and unless someone is willing to tinker with actual implementations - and again, right now their attentions seem to be directed elsewhere, plus while Scaled-sorting Hot may not be perfect it is something (I don’t personally have experience to say if it is better than before b/c I was on Kbin.Social at the time which was totally different - but I thought I heard many people say that it is better now?) - then it is going to be a purely theoretical discussion. Which is probably how Scaled sort got implemented too? Though now that it is built, it could be tinkered with, if there is the will to do so.
But unless you are offering to do any of the actual implementation work yourself, I think you would need to discuss this with the actual admins who you would expect to do that work for you - hence you might try Matrix where they hang out, rather than solely discussing it here.
And then, as you said in your OP, when they say “no” and close all GitHub issues, that, as they say, is that. You can’t “force” someone to do work for you for free - and even if you were offering money, or perhaps offering to do all the “design” work yourself for free, they still would need to agree to actually do the implementation.
Moreover, even if you DID offer to do ALL of the implementation work entirely on your own, unless you do want to spin up your own instance to actually run it on, you still would need the buy-in of the instance admins, for which having the buy-in of the developers would go a LONG way.
So you asked:
Do you have any ideas or suggestions on how Lemmy could better surface content from smaller communities?
And my suggestion is that you cannot walk into someone else’s house and tell them how they should do things. Especially when they have ALREADY said no. They know better what their prioritization is, and what they hope to accomplish over the next month, year, and so on. The absolute beauty of the Fediverse is that you can take all of the existing Lemmy code, which is entirely functional, make a fork of it, and spin up your own instance - and not just run it, but even modify the code to do… whatever you want! And then you can share that code, and benefit all the instances that are running Lemmy too! Discuss.Online, Lemmy.World - all of them, well, those that choose to keep your suggestions, though it is up to each one individually to either accept or reject them, and it is ultimately their call. Reddit does not work this way, nor FaceBook/Meta/Threads, Instagram, Xhitter, etc., but we do, b/c it is “open-source”. The caveat to that being… that someone, somewhere, must put in the actual effort to get it done.
And the people who would normally do that, seem to have said no. I gather that you feel frustrated but… it is what it is. Therefore, of what use is it to talk about any of this, when there is no path forward for it? THAT is how to move forward your ideas: either find or become someone yourself who can implement them, and THEN in talking to them you will actually be in an even better position to understand how it all works, and how it might be changed to work even better than it does now.
I dunno, perhaps I should not have replied at all? Sometimes I do overshare my thoughts and if you disliked that here then I apologize:-). It was my hope to help spur your thoughts along these lines that I was thinking, since it seems to me to be the only way forward. But I guess please ignore me if you think I am wrong, and I wish you luck either way!:-)
Fwiw, I do agree that eventually, when the developers are ready to move forward with this again, they indeed might appreciate a ready-made solution if one happened to be already available by then, but again that assumes that one could be made purely on theoretical grounds alone?
lambalicious@lemmy.sdf.org 9 months ago
Any metric that ultimately depends on frequency and variety of posts is going to bury niche communities more or less by design.
Dunno what we need, honesdly, but I would venture that one thing that could help would be like, for exxample, StackOverflow’s “Unanswered” view, where you can check which topics have not garnered conversation attention. It would probably have to be tuned so that it specifically ignores 0-comment posts that are also links, because those tend to be reposts, news or stuff like that that’s easy to spam.
SnokenKeekaGuard@lemmy.dbzer0.com 9 months ago
I just want the ability for people to deploy their own sorting systems. Then we could have opensource projects and everyones happy (ok maybe not, but thats as close as we get to it)
The_Lemmington_Post@discuss.online 9 months ago
SnokenKeekaGuard@lemmy.dbzer0.com 9 months ago
This is not possible because sorting is done in the database, so adding a new sort option requires a database migration with new indexes, columns and updated queries. Not something that can be done with a simple plugin.
Ah that’s a shame