Comment on Is my domain "burnt" when hosting my first Fediverse technology?
squaresinger@lemmy.world 2 days agoThe design choice to hard-bind an instance to a host name still baffles me. It’s an incredibly brittle choice. It makes it pretty much impossible to ever move an instance to another hostname.
Sadly, the fediverse is full of amateurish design choices because it was designed by hobbyists who apparently don’t have anything but a very basic understanding of distributed systems.
Also the concept of “Every instance needs to keep a full copy of everything” and “Every instance has to re-moderate everything to not be legally liable for illegal content” is really bad for scalability.
sugar_in_your_tea@sh.itjust.works 2 days ago
Yeah, I really don’t get the “everything stores a copy of everything” model. It should instead work like a cache, where the OG instance is the source of truth, and instances just keep a cache of that data. Instances should be able to refresh data, or have no cache at all.
I get the desire to not lose data, but that comes with a huge storage cost. If we want redundancy, we should have dedicated caches instead of everything having a copy.
But hey, the Fediverse exists and I’m too lazy to build something better, so here I am.
squaresinger@lemmy.world 2 days ago
Yeah, with the current “everything replicates everything” model Lemmy is close to the workable limit of users.
Currently, there are roughly 50k monthly active users on Lemmy, and the hosting cost is approaching unsustainability for hobby instances with a decent amount of users. Monetizing Lemmy is close to impossible with donations being the only real revenue stream, so there’s pretty much no business case for anything but a hobby instance.
If for some reason even just 1% of Reddit users were to migrate to Lemmy (that would be ~10mio monthly active users) Lemmy would instantly crumble under that load no matter how many new instances would be added since every instance stores everything.
Moderation would also all but collapse since each instance needs to moderate everything as well, due to legal reasons. (If someone posts something illegal on a remote instance and it gets replicated to your instance and you don’t delete it, you are legally liable for it since it’s stored on your server.
Max_P@lemmy.max-p.me 1 day ago
Technically it wasn’t really designed with megainstances in mind that swallows the entire fediverse.
My instance has no problem whatsoever keeping up and storage is well under control. But we’re few here subscribed to a subset of available communities so my instance isn’t 90% filled with content I don’t care about and will never look at. Also reduces the moderation burden because it’s slow enough I can actually mostly see everything that comes through.
Lemmy itself is also pretty inefficient in that regard, you can very much make software that pulls instead and backfill local cache as needed.
Even my Reddit subscriptions would be pretty easy on my instance.
squaresinger@lemmy.world 1 day ago
You need really small instances for that to do something. The issue here is not only mega instances, but more curcially mega communities.
If people on your instance subscribe to the top 50 communities you already have more than 50% of the whole lemmy traffic on your instance. And 50 subscriptions isn’t all that much for even a single user.
And mega communities is kinda the whole point of any reddit-like service. The really cool thing about reddit is that no matter how obscure the topic, there’s a subreddit for it with experts in the field. Lemmy is still lacking that for most topics, but that would be where a real Reddit alternative would want to end up.
If you have a look at reddit, they have over 1000 subreddits with over a million subscribers each. Every single one of these subreddits has around 200x the traffic of all of Lemmy combined. So if Lemmy were to grow to Reddit levels and a single user subscribes to a single community like that, your whole instance is cooked.
sugar_in_your_tea@sh.itjust.works 2 days ago
Yup, and this has been my chief concern since I came to Lemmy 2-3 years ago and read the implementation details. And it’s not something that can be patched in easily or I’d work on it, it’s a fundamental design choice.
I began working on a distributed alternative, but quickly ran into issues in the design phase that Plebbit is currently running into: moderation is a tough nut to crack. I have ideas on how to mostly solve it, but between a full-time job and young kids, it just hasn’t been a priority.
I hope someone with more time than me can tackle it, especially since I’m not 100% confident in my own solution.
squaresinger@lemmy.world 2 days ago
It’s a worthy and huge endeavor.
I think the only somewhat sustainable way to get around the moderation problem is to get rid of storing a copy of everything.
That way you don’t have incriminating data on your server and then you just mark every external community as “out of bounds of my moderation, there be dragons, go at your own risk” and call it a day.
If a Lemmy/ActivityPub alternative was designed from the ground up a decent option would be to limit federation to a single-signon and private messages. In that case when you visit a remote community, the client directly goes to that remote community to fetch data from there.
Basically like a set of separate forums with a federated login.
That would solve the “everything copies everything” issue and the “everyone has to moderate everything” issue as well. If someone posts illegal crap on a remote instance, that data stays on that remote instance and you aren’t responsible for them. And the users can themselves decide what communities on what instances fit to what they want to look at.
That would mean that if an instance goes down their communities do as well, but that’s (at least to me) less of an issue than the current state. It’s not like these zombie communities work fine right now. With the source instance being down, federation is gone and thus posting on these instances means there’s only a fraction of the audience left.