It rather abruptly stops at a few thousand users and after that it becomes much harder and more expensive to scale further.
As a fellow Lemmy admin of a smaller instance, do you have any advice? Any resources that might be worth checking out?
Comment on I reckon this is the usage distribution of Lemmy servers that we'll end up with.
poVoq@slrpnk.net 5 weeks agoAs someone who run a Lemmy server I can tell you that it isn’t as simple as that.
Yes, there is an initial benefit from having more users on an instance, but this initial scaling benefit isn’t linear. It rather abruptly stops at a few thousand users and after that it becomes much harder and more expensive to scale further. Only after going over that hump it might become cheaper again at the scale of hundred thousand of users or so, but Lemmy the software is currently also unlikely to scale as a single instance to such numbers, so it isn’t just a system operator question.
So no, unless you wsnt to fully commercialize the Fediverse and bring in external investors to fund the getting over that initial bumb, semi-centralisation is not a feasible way forward. And what would even be the point of that? Reddit exists and is basically the same.
Luckily ActivityPub is designed to scale horizontially through lots of smaller (but not tiny) instances, so I think we can manage without the above.
SorteKanin@feddit.dk 5 weeks ago
poVoq@slrpnk.net 5 weeks ago
Well, like I wrote below, it is basically about scaling Lemmy across multiple servers, with all the complexity that entails. I am probably not the best person to given advise on that though.
If our smaller instance ever gets to a size that is not feasible to run on a single server anymore, we will likely close registrations.
SorteKanin@feddit.dk 5 weeks ago
Thanks. Personally I haven’t been impressed either by the stability and performance of Lemmy. It is what it is I guess.
LesserAbe@lemmy.world 5 weeks ago
My argument isn’t about the fediverse specifically. It’s that centralization is a naturally occurring phenomenon, and the lack of friction resulting from centralization can make it more competitive.
What is the reason the cost per user of hosting a Lemmy server goes up after a few thousand users? If it were say, you need more expensive hardware, that doesn’t necessarily disprove my argument. Just because a bigger investment is needed doesn’t mean it’s not cheaper per user or not more competitive. Just that you or I don’t have the capital, or that we might see centralization bad because we have bad experiences with centralized entities.
Also just because something is more competitive doesn’t mean it’s morally or aesthetically more desirable. The specialized army fed and trained by an empire overruns the brave and happy tribe of hunter gatherers.
What I’m saying is since we know the phenomenon of centralization occurs, we should try to subvert it as much as possible by introducing democratic structures.
poVoq@slrpnk.net 5 weeks ago
Lets agree to disagree on the “natural” centralisation aspect, which is IMHO nonsense. And very recently the US empire was beaten by some tribes in Afghanistan, so I think your argument needs some further thinking 😏
The reason it gets so much more expensive after a few thousand users is complexity. Up to that point a single server can be used and the necessary sysadmin skills are not very high. Basically anyone with a few weeks of training can rent a server and run such an instance.
After a few thousand users it gets steeply more complex, when you need to think about running a database cluster and load-balance the frontends etc. Not very many people have the necessary skillset for that, and even less are volunteering to do this. So you end up being forced to hire someone expensive with a high in demand skill. Basically you operation suddenly jumps from a easy to fund with donations volunteer effort, to a must commercialize or otherwise fund venture that is highly unsustainable in the short term.
LesserAbe@lemmy.world 5 weeks ago
I don’t think it’s unreasonable to argue centralization is a naturally occurring phenomenon. It’s everywhere. The U.S. left Afghanistan and was replaced by a different centralized entity. One could argue how decentralized those “tribes” were, but regardless, after the U.S. departure they recreated a similar structure.
Complexity comes hand in hand with size. The OP is a chart of the different email providers. Can an individual run their own email server? Yes. And doesn’t it get more difficult after a certain number of users and require hiring specialists? Yes. But still, such large services exist, and a majority of users turn to them.
If the fediverse lives there will always be small servers, but we can expect to see really big ones. If we don’t want them to be corporate recreations of gmail and yahoo and hotmail I’d argue we should figure out a platform co-op/worker co-op model, including the necessary funding and specialists.