Why is data congregation so hard on Mastodon?

Submitted ⁨⁨18⁩ ⁨hours⁩ ago⁩ by ⁨noodlesreborn@lemmy.world⁩ to ⁨fediverse@lemmy.world⁩

This applies to any of the microblogging software. Akkoma, IceShrimp, etc. I go to any Lemmy instance, big or small, and the up/downvote data and replies are basically all the same. The same goes for Peertube, and most services that aren’t Mastodon and the gang. Why is this? Is it because of older design? Unexpected issues cropping up with scale? It seems to be such a big struggle over there, but for everyone else, it’s whatevs.

I would love to permanently reside on a smaller Mastodon instance or host my own, but I often find that many posts are unavailable and a lot of replies I want to reply to don’t exist. It is an incredibly frustrating experience.

source

Comments

Sort:hotnew top

julian@community.nodebb.org ⁨18⁩ ⁨hours⁩ ago
Lemmy and lots of other software use a fediverse extension called 1b12 to keep everything in sync.

In a nutshell it means Lemmy communities can follow other communities, and they keep each other in sync. The same applies for other types of communities, like PieFed communities, Mbin magazines, NodeBB categories, etc.

Mastodon doesn't have a concept of community or categories, so they don't support this kind of synchronization.

source
- noodlesreborn@lemmy.world ⁨17⁩ ⁨hours⁩ ago
  Thank you, this makes sense. Is there any hope for Mastodon and other services to achieve a similar level of parity without eating up a ton of space? I feel like it is a big hurdle for Fedi, but I understand these things take time and a ton of work.
  
  source
  - julian@community.nodebb.org ⁨17⁩ ⁨hours⁩ ago
    Sure, check out my post about it here:
    
    https://community.nodebb.org/topic/18844/backfilling-conversations-two-major-approaches
    
    There are steps being taken in the right direction.
    
    source
    -> View More Comments
muntedcrocodile@hilariouschaos.com ⁨17⁩ ⁨hours⁩ ago
Its all about pub sub relations. Being on a smaller instance in mastodon means u miss a whole bunch of activity and thus don’t have the full picture.

Lemmy and others have a community which means a community receives an activity and then that community pushes that activity to all its subscribed instances. So in lemmy u need at least 1 user from ur instance to subscribe to a community for its activity to be sent to ur instance.

On mastodon all activity by a user is sent to all instances with at least one user who follows that user. So if nobody on ur instance follows a specific user then their content will never show up on ur instance. Its a difficult problem to solve as u can’t just send all activity to all instances cos that will overload most instances. Their are relays that relay activities from its member instances to other instances but as u would expect the larger the relay the more content u must deal with on ur instance and u can get overloaded pretty quick.

The whole point of federation is to send activities only to instances that need them as not to force everyone to have the entire fediverse on their server and handle and activities from the entire fediverse (one of the reasons bluesky federation doesn’t work for small instances).

source
voidspace@lemmy.world ⁨13⁩ ⁨hours⁩ ago
I started my own GoToSocial instance this week, and found the same. TBH, from my exported list of about 160 people I from followed on Mastodon, I only re-followed 6.

These were all longstanding relationship - a couple of which started on Twitter when it was OK.

source