Ongoing federation issues

Comments

• No1@aussie.zone ⁨1⁩ ⁨year⁩ ago

  I’m claiming federation issues as the reason my posts aren’t getting thousands if upvotes.

  source

  • maniacalmanicmania@aussie.zone ⁨1⁩ ⁨year⁩ ago

    I still can’t believe you’re not president of the world yet. Like wtf?

    source

    • No1@aussie.zone ⁨1⁩ ⁨year⁩ ago

      Careful, or I’ll turn to the dark side and federation will become a dictatorship so fast it’ll make Stalin spin in his grave 🤣

      source
• lodion@aussie.zone ⁨1⁩ ⁨year⁩ ago

  ☹️

  I’ll see what I can see…

  source

  • lodion@aussie.zone ⁨1⁩ ⁨year⁩ ago

    Ok, for some reason practically all instances were flagged as “dead” in the database. I’ve manually set them all to be requeued… server is now smashed as it attempts to updated the ~4000 instances I’ve told it are no longer dead. See how this goes…

    source

    • lodion@lemmy.world ⁨1⁩ ⁨year⁩ ago

      Replying from my lemmy.world account, seems to have caught up eventually.

      source

      • -> View More Comments
    • lodion@aussie.zone ⁨1⁩ ⁨year⁩ ago

      Seems to have sorted it for the most part… not sure what caused it, will do some more digging.

      source
    • Zagorath@aussie.zone ⁨1⁩ ⁨year⁩ ago

      Boss!

      source
• lodion@aussie.zone ⁨1⁩ ⁨year⁩ ago

  Ok, something is busted with the lemmy API endpoint that shows current federation state. It is currently showing nearly all remote instances as dead:
  Image

  But “dead” instances are still successfully receiving content from AZ, and sending back to us.

  source

  • lodion@aussie.zone ⁨1⁩ ⁨year⁩ ago

    Ok, there’s more to this than I first thought. It seems there is a back end task set to run at a set time every day, if the instance is restarting at that time the task doesn’t run… this task updates the instances table to show remote instances as “seen” by AZ. With the memory leaks in 0.19.1, the instance has been restarting when this task is running… leading to this situation.

    I’ve updated the server restart cronjob to not run around the time this task runs… and I’ve again manually updated the DB to flag all known instances as alive rather than dead.

    Will keep an eye on it some more…

    For anyone curious, two of the bugs that are related to this:
    github.com/LemmyNet/lemmy/issues/4288
    github.com/LemmyNet/lemmy/issues/4039

    source

    • Zagorath@aussie.zone ⁨1⁩ ⁨year⁩ ago

      Thanks for the update!

      source
• unionagainstdhmo@aussie.zone ⁨1⁩ ⁨year⁩ ago

  I’ve noticed a few of my comments are federating either

  source

  • synae@lemmy.sdf.org ⁨1⁩ ⁨year⁩ ago

    Seems to be working now!

    source
• trk@aussie.zone ⁨1⁩ ⁨year⁩ ago

  Cant argue with any of those examples.

  In a delicious cherry on top, this thread is also not federating (ex: lemmy.world/c/meta@aussie.zone)

  source
• maniacalmanicmania@aussie.zone ⁨1⁩ ⁨year⁩ ago

  This post’s body text is best viewed on desk/laptop for anyone trying to make heads or tails of all the links on mobile.

  source

  • Zagorath@aussie.zone ⁨1⁩ ⁨year⁩ ago

    Yeah fair call. I’ve edited it into a table which should hopefully be easier to read.

    source