Matrix.org - Post-mortem of the September 2 outage
“Unfortunately, in attempting to do so, we erroneously deleted the data directory of the primary on db-01.”
Wow, mistake on top of mistake on top of mistake it’s impressive they could recover without data loss.
jeena@piefed.jeena.net 2 days ago
One cool thing is that I had no idea that this happened until I just read it on their blog even though me, my friends and family use Matrix extensively.
But all my friends who are on Matrix host their own servers because it’s quite easy, and my family uses my own server, only my brother uses a matrix.org account, but he doesn’t write much.
This is decentralization working as it’s supposed too, when enough participants are federated and not centralized.
On Lemmy the lemmy.world and the piefed.social instances are similar to matrix.org and I think all of them, including mastodon.social should close new registrations. I mean the flagship instances have their place in the beginning, but once they become so big they should be locked. The teams behind them can open another instance if they want to keep growing, but they should be run on a separate infrastructure to prevent bringing them down at the same time.
SnotFlickerman@lemmy.blahaj.zone 2 days ago
I always heard that rolling out an initial Matrix instance isn’t terrible, especially with ansible and/or docker, but I also have heard that a lot of updates have breaking changes and that updating your server is less simple.
As someone who runs your own Matrix server, would you agree with this sentiment or disagree. I have considered rolling out my own many times but get discouraged by those who say keeping it updated is kind of a nightmare. For example, a private tracker I am a member of used to have a Matrix server and an IRC server, but they eventually dumped the Matrix server entirely to reduce complexity as well as the fact that fewer people used it.
Opisek@piefed.blahaj.zone 2 days ago
I run my own matrix server, too. I’ve never had any issues with updates. Honestly, I just pull the new docker image without thinking twice about it. It is risky, of course, but I have daily automated backups, so I’m not too concerned. Personally, I felt like setting up a matrix server for the first time was the biggest pain in the butt. It extremely convoluted with very poorly written documentation that is often outdated or incomplete. After I got through that, it was smooth-sailing from there. Setting up mautrix bridges one you got Synapse to work is actually really easy, though.
superglue@lemmy.dbzer0.com 2 days ago
I run it in docker and an update has never caused issues. Been about 3 years.
jeena@piefed.jeena.net 2 days ago
I run it on Ubuntu and installed it with apt about 7 years ago and had zero problems, it updates itself when I run apt upgrade, it turns itself off for a couple of seconds and then it’s back up.