Looking at the downmeter shot someone posted above, it’s half the SREs in the country. Not sure what the root cause will be, but damn that’s a lot of money down the tubes. I would not want to be the person who cost Meta and Google their precious thirty 9’s of availability lol.
Comment on Facebook and Instagram are currently down.
cbarrick@lemmy.world 8 months ago
Pouring one out for the SREs at Meta
alilbee@lemmy.world 8 months ago
ObviouslyNotBanana@lemmy.world 8 months ago
Which country?
SpaceNoodle@lemmy.world 8 months ago
Sokovia
marcos@lemmy.world 8 months ago
Nah, what was that muddy country from Dilbert?
alilbee@lemmy.world 8 months ago
The country where all of those services are maintained and hosted in… Just colloquial shorthand, not trying to be exclusionary.
ObviouslyNotBanana@lemmy.world 8 months ago
Ok thanks for the clarification!
merc@sh.itjust.works 8 months ago
The country where all of those services are maintained and hosted in…
For Meta, Google, etc. that’s a number of countries all over the world.
don@lemm.ee 8 months ago
Krakozhia
merc@sh.itjust.works 8 months ago
It’s likely there’s a root cause, like a fiber cut or some other major infrastructure issue. But, Down Detector doesn’t really put a scale on their graphics, so it could be that it’s a huge issue at Meta and a minor issue that’s just noticeable for everyone else. In that case, Meta could be the root cause.
If everyone is mailing themselves their passwords, shutting their phones on and off, restarting their browsers, etc. because Meta wasn’t working, it could have knock-on effects for everyone else. Could also be that because Meta is part of the major ad duopoly, the issue affected their ad system, which affected everyone interacting with a Meta ad, which is basically everyone.
alilbee@lemmy.world 8 months ago
I’ve been an SRE for a few large corps, so I’ve definitely played this game. I’m with you that it was likely just the FB identity or ad provider causing most of these issues. So glad I’m out of that role now and back to DevOps, where I’m no longer on call.
merc@sh.itjust.works 8 months ago
Yeah. And when the outage is due to something external, it’s not too stressful. As long as you don’t have absolutely insane bosses, they’ll understand that it’s out of your control. So, you wait around for the external system to be fixed, then check that your stuff came back up fine, and go about your day.
I personally liked being on call when the on-call compensation was reasonable. Like, on-call for 2 12-hour shifts over the weekend? 2 8-hour days off. If you were good at maintaining your systems you had quiet on-call shifts most of the time, and you’d quickly earn lots of days off.
guacupado@lemmy.world 8 months ago
Second half is the closest answer.
Semi-Hemi-Demigod@kbin.social 8 months ago
Hopefully they won't need to cut their way into the data center this time.
ALostInquirer@lemm.ee 8 months ago
What’s a SRE?
el_abuelo@lemmy.ml 8 months ago
Site reliability engineer
lightnegative@lemmy.world 8 months ago
Google terminology leaking its way into mainstream
cbarrick@lemmy.world 8 months ago
Meta has SRE. Not just a Google thing, even if Google invented it.
kinther@lemmy.world 8 months ago
Someone is having a really bad day today. I wonder if your phone dies when you get a certain number of pages or push notifications
Alk@lemmy.world 8 months ago
Fun story. I had a flip phone years ago and you could have multiple recipients to a single text. And if the text was multiple pages, it would split into several texts. And you could resent already sent texts.
So one time I put in my girlfriend’s phone number in all 20 recipient slots. I then filled the text to the max size, though I don’t remember how many it split into. I then resent it over and over. This all took like 2 or 3 minutes.
Her phone was sending notifications over and over for the entire rest of the day. I’d guess at least 8 hours, probably more.
18_24_61_b_17_17_4@lemmy.world 8 months ago
Fucking hell I used to love doing that! Man that brought back some memories. Would do it to my co-worker and just piss myself laughing.
lightnegative@lemmy.world 8 months ago
The classic txt bomb. I use to do this if I had unused txts at the end of the month
maynarkh@feddit.nl 8 months ago
No but it’s unusable. I had a weird bug on one of my phones that sent an SMS over as fast as it could as long as the phone was on. I wrote the initial SMS, the contents were something like “hey, wanna hang?”, and the poor guy on the other side was blasted for several hours of literally constant notifications.
Luckily my plan at the time had unlimited free SMS.