Comment on CrowdStrike downtime apparently caused by update that replaced a file with 42kb of zeroes

• Gork@lemm.ee ⁨1⁩ ⁨month⁩ ago

  Ah, makes sense. I guess a driver would completely freak out if that file gave no instructions and was just like “…”

  source

  • PriorityMotif@lemmy.world ⁨1⁩ ⁨month⁩ ago

    You would think that Microsoft would implement some basic error handing.

    source

    • planish@sh.itjust.works ⁨1⁩ ⁨month⁩ ago

      That’s what the BSOD is. It tries to bring the system back to a nice safe freshly-booted state where e.g. the fans are running and the GPU is not happily drawing several kilowatts and trying to catch fire.

      source
    • Kaboom@reddthat.com ⁨1⁩ ⁨month⁩ ago

      For most things, yes. But if someone were to compromise the file, stopping when they see it invalid is probably a good idea for security

      source
• deadbeef79000@lemmy.nz ⁨1⁩ ⁨month⁩ ago

  Except “freak out” could have various manifestations.

  In this case it was “burn down the venue”.

  It should have been “I’m sorry, there’s been an issue, let’s move on to the next speaker”

  source

  • Strykker@programming.dev ⁨1⁩ ⁨month⁩ ago

    Except since it was an antivirus software the system is basically told “I must be running for you to finish booting”, which does make sense as it means the antivirus can watch the system before any malicious code can get it’s hooks into things.

    source

    • Morphit@feddit.uk ⁨1⁩ ⁨month⁩ ago

      I don’t think the kernel could continue like that. The driver runs in kernel mode and took a null pointer exception. The kernel can’t know how badly it’s been screwed by that, the only feasible option is to BSOD.

      The driver itself is where the error handling should take place. First off it ought to have static checks to prove it can’t have trivial memory errors like this. Secondly, if a configuration file fails to load, it should make a determination about whether it’s safe to continue or halt the system to prevent a potential exploit. You know, instead of shitting its pants and letting Windows handle it.

      source
  • the_crotch@sh.itjust.works ⁨1⁩ ⁨month⁩ ago

    In this case it was “burn down the venue”.

    It was more like “barricade the doors until a swat team sniper gets a clear shot at you”.

    source

    • deadbeef79000@lemmy.nz ⁨1⁩ ⁨month⁩ ago

      Hmmmm.

      More like standing there and loudly shitting your pants and spreading it around the stage.

      source
  • tiramichu@lemm.ee ⁨1⁩ ⁨month⁩ ago

    You’re right of course and that should be on Microsoft to better implement their driver loading. But yes.

    source

    • Morphit@feddit.uk ⁨1⁩ ⁨month⁩ ago

      The driver is in kernel mode. If it crashes, the kernel has no idea if any internal structures have been left in an inconsistent state. If it doesn’t halt then it has the potential to cause all sorts of damage.

      source
  • OozingPositron@feddit.cl ⁨1⁩ ⁨month⁩ ago

    Computers have social anxiety.

    source
  • Thann@lemmy.ml ⁨1⁩ ⁨month⁩ ago

    The envelope contains a barrel of diesel and a lit flare

    source
• sigmaklimgrindset@sopuli.xyz ⁨1⁩ ⁨month⁩ ago

  Great layman’s explanation.

  source
• digdilem@lemmy.ml ⁨1⁩ ⁨month⁩ ago

  Nice analogy, except you’d check the script before you tried to use it. Computers are really good at crc/hash checking files to verify their integrity, and that’s exactly what a privileged process like antivirus should do with every source of information.

  source
• Imgonnatrythis@sh.itjust.works ⁨1⁩ ⁨month⁩ ago

  Maybe. But I’d like to think I’d just say something clever like, “says here that this year the pummel horse will be replaced by yours truly!”

  source

  • Takios@discuss.tchncs.de ⁨1⁩ ⁨month⁩ ago

    Problem is that software cannot deal with unexpected situations like a human brain can. Computers do exactly what a programmer tells it to do, nothing more nothing less. So if a situation arises that the programmer hasn’t written code for, then there will be a crash.

    source

    • deadbeef79000@lemmy.nz ⁨1⁩ ⁨month⁩ ago

      Poorly written code can’t.

      In this case:

      1. Load config data
      2. If data is valid:
         1. Use config data
      3. If data is invalid:
         1. Crash entire OS

      Is just poor code.

      source

      • 5C5C5C@programming.dev ⁨1⁩ ⁨month⁩ ago

        When talking about the driver level, you can’t always just proceed to the next thing when an error happens.

        Imagine if you went in for open heart surgery but the doctor forgot to put in the new valve while he was in there. He can’t just stitch you up and tell you to get on with it, you’ll be bleeding away inside.

        In this specific case we’re talking about security for business devices and critical infrastructure. If a security driver is compromised, in a lot of cases it may legitimately be better for the computer to not run at all, because a security compromise could mean it’s open season for hackers on your sensitive device. We’ve seen hospitals held random, we’ve seen customer data swiped from major businesses. A day of downtime is arguably better than those outcomes.

        The real answer here is crowdstrike needs a more reliable CI/CD pipeline. A failure of this magnitude is inexcusable and represents a major systemic failure in their development process. But the OS crashing as a result of that systemic failure may actually be the most reasonable desirable outcome compared to any other possible outcome.

        source

        • -> View More Comments
      • ChairmanMeow@programming.dev ⁨1⁩ ⁨month⁩ ago

        If AV suddenly stops working, it could mean the AV is compromised. A BSOD is a desirable outcome in that case. Booting a compromised system anyway is bad code.

        source

        • -> View More Comments
      • Takios@discuss.tchncs.de ⁨1⁩ ⁨month⁩ ago

        I agree that the code is probably poor but I doubt it was a conscious decision to crash the OS.

        The code is probably just:

        1. Load config data
        2. Do something with data

        And 2 fails unexpectedly because the data is garbage and wasn’t checked if it’s valid.

        source

        • -> View More Comments
  • Hazzia@infosec.pub ⁨1⁩ ⁨month⁩ ago

    I’m gonna take from this that we should have AI doing disaster recovery on all deployments. Tech CEO’s have been hyping AI up so much, what could possibly go wrong?

    source

    • Couldbealeotard@lemmy.world ⁨1⁩ ⁨month⁩ ago

      What are the chances that Crowdstrike started using ai to do their update deployments, and they just won’t admit it?

      source
• Cocodapuf@lemmy.world ⁨1⁩ ⁨month⁩ ago

  I’m nominating this for the “best metaphor of the day” award.

  Well done!

  source
• JasonDJ@lemmy.zip ⁨1⁩ ⁨month⁩ ago

  The ironic bit is, I’m sure more than a few people at Crowdstrike are preparing 3 envelopes right now.

  source
• crystalmerchant@lemmy.world ⁨1⁩ ⁨month⁩ ago

  This guy ELI5s

  source
• CeeBee_Eh@lemmy.world ⁨1⁩ ⁨month⁩ ago

  Ah yes. So Windows is the screaming in terror version and other systems are the “oh, sorry everyone, looks like there’s an error. Let’s just move on to the next bit” version.

  source