Okay so you know, bunch of shitposting and all that, but a serious interlude:

Someone pushed the button to start this rollout. They are probably having a _really_ bad time right now.

If someone at Crowdstrike knows who that is, please go and check on them, give them a hug, tell them it's not their fault, that it's going to be okay. No matter what the company line is on blameless culture whatever, the lizard brain is in charge right now and needs reassurance.

Also, quick note for crowdstrike execs: everyone can see you looking over at that bus, considering your options, limbering up your throwing arm... Just a note that the people you probably want to hire are watching reeeally closely how you're going to handle this, and are taking notes. The shareholders may be into human sacrifices, but the people you need to run your business aren't. Choose wisely.

@danderson seems worthy of #HugOps tag, no? 🙂

@danderson I deployed an approved change one Friday night in 2008 to every National Australia Bank workstation.

It removed the core banking software. That was ... not the intent.

I spent 12 hours pushing the new software out to every site (~2,000), and writing a script to force reboot every workstation (~30k+). Before 08:00. With the major incident management line going the entire time. (They were great.)

I feel for whomever did this today. Shit happens.

@pinkforest @johnnydecimal @danderson Because we live in open source supermarket ATM ;-)

Back in the day when software was not working, you read the manual, and learned all the details, and finally mastered software.

These days something not working as expected:

1. Install a plugin
2. Switch the software and join the rival camp.

Quite often plugins just dumb down the feature that was pre-existent already in the original software, if the manual was read ;-)

@danderson Serious question: Is there gonna be a CrowdStrike in the future? Won't they be sued to oblivion?

@klefstadmyr @danderson one can hope

@danderson as the person responsible for the 2020 RHEL bootloader update that soft-bricked something like 5% of machines with a very dumb bug (which... relies upon a memory layout that no Intel chipsets claim to support, but that's neither here nor there...), I do appreciate this perspective a great deal. Thanks for saying so.