Setanta wrote on Jul 20
th, 2024 at 11:17pm:
ProudKangaroo wrote on Jul 20
th, 2024 at 11:14pm:
We had a script to rename the whole croudstrike folder and run a reg script as some devices has different paths, but everything critical is alive again, the rest is a job for Monday.
Long night, I'm actually crashing pretty hard right now, dunno why I'm still here haha.
Could you do it via GPO or did you have to do it individually?
Individually.
This was to stop the BSOD when the system files for Crowdstrike load on startup.
Devices weren't even getting to a login page so it was crashing and boot looping before you'd have a chance to utilise active sessions and GPOs.
We were able to do all servers remotely given they're all either VMs or iLO/iDRAC etc.
The HyperV fleet was a pain given they took down multiple VMs at once, which is what killed so many Azure Servers with Microsoft.
Nothing you can do downstream from there until they fix that part, then you hope if you were using CS, which for many cyber insurance plans is mandatory, your VM wasnt also impacted.
The scale is really scary but it could have been so much worse.
Getting physical access with a local admin password was all you need.
If it would have required restoration from last backups, give most houses don't have robust recovery plans or even test their backups, would would have been a nightmare.
Lots of reevaluations happening in a lot of places next week....