CrowdStrike Falcon Sensor Crash Triggers Global IT Outage, Emergency Workaround Released

A critical crash error in CrowdStrike’s Falcon Sensor platform has caused widespread IT disruptions across the globe, affecting critical services like 911 call centers, airlines, banks, and major media outlets.

Over the past 24 hours, numerous users reported experiencing sudden system crashes, primarily on Windows hosts, accompanied by the infamous Blue Screen of Death (BSOD) error. The crashes were linked to the Falcon Sensor, a crucial component of CrowdStrike’s endpoint protection suite.

CrowdStrike, a leading cybersecurity firm renowned for its cloud-based security solutions, acknowledged the issue and rapidly initiated an investigation. Their engineering team traced the cause to a recent content deployment and promptly reverted the changes.

Emergency Workaround:

For systems still experiencing crashes and unable to receive the updated channel file changes, CrowdStrike has issued a workaround:

1. Boot Windows into Safe Mode or the Windows Recovery Environment
2. Navigate to the C:\Windows\System32\drivers\CrowdStrike directory
3. Locate the file matching “C-00000291*.sys”, and delete it.
4. Boot the host normally.

For cloud environments, customers can revert to a snapshot taken before 4:09 am UTC.

For AWS (Amazon Web Services), follow these steps:

1. Detach the EBS volume from the impacted EC2 instance.
2. Attach the EBS volume to a new EC2 instance.
3. Fix the CrowdStrike driver folder.
4. Detach the EBS volume from the new EC2 instance.
5. Attach the EBS volume back to the impacted EC2 instance.

For Azure, follow these steps:

1. Log in to the Azure console.
2. Go to Virtual Machines and select the affected VM.
3. In the upper left of the console, click “Connect”.
4. Click “More ways to Connect” and then select “Serial Console”.
5. Once SAC has loaded, type in ‘cmd’ and press Enter.
6. Type ‘ch -si 1’ and press the space bar.
7. Enter Administrator credentials.
8. Type the following commands:
   • ‘bcdedit /set {current} safeboot minimal’
   • ‘bcdedit /set {current} safeboot network’
9. Restart the VM.
10. To confirm the boot state, run the command: ‘wmic COMPUTERSYSTEM GET BootupState’.

The manual nature of this fix poses a significant challenge for companies, especially those without backups for all VDIs, potentially slowing down the recovery process. Customers will also need a recovery key to access Safe Mode if Bitlocker is enabled on the system disk.

Global Impact:

The repercussions of the Falcon Sensor bug have been far-reaching, causing significant operational disruptions across multiple sectors. Emergency services, financial institutions, transportation networks, and news organizations have all been impacted, highlighting the critical role cybersecurity plays in modern infrastructure. While CrowdStrike’s Falcon platform is designed to protect systems from malicious attacks, the bug inadvertently turned it into a source of instability.

CrowdStrike is actively working to address the issue and prevent similar incidents in the future. The company is urging all affected users to apply the workaround or update their Falcon Sensor to the latest version.