Windows Crashes BSOD Caused by CrowdStrike Falcon Sensor – How to Fix

Overview

Recently, several Windows hosts have experienced crashes due to issues related to the CrowdStrike Falcon Sensor. This has caused significant disruptions, with symptoms including bugcheck and blue screen errors. The issue has not affected Mac or Linux-based hosts. CrowdStrike Engineering has identified the root cause as a content deployment error and has since reverted the changes. However, some systems may still require manual intervention to resolve the issue.

Details of the Incident

On July 19, 2024, reports surfaced about Windows hosts crashing due to the Falcon Sensor. Affected systems displayed bugcheck or blue screen errors, making them unable to stay online long enough to receive the reverted channel file automatically. CrowdStrike has provided a workaround to manually fix the issue for systems that continue to crash.

🚨 Global CrowdStrike Outage: Technical Breakdown 🚨

Recently, a faulty update from CrowdStrike triggered a worldwide outage, resulting in numerous system crashes (BSOD). Let’s dive into an accessible analysis of what happened:

📉 The Root Cause

The issue stems from the CSAgent.sys file, a critical component of CrowdStrike’s security solution. An instruction attempted to access an invalid memory address, leading to the crashes (and not a null pointer, mind you).

🔍 The Role of “C-…sys” Files

Rather than being drivers, the configuration files named “C-…sys” contained obfuscated and poorly formatted data. These files were referenced by CSAgent.sys, and their incorrect format triggered a logical error, causing system crashes.

🚀 Uncontrolled Update

Compounding the situation, this update bypassed standard client deployment controls. It was rolled out globally without proper validation, rapidly propagating the issue.

🛠️ Temporary Fix and Permanent Solution

Removing the faulty files was found to restore normal system operation. CrowdStrike promptly acknowledged the problem and is working on a fix to prevent such errors from recurring.

📊 Conclusion

This outage underscores the importance of rigorous controls and validation during update deployments. It also highlights the complexity and risks associated with managing large-scale security systems.

Workaround Steps

If your Windows host continues to crash, follow these steps to manually delete the problematic file and restore normal operations:

1. Reboot the Host
   • Try rebooting the host to allow it to download the reverted channel file. If the host crashes again, proceed to the next steps.
2. Boot into Safe Mode or Windows Recovery Environment

3. Access the VMware vSphere Client or VMware Workstation and open the remote console for the affected virtual machine (VM).

4. As the VM starts, repeatedly press the Shift + F8 keys to enter the Advanced Boot Options menu.

Make sure you have Admin Rights, or alternatively, boot into Safe Mode with Networking to connect to a Domain Controller using an account with privileged admin rights, assuming that the Active Directory is not also impacted by the BSOD.

5. Select « Safe Mode » or « Safe Mode with Command Prompt » and press Enter.

6. Navigate to the Problematic File

• Open the Command Prompt if not already in it.
• Type the following command and press Enter:
  • cd C:\Windows\System32\drivers\CrowdStrike

7. Delete the Faulty File

• Locate the file matching the pattern C-00000291*.sys by typing:
  • dir C-00000291*.sys
• Delete the file by typing:
  • del C-00000291*.sys

8. Reboot the Host Normally

• After deleting the file, restart the VM.
• The host should now boot normally without crashes.

Note for BitLocker-Encrypted Hosts

If your host is BitLocker-encrypted, you may be prompted to enter the BitLocker recovery key during this process. Ensure you have access to the recovery key before proceeding.

How do I Recover Bitlocker Keys?

For Azure Active Directory:

Under ENTRA ID DEVICES

If you don’t know the name of the device and you have the Bilocker Key I you can search directly the Bitlocker Key from the Bitlocker Keys (preview)

Or simply search about the concerned devices under All Dvices

Click on the specific device and select “Show Recovery Key”

Checking Active Directory for Recovery Keys

1. Open the Active Directory Users and Computers snap-in
2. Navigate to the computer object in question
3. Right-click the computer object and select Properties
4. Go to the BitLocker Recovery tab and view the recovery key

Support

For further assistance and support, visit the CrowdStrike Support Portal.

Additional Resources

For a more detailed guide on bypassing this issue using VMware, refer to our tutorial:

1. Activate the Force Boot Option in vCenter
   • Access the vCenter and enable the force boot option for the affected VM.
2. Enter Safe Mode
   • Press Shift + F8 during boot to access the Advanced Boot Options menu.
   • Select « Safe Mode. »
3. Navigate and Delete the File
   • Navigate to the C:\Windows\System32\drivers\CrowdStrike directory.
   • Delete the C-00000291*.sys file.
4. Reboot in Normal Mode
   • Restart the VM and boot normally.

For more updates and articles on Windows host management, visit msndpoint.com.

Stay tuned for the latest in endpoint security and troubleshooting tips.