CrowdStrike Outage Debrief: The Importance of MDR

In today’s digital landscape, ensuring your organization is protected from cybersecurity threats is essential, but the process doesn’t end at prevention. It’s just as important to have a team on standby and ready to respond when things go wrong. Even the most robust solutions can encounter issues, and having a Managed Detection and Response (MDR) plan in place is crucial to mitigate the fallout from any cybersecurity issues that might arise. This was made especially clear during the CrowdStrike outage earlier this year, where organizations that had a comprehensive response plan in place fared far better than those without one.

At SYSCOM, we specialize in ensuring our customers are prepared to handle the inevitable and the unexpected. The CrowdStrike incident provides a perfect example of why having an MDR solution, and basic security measures like patch management, is indispensable, and how a well-prepared team can make all the difference.

CrowdStrike Outage: What Happened

Earlier this year, CrowdStrike, the leading EDR (Endpoint Detection and Response) solution, experienced a significant outage that impacted organizations worldwide. At 1 AM EST on July 19, 2024, CrowdStrike pushed out a patch intended to improve their system. Unfortunately, that patch contained a critical bug that caused all connected devices to crash, resulting in the dreaded “blue screen of death.”

For organizations whose devices were online at the time, this meant immediate lockout. Systems were entirely inoperable, and remote IT teams were powerless; devices could only be fixed in-person, one by one. Devices that were turned off when the patch was pushed, like those in time zones where it was nighttime, only received the corrected patch when they turned on their systems later. But for those who were already on and received the first path, the damage was already done.

The outage caused significant disruptions, forcing many businesses into crisis mode. Systems were frozen, and operations ground to a halt as IT teams scrambled to understand the issue and respond.

How SYSCOM Responded

While some organizations were left waiting for CrowdStrike to diagnose and address the problem, SYSCOM was able to react swiftly. Our first alert didn’t come from CrowdStrike itself, but rather from our management team at the Tokyo branch, thanks to our global presence and proactive approach. We immediately mobilized our team to assess the scope of the problem.

We didn’t just wait for the official fix from CrowdStrike. We tapped into the global CrowdStrike community of engineers around the world, already at work crowdsourcing a solution much quicker than could be achieved through official channels. Rather than waiting for information to trickle down from official channels, we watched the solution unfold in real time.

As soon as a fix was found, we internally tested the effectiveness and validated the solution, then quickly communicated with our customers, explaining what had happened and outlining the steps we were taking to resolve the issue. Organizations without an MDR team like ours were left waiting for public updates, and their systems remained down much longer.

Our response illustrates something we always tell our customers: having an EDR is not enough. You must also have an MDR, such as SYSCOM, to manage it. This was especially true in the case of the CrowdStrike outage, as it was the EDR itself that failed.

The Importance of MDR Service Providers

The CrowdStrike incident highlights a crucial point: no matter how robust your cybersecurity and incident response infrastructure may be, errors, bugs, and failures entirely outside your control will still happen. This wasn’t a cybersecurity breach, but a software error that froze entire systems. The reality is that even the most trusted software can fail, and when it does, how quickly you respond determines how well you mitigate the issues, minimize the impact on the business, and limit financial loss.

CrowdStrike is an EDR provider, and their technology excels at detecting and responding to cybersecurity threats at the endpoint level. However, while an EDR identifies the problem, you need an MDR solution, staffed by a team like our Security Operations Center, to take action. In fact, in 2023, the Biden administration signed Executive Order 14028, encouraging all federal agencies to have an EDR in place. This time, the issue came from the EDR itself, which makes the case for having an MDR service all the more pressing.

The difference between EDR and a traditional antivirus is often misunderstood. Unlike antivirus software, which passively scans and removes threats, EDR is actively monitoring your environment for threats in real time. But without a team to manage it, an EDR solution is incomplete.

It’s important to plan not just for preventing incidents, but for how you’ll respond when they occur—and they will occur. That’s why many cybersecurity insurers require that you have an EDR solution in place to be compliant. But to truly make the most of it, you need an MDR plan, operated by an SOC team like ours, to manage those responses and ensure that when things go wrong, there’s a clear plan to immediately take action.

Customer Impact and Recovery

SYSCOM prides itself on a customer-first approach, and that commitment was evident during the CrowdStrike incident. Thanks to our swift response and effective recovery strategies, more than 95% of our customers were back up and running within a single day, a validation of our response plan and a testament to the preparedness of our team.

We always recommend that our clients have a regular data backup policy, Disaster Recovery Plan, and a Business Continuity Plan (BCP) in place. A BCP is essential for minimizing the impact of any incident, whether it’s a software bug or an all-out cyberattack. When our strategy is implemented proactively, we’re able to help customers recover quickly, mitigating the impact on their business and ensuring operations can resume with minimal disruption.

Looking Forward

While we’re proud of our response to the CrowdStrike outage and view it as a case study of how to effectively respond to a service outage affecting business-critical operations, we’re always looking for ways to improve. One area we’ve identified for growth is communication, and we are currently running internal drills to ensure even smoother coordination in future incidents.

And while CrowdStrike still remains a top-tier EDR solution, we’re also testing Microsoft Defender for Endpoint as an additional offering, giving our customers the flexibility to choose which of these industry-leading EDR solutions works best for their unique needs.

However, the real message here is that the specific EDR provider you choose is less important than having an MDR service operated by an SOC team to manage it. The digital landscape has become too complex to guarantee that incidents like this won’t happen again. What matters is how prepared you are to respond when they do.

An MDR service, like the one we provide at SYSCOM, ensures that when a major issue arises, your business can recover quickly with minimal damage. In cybersecurity incidents, cost efficiency is a key criteria, and any interruption in your operations can disrupt payroll, business opportunities, new business acquisition, and more. A well-prepared incident response plan can minimize financial loss, protect your organization’s reputation, and get you back to business as quickly as possible.

The CrowdStrike incident was a wake-up call for many, and it underscores the need for a proactive approach to cybersecurity management. Whether you’re a current client reflecting on our CrowdStrike response, or a potential client considering switching providers, our response to this incident demonstrates that SYSCOM has the expertise, resources, and proactive strategy to keep your business protected.

Reach out today to discuss how we can help your organization be better prepared for whatever the future holds.

Continue Reading

read more