Improsec has performed numerous maturity assessments and red team exercises for Danish companies throughout the years. Many of these companies had a managed SOC service, which performed from poorly to disastrous. This blog is based on our experiences of the performance of both national and international SOC providers.
Here are the top-ten red flags, that alert you of your managed SOC provider doing a poor job.
Red Flag #1: No new use cases?
How to tell: The service provider does not regularly tell you about new use-cases or has not shared information about how they update their use-case catalogue.
What it actually means: No new use cases get implemented. The service provider most likely does not spend resources developing their capabilities. At most, they defend you against the threats of yesterday.
What you ideally want: Continuously updated catalogue of use-cases that matches the current threat landscape and your risk appetite.
Red Flag #2: New domain controller?
How to tell: Is there a process in place that describes how the service provider will be notified when you install a new Windows Domain Controller in your environment?
What it actually means: The service provider is clueless about what happens in your environment. There are many processes and integrations that should be defined when you outsource your SOC. A mature provider will tell you this. Without this in place, it is tough to muster even a mediocre defense.
What you ideally want: A set of well-defined and tested processes.
Red Flag #3: Got telemetry?
How to tell: Do you collect Sysmon logs from your Windows endpoints?
What it actually means: The vast majority of intrusions begin at the endpoints. Without collecting the proper telemetry, the service provider is unable to detect much, even if they try their best.
What you ideally want: Collect Sysmon (or equivalent) from all Windows endpoints.
Red Flag #4: AV is king!
How to tell: Your provider relies on your security products to alert on intrusions. This is typically endpoint protection products or networks-based IDS/IPS. No other client monitoring is being performed.
What it actually means: The service provider has not even created their own set of use cases for endpoint monitoring. See also #3.
What you ideally want: A service provider that masters detection engineering.
Red Flag #5: Don’t be ignorant – be enriched!
How to tell: Logs are not being enriched.
What it actually means: Each type of log has a “security business value”. Its value is equivalent to how useful it is. Many log types can be enriched, raising their value substantially. If you are not enriching, you are missing out.
What you ideally want: Enrich external IP addresses with ASN numbers. Enrich DNS lookups with domain age – just to name a few.
Red Flag #6: The journey…
How to tell: Your service provider talks about the journey you are on together.
What it actually means: They do not have an actual SOC service yet. They want you to pay for the development. Note: having 50 monitors in a dimly lit room does not equal a managed SOC. See also #1 and #4.
What you ideally want: A mature service provider.
Red Flag #7: Garbage collection
How to tell: Has the service provider told you how to filter out unnecessary logs?
What it actually means: You are probably collecting a lot of useless data. It is not in their interest to tell you this, because they typically charge you by amount (gigabyte, event per second, etc.).
What you ideally want: Collect high-value data, leave the garbage.
Red Flag #8: The reporting
How to tell: You fall asleep during the monthly/quarterly status meetings. See also #10.
What it actually means: A SOC can provide an organisation with a lot of tangible intelligence and inputs for improvements, on both the tactical and strategic level. If your service provider does not do this, you are missing out.
What you ideally want: A service provider that helps you drive security forward within your organisation.
Red Flag #9: Show it, don’t tell it
How to tell: The service provider “talks the talk”: Threat Hunting, MITRE ATT&CK, Pyramid of Pain….. but do not “walk the walk” by going into further details exactly how they do it.
What it actually means: They are not doing it.
What you ideally want: Someone that elaborates on “why”, “how”, and “what”.
Red Flag #10: Nothing to report – we are secure!
How to tell: The service provider does not find much.
What it actually means: The service provider does not have a current use case library (see #1), they are probably clueless about what happens in your environment (see #2), and as they do not collect the right telemetry (see #3), they have to rely on security products to tell them what is going on (see #4). Conclusion: You have a compliance SOC.
What you ideally want: It is contractionary, but you want them to discover a lot of bad stuff.
In summary, look at your managed SOC offering. How well does it fare against the red flags mentioned above? Is it good enough?The threat landscape is continuously evolving. Can your SOC keep up?
You find more info on Improsec and our services here: Services Overview — Improsec | improving security