SecurityMetrics

Detection Coverage as a Number, Not a Posture: A Methodology for Buyers Tired of Heat Maps

What percentage of real attack techniques does your stack detect? A methodology for turning detection coverage into a number instead of a heat map.

A CISO who can't quote a single coverage number to her board is operating on faith. Most CISOs in 2026 still can't. The question isn't unanswerable. The field has just substituted heat maps for measurement, and the heat maps have been good enough to keep procurement writing checks.

The question is simple. What percentage of the attack techniques a real adversary would use against my environment gets detected by the stack I bought? The answer is a percentage. That's the number. The board wants the number, the auditor wants the number, and the CFO comparing renewal quotes from three vendors wants it too. None of them are getting it.

So I want to propose a way to produce that number, and to be specific about why the current substitutes don't. The usual three are ATT&CK heat maps, vendor coverage claims, and audit checklists. They each fall short for different reasons.

Why ATT&CK heat maps mislead

The MITRE ATT&CK framework is the best inventory we have of adversary behavior. It's also a list, and lists don't produce coverage percentages. They produce coverage checklists, and a checklist is a different animal than a number.

Walk any vendor's heat map. You'll find a green cell next to "T1059.001 PowerShell" and conclude PowerShell abuse is covered. The cell is green because the vendor has at least one detection that fires on at least one variant of PowerShell abuse. Green doesn't promise the detection fires on the variant your actual attacker is running this quarter. It doesn't promise the detection fires before the technique completes. And it says nothing about whether the detection survives a moderately competent evasion attempt.

Green isn't coverage. Green means "we have something." Decisions made on green miss the question that matters.

The framework has a deeper problem too. ATT&CK enumerates techniques, not weighted threat models. A heat map that's 80 percent green by cell count tells you the vendor has detections in 80 percent of the rows. It tells you nothing about whether those rows are the ones a real adversary would actually use against you. The 20 percent of empty cells might be exactly the 20 percent that matter.

What an actual coverage number requires

A defensible coverage percentage has four properties.

One. It's computed against techniques weighted by relevance to the buyer's environment. A coverage percentage that bundles in industrial control system techniques means one thing to a hospital and something else entirely to a power plant. The weight is environment specific. So buying decisions should be made on environment specific weights, not on uniform technique counts.

Two. It's computed against attempted exploitation, not against signatures that match a known artifact. The real question is whether the stack alerts when an attacker performs the technique, not whether it alerts on a specific artifact the technique once produced. Adversaries swap artifacts constantly. The techniques themselves change far less. Track the technique.

Three. It's computed against attack variants, not single instances. A detection that fires on the textbook example and folds against three reasonable evasion variants isn't coverage of that technique. It's coverage of one phrasing of one variant. Real coverage tests against multiple phrasings, multiple evasion variants, and chained sequences.

Four. It's computed under realistic load. A stack that flags the technique cleanly in a quiet lab and misses it in production, with thousands of concurrent legitimate sessions running, isn't actually deployed coverage. Measure under representative volume. This is the hardest requirement to meet, which is why it's the one teams skip most.

A worked methodology

Here's the methodology I'd propose, and I think it should become a buyer side standard inside the next twelve to eighteen months. Six steps.

Step one. Define the environment specific threat model. A list of techniques weighted by likelihood and impact for the specific buyer. The list doesn't have to be perfect. It has to be opinionated. A hospital's list looks nothing like a defense contractor's. Get it on paper and signed by the CISO before any vendor evaluation starts.

Step two. For each weighted technique, generate at least three attack variants. A textbook example, a moderate evasion variant, and an advanced one. Modern offensive AI tooling makes generating these tractable in a way it wasn't five years ago. A small offensive AI deployment can produce a high quality variant set in a day.

Step three. Run every variant against the candidate stack in a representative environment. Representative means production volume of legitimate traffic, production tuning on the detection rules, production response time SLOs. Not a pristine lab. The lab measurement is interesting only to the vendor's marketing team.

Step four. Compute three numbers per technique. Detection rate against textbook variants. Detection rate against moderate evasion. Detection rate against advanced evasion. Multiply each by the technique's weight, sum across techniques, divide by total weight. That's your weighted coverage percentage.

Step five. Compute the trend. A single point measurement is interesting. A trend is actionable. Run the methodology quarterly, track the percentage over time, and watch the gap between textbook and advanced evasion. That gap is a leading indicator of how the stack will hold up against next quarter's threats.

Step six. Publish the methodology, not just the number. A coverage percentage you can't audit isn't credible. Buyers should publish the methodology behind their internal numbers, share the numbers with their boards, and challenge their vendors with both. A vendor who can't replicate your measurement is selling green cells.

What the trend looks like in practice

I've been running a version of this methodology against representative networks for roughly eighteen months. The pattern holds. The textbook variant detection rate is high, often above ninety percent. The moderate evasion rate drops meaningfully, often into the sixty to seventy percent range. The advanced evasion rate falls off a cliff, often below thirty percent.

That gap between textbook and advanced evasion is the part that matters. A stack with a 90 percent textbook rate and a 25 percent advanced evasion rate looks excellent on the heat map and is brittle against a real adversary. A stack with a 75 percent textbook rate and a 60 percent advanced evasion rate looks weaker on the heat map and is far more durable in the field.

The buyer who optimizes for the heat map ends up with the first stack. The buyer who optimizes for the gap ends up with the second. Most boards don't know to ask which one they bought.

What this implies for procurement

The procurement question for detection technology in 2026 should be precise.

What's your detection rate against the OWASP top ten, weighted to my environment, computed against at least three evasion variants per technique, measured under realistic load, with the methodology documented well enough that I can reproduce it?

A vendor who answers that with numbers is doing the work. A vendor who hands you a heat map is selling 2018. The price difference between the two should be real money. The procurement officer who sees this saves money and gets better security. The one who doesn't pays a premium for green cells.

A closing observation

The detection coverage conversation has been allowed to stay qualitative far too long. Part of that is the number being uncomfortable for vendors to produce honestly. Part of it is buyers never insisting.

Buyers who insist get better answers. Vendors who can answer win procurement. Buyers who don't insist keep making security decisions on heat maps that don't survive contact with a real adversary.

The number exists, it's computable, and it isn't flattering on average to most current detection stacks. That's exactly why it should be the procurement standard.

← All Posts

Bring the Data Center to the Workload.

Talk to us about deploying Atlas for your mission.

Contact Us