Detection rules rot, and nobody notices

Ask a security team how good their detection coverage is and you will usually get a count. Nine hundred rules. Eighty percent of a framework’s techniques mapped. A dashboard with a lot of green.

Now ask a harder question: of those nine hundred rules, how many have fired even once in the last quarter, and of those, how many produced a finding that a human acted on? In most programs nobody knows, and the number is smaller than anyone expects. A meaningful fraction of any mature rule set is not detecting anything. Some of those rules are watching for behavior that genuinely stopped. Others broke silently, months ago, and have been reporting all-clear ever since.

Those two states look identical from a dashboard. Both are silence. This is the central problem with detection as most organizations practice it: the failure mode of a detection is indistinguishable from success.

Six ways a working rule dies quietly

Detections do not usually break with an error. They break by continuing to run against data that no longer contains what they were written to find.

• A field got renamed upstream. An agent upgrade changes process.parent.name to process.parent.executable. The rule still executes, matches nothing, and reports no error, because a query that matches zero rows is a perfectly valid query.
• The log source stopped arriving. A collector was decommissioned during a migration and one subnet stopped shipping. Every rule that depends on that source is now watching an empty room. This is the most common cause and the easiest to detect, and it is still usually found by accident.
• Tuning ate the detection. Someone added an exclusion during a noisy week. It was reasonable at the time. Nobody dated it, nobody reviewed it, and it now excludes the exact host class an attacker would use.
• The environment moved. The rule watches for a Windows service creation pattern. That workload moved to containers eighteen months ago. Still enabled, still counted as coverage for that technique.
• Volume limits are silently dropping events. Ingest hit a licence or quota ceiling and started sampling. The pipeline is healthy by its own metrics. Detection is now probabilistic and nobody was told.
• The rule was never right. It was written from a blog post, tested against one synthetic sample, and never validated against the real technique. It has been green since the day it shipped because it has never been meaningfully exercised.

Notice that none of the six produce an alert, a failed job, or a red indicator anywhere. Every one of them increases the rule count.

Give every rule a health record

The fix is unglamorous and it works: stop treating a detection as a query and start treating it as a service with an owner, dependencies, and a heartbeat. Once a rule declares what it depends on, most of the six failure modes become mechanically detectable.

# A detection is not a query. It is a query plus its contract.

id: cred_access.lsass_handle
owner: detection-eng            # a team, never "security"
severity: high

depends_on:                      # makes silent breakage checkable
  sources:   [edr.process, edr.handle]
  fields:    [process.parent.executable, handle.target, host.os]
  platforms: [windows]

expectations:
  # If this is silent longer than this, that is an event in itself.
  max_silence: 30d
  # Volume outside this band means the world changed, not that you won.
  expected_daily: {min: 0, max: 40}

validation:
  # A named, repeatable way to make this rule fire on purpose.
  harness: atomic/T1003.001
  last_verified: 2026-06-14

exclusions:
  - match: host.name = "BUILD-AGENT-*"
    reason: "signed build tooling opens this handle during packaging"
    added: 2026-03-02
    expires: 2026-09-02      # exclusions expire. all of them.
    ticket: SEC-4471

Two fields in there carry most of the value. max_silence converts the ambiguity of quiet into a signal: a rule that has not fired inside its window raises a ticket for a human to explain why. And expires on every exclusion means the accumulated tuning of five years cannot quietly become the largest hole in the program. An exclusion without an expiry date is a permanent decision made under time pressure by someone who has probably left.

Validation means making it fire on purpose

A detection you have never seen trigger is a hypothesis. The only way to convert it into a control is to execute the behavior it claims to catch, in an environment that resembles production, and watch the alert arrive. Then repeat that on a schedule, because the environment keeps moving underneath it.

This does not require a red team engagement. Open-source atomic test libraries cover a large share of common techniques, run in minutes, and can be scheduled. The measurement that matters is not pass or fail but the delta: how long between execution and alert, and how many stages of the chain were visible rather than just the last one.

# Weekly, per rule with a harness. Three outcomes, all informative.
#
#   FIRED, 41s          rule works, and you now know your real latency
#   FIRED, 22m          works, but useless for anything fast-moving
#   NO ALERT            the rule is decorative. find out why today.
#
# Record which stages were visible, not just whether the final alert hit:
#
#   initial exec    visible
#   discovery       visible
#   cred access     visible   <- the rule under test
#   lateral move    NOT VISIBLE
#   exfil staging   NOT VISIBLE
#
# That last column is your actual coverage map. It is usually shorter
# than the framework heatmap, and it is the one that is true.

Running this for the first time is uncomfortable, and that discomfort is the return on the exercise. Teams routinely discover that a technique marked covered on a heatmap produces nothing, while a technique nobody wrote a rule for is caught incidentally by something else.

Alert fatigue is a design failure, not a staffing one

The standard response to a noisy queue is to add analysts or buy a tool that promises to triage. Both treat volume as a fact of nature. It is not. Volume is a property of how the rules were written, and the usual cause is a rule that alerts on a behavior instead of on an anomaly in that behavior.

Administrators use administrative tools. That is not a finding. What is a finding is an administrative tool used by an account that has never used it, on a host it has never touched, at a time that account has never been active. Same underlying event, and the second framing produces two alerts a month instead of two hundred a day.

This requires the thing most detection programs lack: a maintained inventory of normal. Which service accounts exist and what each is supposed to do. Which hosts are build infrastructure. Which admin actions are expected during a change window. That inventory is not a security artifact, it is an operations artifact, and its absence is why so many programs are stuck choosing between blind and buried.

Four numbers worth reporting

Rule count is not one of them. Neither is percentage of a framework mapped, because a heatmap measures intent rather than capability.

• Rules validated in the last 90 days, as a share of enabled rules. This is your honest coverage number. It will start low. It is still the true one.
• Rules silent beyond their declared window. Every one of these is either a rule to retire or a break to fix. Both outcomes are progress; leaving it unresolved is not.
• Median time from behavior to alert, from validation runs rather than from incidents. Incidents give you a sample size of very few, too late.
• Share of alerts that changed someone's actions. Not closed, not triaged: actually caused an investigation, a containment step, or a fix. Everything else is a rule asking to be rewritten.

The reframe underneath all four: a detection is software, so it needs an owner, a test, a dependency contract, and a deprecation path. Written once and left running, it is not a control. It is a claim about the past that nobody has rechecked, and the dashboard will keep reporting it as green for as long as you let it.