Why Your Scanner Lies: What 800 Findings Actually Means

The 800-finding report nobody reads

You ran a scanner. It came back with 812 findings, 47 of them "critical," in red. For an afternoon it felt like progress. Then someone opened it, tried to reproduce the first three, found they were noise, and quietly closed the tab. The report is still sitting in a shared drive, unread, doing nothing.

That is the normal outcome, and it is not because your team is lazy. It is because the number was never trustworthy in the first place.

Why the number is so big

Scanners are pattern matchers. Three habits inflate the count on every run:

• Version matching. "You run Nginx 1.18, here are its 40 CVEs," regardless of whether the vulnerable module is even compiled in or reachable.
• Reflected-input guesses. A parameter shows up in the response, so it is flagged as "possible XSS," without ever proving a script executed.
• Per-URL duplication. One missing header on a site with 300 pages becomes 300 findings.

None of these steps ask the only question that matters: can someone actually do something bad here, right now, on this app?

The false-positive tax

Every unverified finding costs a human. Someone reads it, tries to reproduce it, argues about it in a ticket, and usually closes it as "won't fix, false positive." Do that 800 times and two things happen. Your engineers stop reading security reports. And the three findings that were real get closed in the same dismissive sweep as everything else.

Here is the contrarian part: a scanner that reports more is not more thorough. It is less finished. Raw output is the first 10% of the work handed to you as if it were the last.

The severity number lies too

It is not just the count. The "critical" label is inflated the same way. A scanner marks a finding critical based on the worst case for that vulnerability class in the abstract, not what it can do on your specific app. A theoretical remote code execution in a library you import but never call the vulnerable function of gets the same red badge as an actual authentication bypass on your login route. Both say "critical." One can end your company this afternoon and the other cannot be triggered at all.

That is why "47 criticals" tells you nothing about where to start. Severity only means something once a finding is verified, because only then do you know the real precondition, the real reachability, and the real impact on your data. An unverified critical is a guess wearing a red hat.

What "verified with evidence" means

A finding is not "we saw a pattern that sometimes means trouble." A finding is a saved request and the response it produced, showing the effect happening. If we cannot capture that pair, we do not ship the finding.

{
  "id": "NOTRA-2201",
  "title": "IDOR: any user can read another user's invoices",
  "severity": "high",
  "endpoint": "GET /api/invoices/1042",
  "evidence": {
    "sent_as": "session for user #77",
    "response_status": 200,
    "leaked_fields": ["customer_email", "amount_due", "card_last4"]
  },
  "reproduced": true
}

You can hand that to an engineer and they can confirm it in the time it takes to paste one curl. There is nothing to argue about. The request either returns another user's invoice or it does not.

From raw signals to proven findings

Verification is mostly subtraction. A real run looks like this:

What you do with 12

Twelve verified findings is a to-do list, not a research project. You can triage it in an afternoon, assign owners, and watch the list go to zero. Contrast that with 812 maybes, which is a list you never start.

The findings that survive tend to be the ones scanners are worst at. A wide-open database table behind a public API key does not show up in an HTML crawl at all, which is the whole story in Supabase RLS Gaps. Meanwhile the loudest section of most reports is the one that matters least: missing response headers, which we put in their place in Security Headers That Actually Matter.

Verification is the entire job. Anyone can print 800 findings. Notra ships the handful that are real, each with the receipt. Run a free scorecard to see your number, or dig into more of our method on the blog.