Penetration Testing is Stuck in 2010. Here’s How to Move On.
Today I stumbled across a companion blog post for a talk from BSIDES Atlanta 2010. A talk I was actually in the audience for.
Reading it now, fifteen years later, feels like opening a time capsule and finding an old news clipping about potholes in your hometown: different decade, same mess.
Here’s Dave Kennedy back then, in the plain language of a man who had seen enough:
“Since when did a penetration test primarily become automated tool driven? … Handing a sixty page ‘penetration test’ report with five hundred vulnerabilities does absolutely nothing for a company aside from a check mark for whatever regulatory and compliance initiatives they have underway.”
He was right. For a technical leader under pressure to hit an audit deadline, a massive, context-free data dump isn’t just unhelpful — it’s a source of anxiety. That was the diagnosis in 2010. Unfortunately, the disease has metastasized.
Fast-forward to today, and you can still find vendors bragging about “checkbox penetration tests” like it’s a feature. A scanner run at arm’s length, a PDF dressed up as expertise, and no meaningful improvement to your security posture. The implicit promise is, “Just pay us, and the compliance requirement vanishes.”
Except it doesn’t, not really. A weak auditor might let the report slide, but a sharp one will see it as a red flag, leaving you feeling exposed and facing twice as many questions.
Kennedy also mocked the obsession with trophies:
“WOW I GOT DOMAIN ADMIN!!!!!! OMG I GOT ROOT! That’s fantastic buddy, I’m happy for you but that doesn’t mean squat to me. Present them with their intellectual property… what hurts them the most.”
Fifteen years on, many testers still chase that same cheap dopamine hit. One exploit, a couple “pwned” screenshot, and the work is declared done. Then they sprint to the bar to rehearse the war story for their next conference talk.
The client is told they got their “money’s worth,” while everyone quietly admits there are probably more issues still sitting there waiting to be found in next year’s engagement. This is how teams end up frustrated, having to re-do a pentest that should have been done right the first time.
What matters is not the exploit itself but the consequence. Not the root shell, but the impact to the business. Someone at that BSIDES that year put it in simple terms that endure: loot > root.
This philosophy is why I lean on frameworks like the OWASP ASVS for web app testing. It forces a methodical approach, preventing a test from ending at the first flashy finding. It moves the goalposts from “we got admin” to demonstrating completeness, consistency, and depth.
Likewise, scanners have their place, but they don’t tell you what actually matters. They don’t connect the dots or translate technical flaws into tangible business risk. That only comes from careful manual work. You dig until you can show not just that an attack is possible, but why it matters to the bottom line.
So what’s changed since 2010? Not much. We’re still arguing about overreliance on scanners, bargain-bin pentests, and the illusion that compliance alone equals security. The industry looks a lot like it did fifteen years ago, just with shinier dashboards and a few AI sprinkles.
Maybe that’s the depressing part. Or maybe it’s the reality of this field: there will always be the “fast and cheap” pentests that promise to keep auditors temporarily quiet, and there will be the deeper, risk-focused engagements that empower engineering with actionable guidance, help you pass compliance audits with confidence, and make you look prepared in front of customers and leadership.
The crucial part is knowing which one you’re paying for.
Ready to Make Your Next Pentest Audit-Proof?
If your pentest report looks impressive but leaves you wondering what it actually means for your business, you are not alone.
Our free guide, Audit-Proof Your Pentest, gives you 17 questions that reveal whether you are getting a true security assessment or just another checkbox exercise.
