Writing code used to be the expensive part. Not anymore. An AI agent will happily produce a thousand lines of plausible code in the time it takes you to read this paragraph. What did not get cheaper is knowing whether that code is safe to ship. That gap has quietly turned security testing into something it never used to be: proof of work. In a world drowning in AI slop code, the costly, deliberate act of verification is now the strongest signal that anyone actually checked.
We think this era is real, important, and temporary. Here is the whole argument.
Why Security Testing Became Proof of Work
In Bitcoin, proof of work is computation that is deliberately expensive. Nobody wants the hashes themselves. They want what the hashes prove: that someone burned real resources, which makes the ledger too costly to fake. The expense is not a side effect. The expense is the mechanism. Proof of work exists because trust does not scale on its own.
Software just arrived at the same place from the opposite direction. When every codebase is AI-polished - compiles cleanly, formatted perfectly, comments in complete sentences - surface quality stops signaling anything. Slop looks like craftsmanship now. The only signal left is the one that still costs something: threat modeling, penetration tests, static and dynamic application security testing, dependency audits, and humans reading diffs they did not write.
That is why the security review has become the artifact everyone asks for. Enterprise buyers do not ask "does your code look good?" They ask for the pen test report, the SOC 2 letter, the answers to a 300-row security questionnaire. These are receipts proving that expensive verification happened. Demand for them is climbing precisely because generation got cheap: when anyone can produce a convincing codebase in a weekend, the burn of verification is the only thing separating a real engineering organization from a very confident demo.
The economics underneath are simple. Generation now outruns review by orders of magnitude, and as we covered in the hidden cost of vibe coding, writing code was never the bottleneck anyway. Verification was. AI made that visible by removing everything else.
What the AI Generated Code Security Data Shows
None of this would matter if AI code were safe by default. It is not.
Veracode's 2025 GenAI Code Security Report ran more than a hundred LLMs through the same set of coding tasks and found that roughly 45% of the generated samples introduced known security flaws from the OWASP Top 10. The uncomfortable detail: newer and larger models wrote code that compiled more often, but the security failure rate barely moved. The models got better at looking right, not at being safe.
The human side is worse. A Stanford study, Do Users Write More Insecure Code with AI Assistants?, found that developers using an AI assistant wrote less secure code and were simultaneously more confident it was secure. Read that again. The tools increase both the volume of code and the confidence in it, while scrutiny per line collapses. That is the slop mechanism in one sentence: more output, more trust, same defect rate.
So teams respond the only way they can. They pile on SAST and DAST scans, mandate review on agent-authored pull requests, hire pentesters, and gate releases on security sign-off. All of it is expensive human-paced work standing between a cheap generator and production. All of it is proof of work.
This Era Is Temporary
Here is the part of the argument that most security vendors will not say out loud: the proof of work era has an expiry date, because the same models producing the slop are learning to catch it.
The evidence is already public. In late 2024, Google's Project Zero reported that its Big Sleep agent found an exploitable bug in SQLite - one of the most heavily fuzzed codebases on Earth - before it ever reached an official release. It has kept finding real vulnerabilities since. In 2025, DARPA's AI Cyber Challenge ended with autonomous systems finding and patching real vulnerabilities in widely used open source projects, and the winning systems were released as open source. The same year, an autonomous pentester climbed to the top of a national HackerOne leaderboard, ahead of thousands of human researchers.
There is a structural reason the machines will get good at this. Vulnerability discovery is verifiable: an exploit either reproduces or it does not. Tasks with a crisp pass/fail signal are exactly the tasks reinforcement learning eats. Code generation quality is fuzzy and subjective; "I found a memory corruption and here is the crashing input" is not. Verification is a better training target than generation, which means the capability curve for AI security testing will be steep.
When finding vulnerabilities costs cents instead of consultant-weeks, security testing stops working as proof of work, the same way a signature stops working when everyone owns the pen. What stays scarce is different: deciding what "secure enough" means for your product, whose data deserves which protections, which risks you accept and which you refuse, and who is accountable when the answer is wrong. Machines will do the finding. Humans still have to do the meaning. That is the same conclusion we reached about testing careers in will AI replace QA testers, and it holds even harder for security.
Most AI Labs Are Not Trying to Stop the Slop
You might expect the companies building these models to treat the slop problem as their problem. Mostly, they do not, and the incentives explain why.
Labs compete on capability benchmarks and shipping velocity. Every launch demo is "look how much it built," never "look how carefully it checked." Revenue scales with tokens generated, and no leaderboard anyone competes on measures the vulnerability rate of the code a model ships. A few labs genuinely invest in the other side - red-teaming their models for cyber misuse, publishing security evaluations, teaching models to review code as rigorously as they write it. They are a minority, and market gravity pushes the other way.
There is also a conflict of interest that should bother you: the vendor selling you the generator would like to also sell you the checker, grade its own homework, and keep both report cards. Whatever the verification layer of the AI era looks like, it should not be owned end to end by the party being verified.
Open Source Is How We Resist It
We do not think the answer is waiting for the labs. The answer, as it has been at every previous trust crisis in software, is open source.
Open source keeps the definition of "secure" out of any single vendor's hands. OWASP guides, CWE, and community rulesets are shared, inspectable standards, and open tools like Semgrep and ZAP make them executable by anyone. When one person encodes a check, everyone inherits it. Slop is a volume problem, and shared executable knowledge is the only thing that compounds at the same rate.
Open source keeps verification independent. Open-weight models mean the verifier does not have to come from the company that sold you the generator, and it can run inside your walls on code that is never allowed to leave. The AI Cyber Challenge requiring winners to open source their systems was exactly the right instinct: the tools that check the machines should belong to everybody.
And open source has already proven this model works at scale. Google's OSS-Fuzz has surfaced thousands of vulnerabilities across more than a thousand open source projects by making continuous, automated verification a shared public utility. That is the future of AI security testing in miniature: communal infrastructure, transparent rules, results anyone can audit.
A System of Record for Rules, Not Just Tests
So where does a test management company sit in all of this? Here is our bet.
When agents can regenerate an entire service from a spec overnight, code stops being the durable artifact of software. The durable artifact is the rules: what the product must do, what it must never do, what "secure" means here, and what evidence counts as proof. Today those rules are scattered across Slack threads, CI configs, wiki pages nobody trusts, and the heads of senior engineers. Ask an agent to honor rules stored like that and you get slop with extra confidence.
We are building TestCollab to be the system of record for those rules, managed by human-AI collaboration with a clear division of labor. Humans own judgment: they set intent, approve changes, and stay accountable. AI owns volume: it drafts, maintains, executes, and never gets tired of checking. Concretely, that looks like QA Copilot drafting test cases into pending queues where nothing enters your suite until a human accepts it, agents reading and updating rules through our MCP server instead of guessing, and an audit trail recording which rule changed, when, and on whose authority - human or agent. It is the approval-gate pattern we described in harness engineering, applied to the entire quality and security surface of a product.
The point of the loop is that it survives the transition. Today, humans do most of the verifying and the system records their proof of work. Tomorrow, agents do most of the verifying and the system records their evidence against human-approved rules. The rules and the accountability stay put while the labor migrates. That is the only architecture we have found that does not need to be thrown away when the machines get better.
On a Long Enough Timeline, Every Dev Tool Dies
We want to be honest about the end state, because almost nobody in our industry is.
If AI keeps compounding, the entire developer tool space gets obsoleted or transformed beyond recognition. IDEs, CI dashboards, issue trackers, test management platforms - all of it, ours included, was designed for humans manufacturing software by hand. When software is grown by agents and audited by agents, the tools either disappear into the agent layer or become something we would not recognize today: interfaces for humans to supervise fleets, not screens for humans to do the work. We do not know if that is five years out or twenty-five. Anyone who claims to know is selling something.
Our strategy is not to pretend we are permanent. It is to be the place where human judgment plugs into agent workflows for as long as that judgment still does real work - and every piece of evidence above says it will for a good while yet. The teams that win the transition will not be the ones that automated the most. They will be the ones that kept their rules explicit, their approvals human, and their evidence auditable while everyone else shipped confident slop.
Until Then, We Collab
Proof of work is a strange, expensive, temporary answer to a permanent question: why should anyone trust this software? For now, the answer is that people verified it. Soon, it will be that machines verified it against rules people own. The name of this company was a bet on collaboration between humans; it turns out the second collaborator is arriving with a GPU.
Until the machines can be trusted to check themselves, somebody has to hold the rules. Start with TestCollab and make yours explicit before the next thousand lines of code show up.
