AI Didn’t Break Software Security. Leadership Did.

And leadership can fix it —> if we stop rewarding speed while treating engineering discipline as optional

I am going to start with the controversial part.

AI did not break software security. Leadership did.

We keep pointing at AI as if it is the dragon at the gate.

But the fire was already inside the castle.

Old code. Neglected maintenance. Weak security practices. Years of rewarding more features while treating the care of what we had already built as secondary work.

AI did not light that fire.

It blew through the open doors and made the flames move faster.

Not because leaders wrote every vulnerability.

Not because engineers no longer matter.

And certainly not because AI has made good engineering obsolete.

Leadership created, and continues to protect, the conditions in which old software remains untouched, maintenance loses against feature delivery, security work becomes an invisible list, and engineers are rewarded for adding more while rarely being celebrated for simplifying, upgrading, or deleting.

AI did not invent most of the weaknesses we are now discussing.

It changed the economics around finding and exploiting them.

That distinction matters, because if we decide AI is the problem, we will try to control the tool.

If we recognize that leadership is the problem, we have to change the system.

And systems are harder to blame.

I was worried I had nothing left to contribute

I was sitting in the audience at the Amsterdam JUG (Java User Group) meetup, waiting to join the second panel:

“What Can We All Do to Keep Safe and Benefit?”

The first panel was meant to explore what was happening with AI and where it was going. Our panel was meant to explore the how.

Except the first conversation had already begun moving into our territory.

Modernize old systems. Keep the software we rely on current. Give teams safe, shared ways to build. Automate testing. Improve how we handle security problems. Support open source.

:-/ Eek, I was starting to freak out! Would I have anything else to add? These great panelists were already discussing everything I had on my talk-about list!

It is a familiar fear before stepping onto a stage. We imagine there is a limited amount of valuable insight in the room, and someone else might use it all before the microphone reaches us.

But I was wrong.

Once our panel began, so much more came to mind.

Not because the first panel had failed to answer the questions. Because brilliant conversations do not consume ideas. They create them.

The first panel gave us an extraordinary foundation. My fellow panelists then kept building on it. Every answer opened another question. A technical observation became an organizational one. A vulnerability became a conversation about what companies reward. A toon became a conversation about trust, judgment, and accountability.

The more I listened, the more clearly I saw it:

This was never only a conversation about AI or security. It was a conversation about leadership.

The vulnerabilities are familiar. The economics are not.

SQL injection is not new.

Weak credentials are not new.

Poorly configured systems are not new.

Depending on software that nobody maintains anymore is not new.

Organizations prioritizing new features over upgrades, maintenance, testing, and cleaning up old problems are definitely not new.

What has changed is the speed, scale, and cost of discovering and exploiting those weaknesses.

AI can inspect more code, follow more connections, test more possibilities, and combine weaknesses more quickly than a human attacker ever could. A weakness we previously classified as “unlikely to matter” may now be combined with several other small weaknesses to create a dangerous way into the system.

The weaknesses were already there.

The economics changed. Many of our existing security rules were created around the limits of a human attacker.

Take timeouts.

Fifteen minutes once seemed like a short window. A human attacker could only attempt so much before the session ended.

But an AI agent can perform thousands of actions in those same fifteen minutes.

The timeout has not changed.

What can happen inside it has.

That means time alone is no longer enough. We also need to look at how many actions are happening, how quickly they are happening, and whether the pattern makes sense for a human being.

The same is true for many of our familiar security standards. They were designed when attacks required more human time, more human effort, and therefore more money.

AI changes all three.

1. Finding weaknesses becomes cheaper.

2. Trying different ways into a system becomes faster.

3. Launching many attacks at once becomes possible.

   
   

Meanwhile, many companies are still asking human engineers to investigate every alert, approve every change, and fix every problem one by one.

The attack became cheaper.

The defense remained expensive.

That is the economic shift leaders need to understand.

And many organizations are still trying to respond to machine-speed discovery with human-speed bureaucracy.

A weekly meeting to sort the findings.

A monthly patch cycle.

A change advisory board.

An ever-growing Jira backlog.

An exhausted engineer clicking “approve” for the thirtieth time that day.

AI did not create that mismatch.

Leadership has simply allowed ways of working designed for a slower world to remain unquestioned.

We have survived coding revolutions before

This is not the first time software development has changed beneath our feet.

We moved from mainframes to connected systems. From desktop software to the web. From proprietary components to open source. From manually managed servers to cloud platforms. From long release cycles to releasing changes continuously. From developers handing work to operations to teams caring for what they build from beginning to end.

Every shift changed what was possible.

Every shift created new risks.

Every shift forced us to strengthen our engineering practices.

We did not respond by declaring that brilliant engineers were no longer necessary.

We needed them more.

We needed people who could understand systems, reason about trade-offs, test assumptions, anticipate failure, and take responsibility for consequences, not only produce syntax.

That remains true now.

AI can generate code. It cannot make leadership accountable.

AI can propose a patch. It cannot decide how much risk an organization should accept.

AI can generate a test. It cannot guarantee that we asked the right question.

AI can accelerate our practices. It cannot compensate forever for not having them.

Good engineering practices still matter.

Tests still matter.

Thinking about how systems could be attacked still matters.

Being able to see what is happening inside our systems still matters.

Making the safest choice the easiest choice still matters.

Knowing and maintaining the software we depend on still matters.

Knowing what to do when something goes wrong, and learning from it afterwards, still matters.

How we design and connect our systems still matters.

Great engineering minds still matter.

Perhaps more than ever.

Test the logic before you create the code

Testing was crucial before AI.

Testing is crucial with AI.

Testing will still be crucial after the next revolution arrives.

I started developing software this way a few decades ago:

Think through the logic first. Decide what should happen. Decide what should never happen. Create the tests that prove it.

Then write the code.

This is not an old practice that AI has made irrelevant.

AI makes it even more important.

When code can be generated so quickly, we need a clear way to know whether it is the right code, not merely code that runs.

And the independence matters.

The person or AI agent creating the tests should not be the same one creating the coding logic. Otherwise, the same mistaken assumption can quietly appear in both. The code passes the test, not because the logic is right, bus because the test repeats the same misunderstanding.

Two pieces of output are not two independent perspectives when the same mind, or the same AI agent, created both.

Test the thinking.

Then test the code.

A vulnerability list is not a security strategy

One of the strongest insights from both panels was that a larger number of findings does not automatically create greater safety.

When the volume rises, security can easily become a numbers game.

How many vulnerabilities do we have?

How severe are they?

How many tickets did we close?

But the application with one reported vulnerability is not automatically safer than the application with one thousand. It may simply be less understood.

And a vulnerability that a security tool rates 9.5 out of 10 is not automatically the first thing to fix. A lower-rated weakness that can be reached from the internet, connects to sensitive data, or can be combined with other weaknesses may create far more actual risk.

There is another piece that a technical score, and sometimes an AI agent, can miss entirely:

The business impact.

As I discuss in my book, The Leadership Leap, great engineers understand the business value of what they are building.

They do not only understand the code.

They understand what that code makes possible.

They know which system keeps customer payments moving. Which service contains information people trusted us to protect. Which apparently small component supports a promise the business cannot afford to break. They understand what failure would mean for customers, employees, revenue, reputation, and trust.

An AI agent can examine the code and identify a weakness. It can help us understand how someone might use that weakness to get in. But unless it has the full business context, it may not understand what that system actually means to the organization or the people depending on it.

The technical score might say one vulnerability is worse.

The business impact might tell us to fix another one first.

That is not a coding decision.

It is engineering judgment.

And it is exactly why great engineering minds still matter.

The panels reinforced something critical: we need context.

What are we protecting?

Who might attack it?

What is exposed?

What is reachable?

What would the impact be?

Which piece of outside software is actually being used?

Which weakness could an attacker actually use in our environment?

What needs emergency action today and what can be handled through planned maintenance?

We need to sort the findings quickly by their real urgency and the damage they could cause.

A clear list tells us what exists. Thinking deliberately about how our system could be attacked helps us understand what matters. Seeing how the software is actually being used shows us what can be reached. Continuous testing helps us discover which weaknesses an attacker could really use.

Then skilled people who understand both the technology and the business still need to make decisions.

Security is not the production of a list.

It is the practice of making better decisions under uncertainty.

Our ways of working must change, not merely move faster

This was the contribution I most wanted to bring to the panel.

If vulnerability findings can be regenerated continuously,

why are we treating the vulnerability backlog as an immortal historical archive?

Every day, new findings arrive. Existing findings are duplicated. Context changes. Dependencies are upgraded. Services are retired. New attack paths become possible. Old findings become irrelevant.

Yet organizations keep pouring all of this into the same backlog and asking humans to prioritize an ever-growing pile.

I think we need to be more radical.

Generate the current risk picture every day.

Use AI to deduplicate, cluster, explain, and connect findings to the real system.

Reassess what is reachable and consequential now.

Close what is no longer relevant instead of preserving it forever because deleting backlog items feels uncomfortable.

Reset the view.

Then act on what matters.

That does not mean forgetting history. We can preserve the data without forcing people to treat every historical finding as active work.

It means the list teams work from must reflect today’s real risk, not the accumulated anxiety of every security-checking tool we have ever switched on.

I would also separate two streams of work.

One stream responds to urgent findings and attacks. Rotate engineers through it so security does not become one person’s permanent interruption.

The other stream protects the time needed to remove the causes:

modernizing systems, reducing the amount of outside software we rely on, improving shared tools and safe defaults, strengthening tests, simplifying how systems connect, and removing code we no longer need.

If we allow the reactive stream to consume everything, we will become extremely efficient at patching the same underlying conditions forever.

That is not resilience.

That is survival mode with better tooling.

Use AI for scale. Keep humans for judgment.

Humans alone cannot keep pace with machine-speed attacks.

We need AI to help us:

• deduplicate and group findings;

• connect the results from security tools to how the software is actually used;

• identify likely attack paths;

• continuously test systems in controlled environments;

• monitor unusual behavior;

• propose software upgrades and fixes;

• turn a discovered weakness into a repeatable test that proves whether it is fixed;

• explain findings so engineers can learn; and

• reduce the time between discovery, understanding, and action.

  
  

But using AI does not mean surrendering judgment.

The brilliant experts on the panels repeatedly returned to the same principle:

do not trust AI alone. Check what it suggests against security tools, clear rules, tests, approved software, and evidence from the real system.

And please do not ask the same AI to write a change, create its own tests, review its own work, and approve the change.

That is not independent verification.

That is asking the student to write the exam, take it, and award the grade.

Human involvement still matters, but it needs to happen where human thinking creates real value.

If we make engineers approve hundreds of suggestions, we have not created meaningful human care. We have created mental exhaustion disguised as control.

The question is not:

Where can we keep a human in the loop?

It is:

Which decisions genuinely require human accountability, context, creativity, and care?

Faster feature delivery also gives attackers more possible ways in

We love talking about AI productivity.

More code. More features. More releases. More quickly.

But three more features can also give attackers three more possible ways in.

Every piece of outside software AI selects can quietly bring more software with it, software an engineer never consciously chose. One panelist shared that most of the vulnerabilities they encounter are not in the libraries they selected directly, but in other components those libraries pulled in with them.

Every new line becomes something we may need to understand, test, secure, observe, update, and eventually remove.

If we celebrate the additional features without asking how they might be attacked, what they added to our maintenance burden, or how much more outside software they introduced, we are measuring only one side of progress.

That is not an AI failure.

That is a leadership failure.

And there is another leadership-created risk we are not talking about enough.

Engineers everywhere are being told to become dramatically faster with AI, or risk being replaced by it.

Be more productive.

Deliver more.

Learn every new tool immediately.

Prove that you are still needed.

Then we wonder why people start acting carelessly.

Why they accept code they do not fully understand.

Why they rush the review.

Why they let the same AI create the logic and the tests.

Why they add another outside library without investigating what came with it.

Why they hide how they are using AI instead of asking for help.

That behavior may look like an individual engineering failure.

But when it appears across an organization, leaders need to look at the pressure they created.

Fear does not create thoughtful engineering.

Fear creates shortcuts, silence, and people trying to prove their value as quickly as possible.

Leaders cannot keep telling engineers that AI may replace them and simultaneously expect those engineers to provide the careful human judgment AI cannot provide.

If we want people to challenge AI-generated code, admit when they do not understand something, slow down when a change feels unsafe, and protect the business from risks a tool cannot see, then we must make it safe for them to do those things.

Speed driven by fear is not productivity.

It is risk at scale.

People respond to what organizations reward.

If engineers are praised for shipping while maintenance is treated as an interruption, they will ship.

If roadmaps display features but hide upgrades, security work, simplification, and deletion, the invisible work will keep losing.

If velocity is the loudest measure of value, teams will find a way to demonstrate velocity, even while risk accumulates underneath it.

Real progress is not always additive.

Sometimes progress is deleting code.

Removing an abandoned dependency.

Reducing the number of technologies a team must understand.

Replacing a fragile component.

Closing an unnecessary connection.

Retiring a feature whose value no longer justifies its risk.

The safest code is often the code we no longer need to operate.

Accountability without enablement is abandonment

Developers do not all need to become full-time security specialists.

But teams do need enough knowledge to understand the consequences of what they build, generate, import, and deploy.

A central group of security experts cannot do this alone. It helps to have someone inside each product team who cares about security and stays closely connected to those experts. They can translate new risks into the team’s own work, help people think about how their system might be attacked, support the use of safer tools, and connect everyday engineering decisions to the wider security direction.

Some companies call this person a “security champion.”

But giving someone the title is not enough.

Not if they receive no time.

No authority.

No education.

No recognition.

No community.

Accountability without enablement is abandonment dressed up as empowerment.

Leaders must create the conditions:

• protected time for security and maintaining what teams have already built;

• education grounded in the systems teams actually own;

• shared tools and safe defaults that are easier to use than to avoid; (paved roads)

• a clear way to respond and ask for help when something goes wrong;

• continuously thinking about how systems could be attacked, then testing and watching them;

• learning without blame after something goes wrong;

• permission to slow down when the evidence demands it; and

• recognition that preventing, simplifying, upgrading, and deleting are valuable engineering outcomes.

Giving teams responsibility for security cannot mean:

“Developers are accountable now. Good luck.”

It must mean security is integrated into how the organization thinks, funds, builds, rewards, and learns.

Blocking AI can make the organization less safe

Here is another controversial opinion.

When leaders are frightened by AI, they often reach for prohibition.

No AI.

No external tools.

No experimentation.

It feels decisive. It may even look responsible on a policy slide.

But engineers are excellent at finding workarounds. We have been trained by years of impossible deadlines, restrictive processes, and badly designed internal tools.

If approved AI tools are unavailable while delivery pressure remains, AI use does not disappear.

It becomes invisible.

That is far more dangerous.

Leaders need to offer safe, usable tools and help engineers understand the new risks, including the extra tools and instructions they connect to AI, the software running on their own machines, and the outside code AI selects. The panels reminded us that attackers may not begin with the product itself. They may begin with an engineer who has the passwords and access they want.

We need guardrails people can realistically follow while doing the job they are still expected to do.

The safest AI policy is not automatically the strictest one.

It is the one that creates visible, educated, accountable use.

Open source is not the problem, but our relationship with it may be

Another powerful insight from the panels challenged an easy assumption:

open-source software is not inherently less secure than proprietary software.

Its code can be examined. Vulnerabilities can be disclosed. Communities can respond quickly. Engineers can propose fixes instead of waiting silently for a vendor.

Proprietary does not mean invulnerable. It may simply mean invisible.

The deeper problem is that companies consume open source as though “free to use” means “free of responsibility.”

And this brings me to the final contribution I wanted to make that evening.

Then the fire alarm went off

Near the end of the panel, while we were discussing whether changing programming languages could make systems safer, an alarm began to sound.

For a moment, it almost felt as though the building had decided we had discussed risk for long enough.

The meetup ended with a literal fire alarm.

And the point I most wanted to make stayed with me as we left the room.

So I will make it now.

Put budget behind what you use

Companies cannot keep extracting enormous value from open-source software and respond to its maintainers with nothing more than automated lists of everything they need to fix.

A vulnerability report is not a contribution.

A demand for an urgent fix is not stewardship.

And “open source” does not mean an unpaid maintainer has silently agreed to become the 24/7 security team for every profitable company that chose to depend on their work.

If an open-source component is important enough to sit inside your product, platform, or revenue stream, it is important enough to appear somewhere in your budget.

Fund maintainers.

Sponsor the foundations and infrastructure that sustain the ecosystem.

Give engineers time to contribute fixes and improvements back to the projects.

When your company creates a fix, do not keep it hidden forever inside your own private copy. Give it back to the project when you can.

Help improve the instructions. Review changes other people have suggested. Share the maintenance work. Participate in the ecosystem you profit from.

AI can help companies discover more vulnerabilities and generate more fixes. But if every company merely fires more findings at the maintainers, we have used acceleration to dump even more work onto people who may already be donating their labor.

That is not a sustainable security model.

We must move from consuming open source to participating in it.

Put budget behind what you use.

What I learned about my own voice

I arrived worried that all the useful answers might already have been spoken.

I left having been asked to return and give a 45-minute talk of my own.

The organizer told me he loved my fresh perspective on the subject.

That stayed with me.

My role was never to out-security the security specialists beside me.

My contribution was to see the connections:

• Between vulnerability backlogs and cognitive load.

• Between feature targets and accumulated risk.

• Between AI restrictions and hidden behavior.

• Between developer accountability and leadership enablement.

• Between open-source consumption and organizational responsibility.

That perspective comes from developing software since the late 1990s and spending the last 18 years in engineering leadership. It comes from living through coding revolutions. It comes from watching organizations repeatedly try to solve systemic problems with another ticket, another tool, another gate, or another demand on people who are already overloaded.

Sometimes the most valuable person in a technical conversation is not the one with the deepest answer inside one domain.

Sometimes it is the person who can see how the domains affect one another and ask what our choices are doing to the people inside the system.

I learned something about speaking too.

I did not need to protect a perfectly prepared answer until it was my turn.

Listening did not empty me.

It gave me more to say.

The ideas I arrived with were only a starting point. The real insight emerged through tension, disagreement, curiosity, and connection.

That is what a brilliant panel can do.

And it is what leadership must do now.

Not panic because the threat is accelerating.

Not pretend banning a tool will repair decades of neglected engineering discipline.

Not bury teams beneath vulnerability tickets while continuing to reward only new features.

Not declare that AI has made engineers irrelevant.

Leaders need to step up.

Change what we reward.

Change the ways of working.

Protect the practices that create quality.

Invest in the engineering minds who understand the systems.

Fund the open-source ecosystem those systems depend on.

And create the conditions in which people can see clearly, learn quickly, challenge assumptions, and make responsible decisions.

Because AI did not create all of our security problems.

It did something far more uncomfortable.

It made the consequences of our leadership choices faster, louder, and much harder to ignore.

AI is not the dragon we need to slay.

It is revealing where the walls were already weak and where leaders had stopped looking.

So perhaps the question is not:

“Did AI break software security forever?”

Perhaps the question is:

“Will leaders finally take responsibility for engineering it properly?”

The alarm interrupted the panel.

It did not end the conversation.

By the way, it wouldn't be complete to not thank my Xebia collegues I had great conversations with the week before the panel who discussed all things security with me. Thank you Florian Buetow, Jafar Shayan, and Geert-Hinke Addink. You are awesome for reaching out to discuss this great topic with me.