DevOps meets AI agents: Risk, audit, and the Deming playbook

About this episode

AI is no longer a lab tool; it’s showing up in pipelines, production systems, and the places where “seemed like a good idea” becomes a 2 a.m. incident. In this episode of Pop Goes the Stack, Lori MacVittie and Joel Moses are joined by John Willis, known for his work on DevOps and Deming, to separate what’s genuinely new about AI from what looks like the same organizational patterns repeating under a new label.

John frames the shift in two parts. First, the human side: every major technology transition triggers the same dynamics, and there’s a century of first principles from Deming and others that still apply. Second, the operational side: AI introduces a different kind of authority into the delivery loop. DevOps optimized for speed with reasonably deterministic pipelines. AI pushes systems into probabilistic behavior, where correctness is no longer guaranteed 100% of the time and audits can’t pretend “this will never happen.”

The conversation gets practical about what that means for enterprise teams adopting agents. The real questions aren’t whether tools use MCP or a CLI, but what authority an agent has: read-only, write/mutate, or execute. From there, you need boundaries, containment, escalation policies, kill switches, stronger logging, replayability, and the ability to justify decisions after the fact.

The main takeaway is permission to slow down. Step back, define what risk you’re willing to accept at each stage, and build guardrails that match that risk. AI isn’t going away, but “move fast” without a risk model is just handing operational authority to a very smart script and hoping it behaves.