When AI Decides You're a Threat — Brad Carson

About this episode

Brad Carson was the Army's General Counsel, served two terms in Congress and was Acting Under Secretary of Defense for Personnel and Readiness. He now heads Americans for Responsible Innovation, the AI-policy advocacy group he co-founded. Keith Duggar spends roughly eighty minutes pushing back.

SPONSOR:

---

Cyber Fund built the Monastery to help founders ship products that were impossible a year ago. Applications for Batch 1 are now open.

Apply now: https://cyber.fund

---

Carson's whole case rests on one line: the genie is not out of the bottle. We have pulled dangerous tech back before. Asilomar halted recombinant DNA in 1975, and the West still controls the chips AI runs on. Calling it unstoppable, he says, is the most dangerous idea in the room.

Then Keith drags him somewhere darker. A Palantir heat map scores you 0.73 on whether you are a combatant, and a strike follows. The model is wrong some accepted share of the time, and when it is, nobody answers for it. You cannot court-martial a model, and not even the interpretability researchers can say why it picked you.

—

Note: after recording, we learned that Americans for Responsible Innovation is backed by EA-aligned philanthropy (not sponsored)

---

TIMESTAMPS:

00:00:00 From the Pentagon to AI governance

00:04:52 Regulatory capture vs Silicon Valley networks

00:07:56 Transparency and the Claude tier changes

00:09:40 Tort liability when AI tools cause harm

00:13:40 AI is a product, not a person

00:16:01 Children, suicide, and the suicide business

00:19:59 Opaque neural nets and the law of war

00:25:54 Probabilistic targeting and the death of accountability

00:28:47 The arms race fallacy: Asilomar and restraint

00:34:02 Talking to China: track 2 talks and chip leverage

00:39:45 Air power never wins: capital for labour

00:43:29 Anthropic vs the Department of War

00:51:29 Concentration, open source, and brain drain

01:00:18 DeepSeek, Chinese culture, and AI as diplomacy

01:12:25 Upskilling Congress and why public trust matters

---

REFERENCES:

organization:

[00:02:45] ICRC position on autonomous weapons

https://www.icrc.org/en/law-and-policy/autonomous-weapons

[00:05:22] Americans for Responsible Innovation (ARI)

https://ari.us

[00:07:20] Andreessen Horowitz (a16z)

https://a16z.com/

[01:16:05] Office of Technology Assessment

https://en.wikipedia.org/wiki/Office_of_Technology_Assessment

other:

[00:03:35] Beneficial AGI 2019 Conference (Future of Life Institute, Puerto Rico)

https://futureoflife.org/event/beneficial-agi-2019/

[00:18:30] Section 230 of the Communications Decency Act

https://en.wikipedia.org/wiki/Section_230

[00:19:59] Lethal Autonomous Weapons (LAWS)

https://en.wikipedia.org/wiki/Lethal_autonomous_weapon

[00:31:35] Strategic Arms Limitation Talks (SALT)

https://en.wikipedia.org/wiki/Strategic_Arms_Limitation_Talks

[00:32:28] Asilomar Conference on Recombinant DNA (1975)

https://en.wikipedia.org/wiki/Asilomar_Conference_on_Recombinant_DNA

[00:39:45] The New Iron Triangle (ARI policy byte)

https://ari.us/policy-bytes/the-new-iron-triangle/

[00:48:05] Defense Production Act

https://en.wikipedia.org/wiki/Defense_Production_Act

person:

[00:03:35] Anthony Aguirre

https://en.wikipedia.org/wiki/Anthony_Aguirre

[00:06:48] Dean Ball — Hyperdimensional

https://www.hyperdimensional.co/

[00:23:13] Neel Nanda — mechanistic interpretability

https://www.neelnanda.io/

[00:36:02] Jack Clark (Anthropic) on Conversations with Tyler

https://conversationswithtyler.com/episodes/jack-clark/

[00:39:15] Robert Trager — Centre for the Governance of AI

https://www.governance.ai/team/robert-trager

[00:41:55] Giulio Douhet

https://en.wikipedia.org/wiki/Giulio_Douhet

[01:15:05] Don Beyer (US Congress)

https://en.wikipedia.org/wiki/Don_Beyer

tool:

[00:22:19] Phalanx CIWS

https://en.wikipedia.org/wiki/Phalanx_CIWS

---

ReScript:

https://app.rescript.info/public/share/9405ff35c0215b7cdae6402d41284171

https://app.rescript.info/api/public/sessions/0a6c081b8e5fe413/pdf