Is Offense or Defense Dominant? FAR.AI's Adam Gleave on the AI Security Leaderboard

About this episode

FAR.AI co-founder and CEO Adam Gleave joins Nathan to discuss FAR.AI’s AI Security Leaderboard, the first systematic head-to-head evaluation of the misuse safeguards frontier developers actually ship. The findings expose a major measurement gap: while Claude Fable 5 and GPT-5.6 Sol withstood FAR.AI’s suite, Grok 4.5 and Gemini 3.1 Pro yielded hundreds of universal jailbreaks at low cost. Adam explains why many effective attacks look more like social engineering than advanced ML, why “jailbreak tax” should not be relied on for safety, and how FAR.AI scores whether a model is genuinely helping an attacker. The episode’s stakes are whether AI developers can measure and harden real deployed defenses before threat actors make routine use of increasingly capable systems.

- FAR.AI AI Security Leaderboard: http://leaderboard.far.ai/

- People can e-mail [email protected] if they're interested in the open-weight safety accelerator grantmaking program.

For full show notes, links, and references, read the episode page:
https://www.cognitiverevolution.ai/is-offense-or-defense-dominant-far-ai-s-adam-gleave-on-the-ai-security-leaderboard/

Sponsor:

Claude:

Claude by Anthropic is an AI collaborator that understands your workflow and helps you tackle research, writing, coding, and organization with deep context. Get started with Claude and explore Claude Pro at https://claude.ai/tcr

CHAPTERS:

(00:00) About the Episode

(03:22) AI security leaderboard

(07:56) Universal jailbreaks explained

(16:26) Finding social jailbreaks (Part 1)

(16:31) Sponsor: Claude

(18:01) Finding social jailbreaks (Part 2)

(30:48) Layered safeguard defenses

(42:25) Uneven frontier safeguards

(51:10) Sharing safety standards

(01:00:30) Offense versus defense

(01:08:50) Open-weight model safety

(01:17:25) Control failure warnings

(01:30:05) Coordination and risk

(01:39:24) Episode Outro

(01:42:52) Outro

PRODUCED BY:

https://aipodcast.ing

SOCIAL LINKS:

Website: https://www.cognitiverevolution.ai

Twitter (Podcast): https://x.com/cogrev_podcast

Twitter (Nathan): https://x.com/labenz

LinkedIn: https://linkedin.com/in/nathanlabenz/

Youtube: https://youtube.com/@CognitiveRevolutionPodcast</p