Context Engineering, Context Rot, & Agentic Search with the CEO of Chroma, Jeff Huber

About this episode

Jeff Huber is the CEO of ​Chroma, working on context engineering and building reliable retrieval infrastructure for AI systems.

Context Engineering, Context Rot, & Agentic Search with the CEO of Chroma, Jeff Huber // MLOps Podcast #348.

Join the Community:

https://go.mlops.community/YTJoinIn

Get the newsletter: https://go.mlops.community/YTNewsletter

// Abstract

Jeff Huber drops some hard truths about “context rot” — the slow decay of AI memory that’s quietly breaking your favorite models. From retrieval chaos to the hidden limits of context windows, he and Demetrios Brinkmann unpack why most AI systems forget what matters and how Chroma is rethinking the entire retrieval stack. It’s a bold look at whether smarter AI means cleaner context — or just better ways to hide the mess.

// Bio

Jeff Huber is the CEO and cofounder of Chroma. Chroma has raised $20M from top investors in Silicon Valley and builds modern search infrastructure for AI.

// Related Links

Website: https://www.trychroma.com/

~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~

Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore

Join our Slack community [https://go.mlops.community/slack]

Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)]

Sign up for the next meetup: [https://go.mlops.community/register]

MLOps Swag/Merch: [https://shop.mlops.community/]

Connect with Demetrios on LinkedIn: /dpbrinkm

Connect with Jeff on LinkedIn: /jeffchuber/

Timestamps:

[00:00] AI intelligence context clarity

[00:37] Context rot explanation

[03:02] Benchmarking context windows

[05:09] Breaking down search eras

[10:50] Agent task memory issues

[17:21] Semantic search limitations

[22:54] Context hygiene in AI

[30:15] Chroma on-device functionality

[38:23] Vision for precision systems

[43:07] ML model deployment challenges

[44:17] Wrap up