EP23: Building Open Source AI Frameworks: David Mezzetti on TextAI and Local-First AI

About this episode

David Mezzetti, creator of TextAI, joins us to talk about building open source AI frameworks as a solo developer - and why local-first AI still matters in the age of API-everything.

David's path from running a 50-person IT company through acquisition to building one of the most well-regarded AI orchestration libraries tells you how sometimes constraints breed better design. TextAI started during COVID when he was doing coronavirus literature research and realized semantic search could transform how we find information.

We get into the evolution of the AI framework landscape - from the early days of vector embeddings to RAG to LLM orchestration. David was initially stubborn about not supporting OpenAI's API, wanting to keep everything local. He admits that probably cost him some early traction compared to LangChain, but it also shaped TextAI's philosophy: you shouldn't need permission to build with AI.

We also talk about small models and some genuinely practical insights: a 20-million parameter model running on CPU might be all you need. On the future of coding with AI, David's come around on "vibe coding" and notes that well-documented frameworks with lots of examples are perfectly positioned for this new world.

Takeaways:

• Local-first AI gives you control, reproducibility, and often better performance for your domain
• Small models (even 20M parameters) can solve real problems on CPU
• Good documentation and examples make your framework AI-coding friendly
• Open source should mean actually contributing - not just publishing code
• Solo developers can compete by staying focused and being willing to evolve

  ---

Timeline:

(00:14) Introduction and David's Background

(07:44) TextAI History and Evolution

(12:04) Framework Landscape: LangChain, LlamaIndex, Haystack

(15:16) Can AI Re-implement Frameworks?

(24:14) API Specs: OpenAI vs Anthropic

(26:46) Running an Open Source Consulting Business

(32:51) Origin Story: COVID, Kaggle, and Medical Literature

(43:08) Open Source Philosophy and Giving Back

(47:16) Ethics of Local AI and Developer Freedom

(01:06:44) Human in the Loop and AI-Generated Code

(01:09:31) The Future of Work and Automation

Music:

• "Kid Kodi" — Blue Dot Sessions — via Free Music Archive — CC BY-NC 4.0.
• "Palms Down" — Blue Dot Sessions — via Free Music Archive — CC BY-NC 4.0. Changes: trimmed

  ---

About:

The Information Bottleneck is hosted by Ravid Shwartz-Ziv and Allen Roush, featuring in-depth conversations with leading AI researchers about the ideas shaping the future of machine learning.