Arize AI in Asia Pacific: LLM Evaluation, Observability & Scale with Patrick Kelly

About this episode

Fresh out of the studio, Patrick Kelly, Vice President for Asia Pacific at Arize AI, joins us to explore the critical world of AI observability, evaluation, and infrastructure and how Arize AI will start their go to market across the region. Beginning with his transition from Databricks to Arize AI, Patrick explained how the company's mission centers on making AI work for people by helping teams observe, evaluate, and continuously improve their AI agents in production. Emphasizing that evaluations are the most important requirement for AI systems in 2025-2026, he revealed a striking insight: approximately 50% of AI agents fail silently in production because organizations don't know what's happening. Through compelling case studies from Booking.com, Flipkart, and AT&T, Patrick explained how Arize AI enables real-time observability and online evaluations, achieving results like 40% accuracy improvements and 84% cost reductions. Patrick concluded by sharing his vision for success across Asia Pacific's diverse markets - from regulatory frameworks in Korea and Singapore to language localization challenges in Vietnam - emphasizing the three pillars that remain constant: helping customers make money, control costs, and manage risk in an era where AI governance has become paramount. Last but not least, he shares what great would look like for Arize AI in the Asia Pacific

"The mission is to make AI work for the people. It’s about getting AI working for everybody—consumers, customers, and businesses at large. Evals are the most important things that we’ve seen through 2025 and will see more of into 2026; they are the most important thing for systems to work. When I'm working with a customer, I ask: How are we going to help them make money? How are we going to help them control costs? And how are we going to help them manage risk? A lot of AI now is about managing risk."

Episode Highlights:
[00:00] Quote of the Day by Patrick Kelly
[01:10] Bernard introduces AI evaluation and infrastructure topic
[02:24] Patrick's journey from Databricks to Arize AI
[03:20] Arize AI's mission: making AI work for people
[04:00] Understanding agentic systems and their complexity
[05:18] Observability, evaluation, and development framework explained
[06:27] Creating continuous feedback loops for AI improvement
[07:00] On-premises and air-gapped deployment capabilities
[08:00] Open Telemetry and Open Inference standards
[09:08] Evaluations are critical for 2025-2026 success
[10:36] Booking.com case: real-time production AB testing
[14:36] Phoenix open source and Open Inference: entry to Arize ecosystem
[16:00] Travel industry use cases: Skyscanner and Flipkart
[17:53] AT&T case: 40% accuracy improvement, 84% cost reduction
[19:36] 50% of production agents fail silently
[20:26] Korea and Singapore MAS launches AI risk management framework
[22:08] Arize AI CEO's 10 predictions for AI 2026
[22:41] Cursor for X: AI engineering everywhere
[24:06] Context and session state matter critically
[26:27] Harness: new buzzword for agent orchestration
[34:13] Three pillars: make money, control costs, manage risk
[36:00] Asia Pacific diversity: India to Japan
[37:12] Language and cultural nuances in evaluations
[38:00] Closing

Profile: Patrick Kelly, Vice President, Asia Pacific, Arize AILinkedIn Profile: https://www.linkedin.com/in/patrick-kelly-aab6168/?ref=analyse.asia

Podcast Information: Bernard Leong hosts and produces the show. The proper credits for the intro and end music are "Energetic Sports Drive." G. Thomas Craig mixed and edited the episode in both video and audio format.