Do LLMs Understand Meaning? Neuroscience, Evaluation, and the Future of AI, with Maria Ryskina

About this episode

Do large language models actually understand meaning — or are we over-interpreting impressive behavior?

In this episode, we speak with Maria Ryskina, CIFAR AI Safety Postdoctoral Fellow at the Vector Institute for AI, whose research bridges neuroscience, cognitive science, and artificial intelligence. Together, we unpack what the brain can (and cannot) teach us about modern AI systems — and why current evaluation paradigms may be missing something fundamental.

We explore how language models can predict brain activity in regions linked to visual processing, what this reveals about cross-modal knowledge, and why scale alone may not resolve deeper conceptual gaps in AI. The conversation also tackles the growing importance of interpretability, especially as AI systems become more embedded in high-stakes, real-world contexts.

Beyond technical questions, Maria shares why community matters in AI research, particularly for underrepresented groups — and how diversity directly shapes the kinds of scientific questions we ask and the systems we ultimately build.

REFERENCES

• Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification
• Stereotypes and Smut: The (Mis)representation of Non-cisgender Identities by Text-to-Image Models
• Language models align with brain regions that represent concepts across modalities
• Elements of World Knowledge (EWoK): A Cognition-Inspired Framework for Evaluating Basic World Knowledge in Language Models
• Prompting is not a substitute for probability measurements in large language models
• Auxiliary task demands mask the capabilities of smaller language models

🎧 Subscribe to stay updated on new episodes spotlighting brilliant women shaping the future of AI.

• WiAIR website

Follow us at:

• LinkedIn
• Bluesky
• X (Twitter)