Climate Knowledge in Large Language Models

About this episode

Climate Knowledge in Large Language Models

Kuznetsov, I., Grassi, J., Pantiukhin, D., Shapkin, B., Jung, T., & Koldunov, N. (2025). Alfred Wegener Institute, Helmholtz Centre for Polar and Marine Research.

LLMs have an internal "map" of the climate, but it is fuzzy: Without access to external tools, Large Language Models (LLMs) can recall the general structure of Earth’s climate—correctly identifying that the tropics are warm and high latitudes are cold. However, their specific numeric predictions are often inaccurate, with average errors ranging from 3°C to 6°C compared to historical weather data.

Location names matter more than coordinates: The study found that providing geographic context—such as the country, region, or city name—alongside coordinates reduced prediction errors by an average of 27%. This suggests models rely heavily on text associations with place names rather than possessing a precise spatial understanding of latitude and longitude.

Performance struggles with altitude and local trends: Models perform significantly worse in mountainous regions, with errors spiking sharply at elevations above 1500 meters. Furthermore, while LLMs can estimate the global average magnitude of warming, they fail to accurately reproduce the specific local patterns of temperature change that are essential for understanding regional climate dynamics.

Caution is needed for scientific use: The results highlight that while LLMs encode a static snapshot of climatological averages, they lack true physical understanding and struggle with dynamic trends. Consequently, they should not be relied upon as standalone climate databases; reliable applications require connecting them to external, authoritative data sources.