UNESCO and University College London are warning policymakers not to treat artificial-intelligence models as dependable authorities when developing climate policy. A joint report presented Monday for discussion at the COP31 climate conference in Antalya, Turkey, found that the systems could answer most climate-related questions accurately but were prone to changing their responses when challenged.
The report, titled “Can We Trust AI on Climate?”, found that AI models answered 90 percent of climate questions correctly. Their reliability declined, however, when users pushed back on an answer or questioned its assumptions. According to a UNESCO statement, the models could abandon established scientific facts rather than consistently maintaining conclusions grounded in climate research.
Accuracy varied by language
The findings raise concerns about using general-purpose AI systems in decisions that affect emissions policy, energy planning and climate adaptation. UNESCO and UCL said model performance varied significantly across languages, reflecting differences in the amount and quality of training data available for each language.
| Finding | Reported result |
|---|---|
| Climate-related questions | Models answered 90 percent correctly |
| Responses challenged by users | Models quickly moved away from established scientific facts |
| Less widely spoken languages | Error rates increased |
| New information | Models were weak at analyzing it |
That language gap is significant for international climate decision-making, where governments and institutions operate across regions with very different levels of digital and scientific resources. The report said error rates rose when models responded in less widely spoken languages, making results less consistent between users.
Limits on policy use
The researchers also found that the models struggled to analyze new information. That limitation is relevant to climate policy, where governments regularly assess changing scientific evidence, updated emissions data and evolving conditions. UNESCO and UCL cautioned that model outputs should not be used as a substitute for established scientific evaluation or independent policy analysis.
- AI models performed well on a large majority of climate questions in the assessment.
- They were less dependable when users challenged their answers.
- Performance differed across languages and weakened when models handled new information.
The report was presented as COP31 was underway in Antalya. Its conclusions place the focus not only on the potential usefulness of AI in climate work, but also on the need for safeguards around how such systems are evaluated and used. UNESCO and UCL specifically cautioned policymakers and other decision-makers against relying fully on AI models to develop climate policy because the systems do not always remain aligned with established climate science.