Assessing the quality of generative artificial intelligence for science communication in environmental research

Abstract

The adoption of Generative Artificial Intelligence (GenAI) tools is drastically changing the way that researchers work. While debate on the quality of GenAI outputs continues, there is optimism that GenAI may help human experts to address the most significant environmental challenges facing society. No previous research has quantitatively assessed the quality of GenAI outputs intended to inform environmental management decisions. Here we surveyed 98 environmental scientists and used their expertise to assess the quality of human and GenAI content relevant to their discipline. We analysed the quality and relative preference between human and GenAI content across three use cases in environmental science outreach and communication. Our results indicate that the GenAI content was generally deemed adequate in quality by human experts, with an average of 82% of respondents indicating a quality of "adequate" or better across the three use cases. Respondents exhibited strong preferences for GenAI over human-only content when using GenAI imageery of future park management scenarios. For the use cases of generating a wetland planting guide and answering a question about invasive species management, preferences were heterogeneous amongst respondents. Our findings raise substantive questions about GenAI content as a complement to human expertise when research is transferred to public audiences.