The threat of climate change has become one of the greatest concerns for all ecosystems, including humanity. Dr. Marco Fisichella and Dr. Ivana Marenzi, together with their team of doctoral students are currently working on the SoMeCliCs research project that supports the discussion about how climate change is addressed in informal learning environments through Social Media from the perspective of Science Education (http://someclics.com/).
“At L3S, we strongly believe that AI could be invaluable in educating our youth about the current climate crisis and preventing worse impacts of climate change”, says Dr. Ivana Marenzi, Project Manager Digital Methods Subproject. In this context, the L3S team has been collaborating with students from class 9f at IGS List to find out if Artificial Intelligence can help students understand views on climate change and improve the informal learning environment for students in the current climate crisis. This will contribute to a better understanding of climate change discourse in social media from the science education and computer science perspectives. “On 26 January 2023, we conducted a pilot study with the students and the AI model for stance detection and sentiment analysis. Both the AI model and the students were given the same set of Twitter data with the task of rating tweets in terms of the attitude (believe/deny/don’t know) and sentiment (positive/negative) of the tweet on climate change” explains Apoorva Upadhyaya, PhD student in the project. The pilot study was conducted on the Learnweb platform, which served as a user interface for the students of IGS List and was developed by Oleh Astappiev, PhD student of our project. To determine the AI performance, a so-called single-task variant of the AI model was used, which was recently published in a scientific article by one of the L3S researchers (“A Multi-task Model for Sentiment Aided Stance Detection of Climate Change Tweets, Accepted in AAAI Conference on Web and Social Media 2023”). The PhD students then evaluated and compared the performance of the individual students and the pre-trained AI model for the corresponding set of the same tweets.
The researchers observed that students and AI performed similarly on the stance detection task, with a difference of 4.52 in the average accuracy of students and AI. It is also interesting to note that 32% of the students answered all the stance labels correctly and had more correct labels for believer climate change class as they agreed and related more with the notion of climate change. However, the AI model outperformed the students on the sentiment task with a higher average accuracy than the students (AI: 82.08, Students: 66.94). “Our careful analysis and another round of discussion with the students yielded interesting findings. For some tweets that contained irony or sarcasm and required real-world knowledge and experience to understand the point of view of the authors of the tweets on climate change, the students predicted the correct attitude, while the AI failed to do so” Upadhyaya further reports. In addition, removing hashtags changed the stance of the tweet, which motivates the removal of hashtags (query hashtags that are used to collect and prepare the dataset) from tweets for future studies. As a part of future works, the researchers are planning to focus on using AI to train students based on their understanding of climate tweets, and it would be exciting to explore the behavior of students on how they perceive the AI suggestions toward climate change tweets (blindly following the AI/rejecting the AI/using their intelligence in case of wrong AI predictions).
SoMeCliCs Project Team:
Apoorva Upadhyaya (upadhyaya@l3s.de)
Oleh Astappiev (astappiev@l3s.de)
Dr. Ivana Marenzi (marenzi@l3s.de)
Dr. Marco Fisichella (mfisichella@l3s.de)
