Websci Companion '24: Companion Proceedings of the 16th ACM Web Science Conference

This study explores the interplay between YouTube engagement metrics and the academic impact of cited publications within video descriptions, amid declining trust in traditional journalism and increased reliance on social media for information. By analyzing data from Altmetric.com and YouTube’s API, it assesses how YouTube video features relate to citation impact. Initial results suggest that videos citing scientific publications and garnering high engagement—likes, comments, and references to other publications—may function as a filtering mechanism or even as a predictor of impactful research.

Understanding the structural and linguistic properties of conversational data in social media is crucial for extracting meaningful insights to understand opinion dynamics, (mis-)information spreading, and the evolution of harmful behavior. Current state-of-the-art mathematical frameworks, such as hypergraphs and linguistic tools, such as large language models (LLMs), offer robust methodologies for modeling high-order group interactions and unprecedented capabilities for dealing with natural language-related tasks. In this study, we propose an innovative approach that blends these worlds by abstracting conversational networks via hypergraphs and analyzing their dynamics through LLMs. Our aim is to enhance the stance detection task by incorporating the high-order interactions naturally embedded within a conversation, thereby enriching the contextual understanding of LLMs regarding the intricate human dynamics underlying social media data.

Search query suggestions affect users’ interactions with search engines, which then influences the information they encounter. Thus, bias in search query suggestions can lead to exposure to biased search results and can impact opinion formation. This is especially critical in the political domain. Detecting and quantifying bias in web search engines is difficult due to its topic dependency, complexity, and subjectivity. The lack of context and phrasality of query suggestions emphasizes this problem. In a multi-step approach, we combine the benefits of large language models, pairwise comparison, and Elo-based scoring to identify and quantify bias in English search query suggestions. We apply our approach to the U.S. political news domain and compare bias in Google and Bing.

In this poster, we present an approach to study the structure of information diffusion cascades in Telegram, incorporating its group-oriented structure and multiple means of diffusion to gain a better coverage and higher trust.

Our research sheds light on the intricate workings of GameFi ecosystems by analyzing The Sandbox’s transaction graph. We fit this graph to the bow-tie model and find that outside support does little to boost activity and that a dedicated user base becomes resistant to scandals in the long term. We also find that whales, i.e., users with significant holdings, are involved in most transactions and exercise major influence. Our findings hold significant implications for the future development of equitable and sustainable GameFi platforms, offering valuable insights into stakeholder behavior and network resilience in the face of external challenges and opportunities.

While several case studies have shown positive correlations between research publications’ presence in press releases or news media and their citation impact, the effects explaining these associations remain uncertain. The study presented on this poster tests two frequently hypothesized effects – the earmark and the publicity effect – by investigating whether publications selected for press releases have a citation advantage compared to similar publications without press release, even if none of the publications did receive substantial press coverage, thus could not have benefited from a meaningful publicity advantage. Results from our statistical analysis of a comprehensive dataset of publications, press releases, citations, and altmetrics support the significance of the publicity effect, while providing no evidence to support any earmark effect.

An increase in active users on social networking sites (SNSs) has also been observed, as well as an increase in harmful content on social media sites. Harmful content is described as an inappropriate activity to harm or deceive an individual or a group of users. Alongside existing methods to detect misinformation and hate speech, users still need to be well-informed about the harmfulness of the content on SNSs. This study proposes a user-interactive system, TweetInfo, to mitigate the consumption of harmful content by providing metainformation about the posts. It focuses on two types of harmful content: hate speech and misinformation. TweetInfo provides insights into tweets by doing content analysis. Based on previous research, we have selected a list of metainformation. We offer the option to filter content based on meta information Bot, Hate Speech, Misinformation, Verified Account, Sentiment, Tweet Category, Language. The proposed user interface allows customising the user’s timeline to mitigate harmful content. This study presents the demo version of the proposed user interface of TweetInfo.

Voice-only telehealth is often more practical for lower-income patients who may lack stable internet connections. Thus, our study focused on using voice data to predict depression risk. The objectives were to: 1) Collect voice data from 24 people (12 with depression and 12 without mental health or major health condition diagnoses); 2) Build a machine learning model to predict depression risk. TPOT, an autoML tool, was used to select the best machine learning algorithm, which was the K-nearest neighbors classifier. The selected model had high performance in classifying depression risk (Precision: 0.98, Recall: 0.93, F1-Score: 0.96), compared to previous models. These findings may lead to a range of tools to help screen for and treat depression.

The 2023 Israel-Hamas conflict began in early October 2023, and as the conflict progressed, various instances of fake news began spreading worldwide. Fake news appeared on different platforms and was in multiple languages. For the scientific study of fake News, claims, and propagation, we contribute the first publicly available dataset of claims, fake social media posts, and fact-checked articles on the 2023 Israel-Hamas war. The WarClaim data is collected from 131 fact-checking organisations in 40 languages. We explain the data collection approach and source of the dataset. The WarClaim dataset is enriched by providing the information used by trained journalists specialising in fact-checking organisations. The dataset presented here can be utilised in various research endeavours, such as developing machine learning models and conducting exploratory studies. The dataset will be available on Github1.

TikTok is rapidly gaining momentum as a leading social media platform for the production, dissemination, and consumption of tailor-made short (news) videos. Thus far, the narrow body of scholarship on the intersection of TikTok and news has predominantly highlighted the production processes and output of traditional journalists working at legacy news organisations, neglecting non-traditionally journalistic users producing news-like videos. We conduct various computational analyses on all #news and/or #nachrichten videos uploaded to TikTok in 2023 from within Germany, Austria, or Switzerland, comparing the videos from 17 major journalistic news outlets with those from general users. To pinpoint differences between videos labeled as news-related by professional news outlets and ordinary users, we structure our research around hashtag, auditory and visual characteristics, the key elements of content creation, and - in such a combination - unique to TikTok. Our results show clear differences between the two video creator sets in the hashtag and visual characteristics but similarities in the auditory.

Social groups are central to political discussions. However, detecting social groups in text often relies on pre-determined socio-demographic categories or supervised learning methods that require extensive hand-labeled datasets. In this paper, we propose a methodology designed to leverage the potential of Large Language Models (LLMs) for the identification and annotation of social groups in text. The experiments show that open LLMs like Llama-2-70B-Chat and Mixtral-8-7B can reliably be used to annotate social groups in a few-shot scenario without the need for supervised learning. The automatically obtained annotations largely match human annotations on random samples from the Reddit Politosphere, resulting in micro-F1 scores of 0.71 and 0.83, respectively.

The Web Science conference series (WebSci) has become a prime venue for Web related research across disciplines. The interdisciplinary PhD symposium is an important component of this endeavor. Bringing PhD students from various disciplines together is, according to our understanding, an important step towards a comprehensive investigation of the Web. As such, we are trying to establish a forum for the next generation of Web Scientists. Following the spirit of ’unity in diversity’ we hope to bring together the many facets of Web related research covering computer science, the social sciences, marketing, economics, and other fields. We believe that a joint understanding beyond the limits of one’s own discipline will allow the Web to develop into a more interdisciplinary and inclusive future.

Transnational online platforms continuously navigate a mix of international and national laws, cultural norms, and political sensitivities to define what constitutes legitimate content moderation. While there is extensive research on platform governance in established democracies, less focus on the complexities of content moderation in less democratic developing countries. This integrated PhD thesis aims to contextualise how global online platforms hosting user-generated content (UGC) govern online speech within national systems that have to date been overlooked in scholarship. Sitting at the intersection of research on international political economy, platform governance, and human centred computing, the thesis draws on literature on legitimacy to offer a detailed examination of how a UGC platform originating from China (i.e., TikTok) governs online speech in two South and Southeast Asia countries - Indonesia and Pakistan. More importantly, by emphasising user insights in online speech governance, the thesis examines how users perceive the legitimacy of platformised content moderation, particularly in relation to sensitive and controversial borderline topics (i.e., gender/sexual, religious and political content). The comparative approach presents opportunities for understanding how on-the-ground insights can be mobilised to better govern algorithms and online platforms.

As the landscape of web science expands, handling the vast datasets collected from the Web while preserving computational efficiency and privacy remains a significant challenge. Data distillation offers a compelling solution by condensing large datasets into a distilled subset that retains essential characteristics. Part of my ongoing thesis work on tabular data distillation has shown that autoencoders and clustering algorithms can effectively distill tabular datasets, offering a promising solution for handling large datasets. Building upon this, my next step is to develop a versatile pre-trained model analogous to BERT and RoBERTa. This model can distill arbitrary tabular datasets, streamlining processes like data size reduction, synthetic data generation, large-scale analysis, reproducibility, and privacy preservation. Such a foundation model will serve as a versatile tool for web science research, making both data and research more accessible and computationally efficient. This model will not be limited to downstream classification and will be applicable to many further uses, such as reducing dataset sizes for efficient analysis, producing privatized synthetic datasets, or enhancing reproducibility through shared distilled data. By developing a foundation model for tabular data distillation, I aim to unlock new avenues in web science and improve computational accessibility, privacy protection, and reproducibility. The proposed direction holds promise for a versatile tool for handling the large amounts of data generated from the web while preserving its essence.

The spread of misinformation and the presence of ideological echo chambers online create problematic information environments that deprive users of accurate and unbiased information that they rely on. My research aims to understand the impacts and utility of interventions designed to mitigate problems of misinformation and echo chambers in online information environments as well as to develop future interventions. The first aspect of my research agenda is understanding the impact of previous interventions, which I explore through two studies – one evaluating the impacts of deplatforming as a misinformation moderation tool and the second assessing the current state of political factchecking. The next thrust of my work looks at identifying how current platform algorithms may be exacerbating problematic information environments, which I address through algorithmic audits of Twitter’s Who to Follow and Home Timeline algorithms and their potential to influence users’ encounters with election misinformation and political echo chambers. Finally, I propose research into future interventions and how we may measure their utility via a browser-extension based field experiment on content recommendation algorithms and network simulation experiments of deplatforming as a misinformation mitigation strategy.

This paper seeks to examine the concept of meaningfulness of consent with a focus on consent in digital transactions. To that end, it proposes a “consent matrix”, depicting the structure of consent transactions across two dimensions- the realm of consent-objects and the modes of obtaining consent. The matrix maps the two dimensions on orthogonal axes to create four quadrants, highlighting pathologies endemic to the structure of consent transactions in each quadrant. This approach helps to differentiate between structural and functional pathologies of consent, and move towards a more robust conceptualisation of meaningful consent.

Large Language Models (LLMs) are becoming increasingly powerful tools for social science research. We present work in progress on in silico longitudinal surveys on LLMs with modular adapters. This approach addresses issues of bias and prompt variation found in in silico surveys so far. While our initial implementation of this setup demonstrated abstraction capabilities of LLMs over digital trace data, validation on factual questions or seasonal trends is still required. In the long term, in silico surveys could have the potential to enrich survey data gathered from humans by integrating abstractions over digital trace data or transcripts from interviews.

This article investigates how large language models (LLMs) reflect human preferences and exhibit biases influenced by the diversity and nature of their input data. We used survey data related to Turkish presidential elections alongside tweets to assess the predictive performance and bias manifestations of LLMs under three different data inclusion strategies: (1) using only demographic information, (2) integrating demographic information with tweets, and (3) relying solely on tweets. Our findings reveal that prompts enriched with tweets typically achieve higher F1 Macro scores. However, this trend differs significantly when examining classes individually. While user-generated content significantly improves performance in predictions related to Recep Tayyip Erdoğan, it does not show the same effect for Kemal Kılıçdaroğlu. This study shows that different models and prompting styles result in varied biases for each candidate, leading to mixed outcomes. These results underscore the importance of exploring how biases vary across different scenarios, models, and prompting strategies for each case.

This paper tests the validity of a digital trace database (Politus) obtained from Twitter, with a recently conducted representative social survey, focusing on the use case of religiosity in Turkey. Religiosity scores in the research are extracted using supervised machine learning under the Politus project. The validation analysis depends on two steps. First, we compare the performances of two alternative tweet-to-user transformation strategies, and second, test for the impact of resampling via the MRP technique. Estimates of the Politus are examined at both aggregate and region-level. The results are intriguing for future research on measuring public opinion via social media data.

With the advancement of digital technologies and gadgets, online content has become easily accessible by users. At the same time, harmful content also spread widely. There are different harmful content types present on various platforms in multiple languages. The topic of harmful content is broad and covers multiple research directions. In research, the different forms are mostly analyzed separately, e.g., misinformation, cyberbullying, and hate speech. Most research has been conducted for a specific platform, monolingual situation, or particular topics. Counter-measures like blocking or down-ranking can make harmful content spreaders switch platforms and languages to continuously reach a user base. Harmful content does not only appear on social media but also on news media. Spreaders share harmful content in posts, news articles, comments, and hyperlinks. There is a great need to study harmful content across platforms, languages, and topics. We plan to bring the research on harmful content under one umbrella so that different approaches and novel methods can be shared. We organized the 1st workshop on DHOW: Diffusion of Harmful Content on Online Web at WebSci 2024, which brings together research on different topics related to harmful content. We expect to discuss innovative research work and future research directions.

Online harm spreaders are a multiplicity of users acting maliciously on social media by disseminating misinformation, propagating rumors or even making hateful comments targeting individuals or groups. Among these users are fake news spreaders, which are likely to distribute such type of news. The potential social harm of these users can be determined by examining the intuitions behind the content they generate that potentially enclose immoral behaviors. Therefore, the goal of this work is to examine the moral foundations of fake news spreaders. A dictionary-based tool for extracting moral content from texts is used to capture intuitive judgments of morally relevant information in text messages.

In today’s information-rich world, researchers often find themselves overwhelmed by the sheer amount of data available. This can make it challenging for them to locate relevant publications that align with their research interests. To address this issue, a tool called "SmartLitReview" has been developed. This tool automates the process of gathering literature from two influential sources, Google Scholar and Springer, thereby reducing the need for manual labour and streamlining the review process. This tool provides fully automated and user-friendly literature selection and download options. The study further explored the application of that tool and conducted a Systematic Literature Review (SLR) on the subject of "COVID-19 fake news," with a brief focus on the most prevalent fake news on COVID-19, the source of fake news dissemination, and its effects. The tool can, however, also be used with any other research topics that other researchers might consider intriguing.

Proliferation of online political hate speech through social media has been a persisting problem and is being recently compounded by the arrival of AI-boosted content. This can lead to wanton dissemination of misinformation/disinformation and can cause extremist radicalisation or influence national electoral processes. Given the high stakes of negative social impact, it is becoming increasingly important to address the sensitive topic of content moderation on social media platforms, the debate being the dichotomy of free speech versus content harm. From that perspective, it is crucial to establish a nuanced definition and categorisation of harmful content that is sensitive to the culture and language of the place of dissemination, which is different from the current one-size-fits-all approach, where content moderation is performed by social media companies behind closed doors. In this paper, we present a democratized solution to this problem through a crowdsourced annotation process that may be used to have a transparent method of identifying harmful content, which can then be used to make moderation decisions like contextually weighted downranking of harmful content. We present proof of concept case studies in the Indian political electoral discourse. We introduce a curated dataset of tweets labeled by multiple annotators from diverse backgrounds and visualize insightful statistical patterns emerging therefrom. This is the first stage of a multi-year Global Partnership on AI (GPAI) project on responsible AI for social media governance. In 2024 and beyond, we plan to expand the work to include both memes and tweets, that are multilingual (a mixture of Hindi/Bengali, English, and romanised Hindi/Bengali).

In the current landscape of online abuses and harms, effective content moderation is necessary to cultivate safe and inclusive online spaces. Yet, the effectiveness of many moderation interventions is still unclear. Here, we assess the effectiveness of The Great Ban, a massive deplatforming operation that affected nearly 2,000 communities on Reddit. By analyzing 16M comments posted by 17K users during 14 months, we provide nuanced results on the effects —both desired and otherwise— of the ban. Among our main findings is that 15.6% of the affected users left Reddit and that those who remained reduced their toxicity by 6.6% on average. The ban also caused 5% users to increase their toxicity by more than 70% of their pre-ban level. Overall, our multifaceted results provide new insights into the efficacy of deplatforming. As such, our findings can inform the development of future moderation interventions and the policing of online platforms.

There is an increase in the proliferation of online hate commensurate with the rise in the usage of social media. In response, there is also a significant advancement in the creation of automated tools aimed at identifying harmful text content using approaches grounded in Natural Language Processing and Deep Learning. Although it is known that training Deep Learning models require a substantial amount of annotated data, recent line of work suggests that models trained on specific subsets of the data still retain performance comparable to the model that was trained on the full dataset. In this work, we show how we can leverage influence scores to estimate the importance of a data point while training a model and designing a pruning strategy applied to the case of sexism detection. We evaluate the model performance trained on data pruned with different pruning strategies on three out-of-domain datasets and find, that in accordance with other work a large fraction of instances can be removed without significant performance drop. However, we also discover that the strategies for pruning data, previously successful in Natural Language Inference tasks, do not readily apply to the detection of harmful content and instead amplify the already prevalent class imbalance even more, leading in the worst-case to a complete absence of the hateful class.

Warning: This paper contains instances of hateful and sexist language to serve as examples.

In the midst of the widespread adoption of technology, particularly among younger generations, the increasing prevalence of hate speech online has become a pressing global concern. This research paper aims to address this urgent issue by conducting a thorough investigation into hate speech detection in Hindi-English code-mixed data. Existing research has largely approached hate speech recognition as a text classification problem, focusing on predicting the class of a message based solely on its textual content. Our task, however, delves into the classification of hateful content disseminated through tweets, comments, and replies on Twitter, taking into account the contextual intricacies inherent in social media communication. In this context, contextual nuances play a crucial role in understanding communication dynamics. By employing state-of-the-art deep learning techniques tailored to the unique linguistic characteristics of each language, this research makes a significant contribution to the development of robust and culturally sensitive hate speech detection systems. Such systems are essential for creating safer online environments and promoting cross-cultural understanding.

Warning: The content of this paper may contain offensive material, reader discretion is advised.

TikTok is a social media platform that has gained immense popularity over the last few years, particularly among younger demographics, due to the viral trends and challenges shared worldwide. The recent release of a free Research API opens the door to collecting data on posted videos, associated comments, and user activities. Our study focuses on evaluating the reliability of the results returned by the Research API, by collecting and analyzing a random sample of TikTok videos posted in a span of 6 years. Our preliminary results are instrumental for future research that aims to study the platform, highlighting caveats on the geographical distribution of videos and on the global prevalence of viral and conspiratorial hashtags.

Online social media platforms can be (mis)used to spread state-sponsored online influence operations. This could involve the manipulation of digital content to shape public opinion. Prior works have predominantly focused on the role of textual content in the spread of influence operations. In this work, we study the role of image coordination patterns across multiple influence operations on a social media platform (Twitter). We focus on two tasks. First, we identify image coordination by constructing image-image similarity graphs based on only image features. Second, we characterize images involved in influence operations to unveil politically and non-politically motivated images. We find varying degrees of coordination through images in different influence operations by the same state. Further, we also observe the use of politically motivated similar images extensively in these influence operations.