Vulnerabilities of the Online Public Square to Manipulation

Social media, the modern public square, is vulnerable to manipulation. By controlling inauthentic accounts impersonating humans, malicious actors can amplify disinformation within target communities. The consequences of such operations are difficult to evaluate due to the ethical challenges posed by experiments that would influence online communities. Here we use a social media model that simulates information diffusion in an empirical network to quantify the impacts of adversarial manipulation tactics on the quality of content. We find that social media features such as high information load, limited attention, and the presence of influentials exacerbate the vulnerabilities of online communities. Infiltrating a community is the most harmful tactic that bad actors can exploit and the most likely to make low-quality content go viral. The harm is further compounded by inauthentic agents flooding the network with engaging low-quality content, but is mitigated when influential or vulnerable individuals are targeted. These insights suggest countermeasures that platforms could employ to increase the resilience of social media users to manipulation.