The Consequences of Online Partisan Media

PNAS, (2021)

Authors

• Andrew M. Guess, Princeton
• Pablo Barberá, University of Southern California
• Simon Munzert, Hertie School
• JungHwan Yang, University of Illinois at Urbana-Champaign

What role do ideologically extreme media play in the polarization of society? Here, we report results from a pre-registered, randomized field experiment embedded in a nationally representative online panel survey (N = 1,037) in which participants were incentivized to change their browser default settings and social media following patterns, boosting the likelihood of encountering news with either a left-leaning (HuffPost) or right-leaning (Fox News) slant during the 2018 U.S. midterm election campaign. Data on ≈19m web visits by respondents indicate that resulting changes in news consumption habits persisted for at least eight weeks. While we find little evidence of a direct polarizing effect on opinions or affect, a large, durable decrease in overall media trust emerges among those exposed to Fox News within the first few weeks of the intervention (CACE: -0:55 SD; p < 0:05). These consequences appear to broaden over the longer term: Nearly five months later, participants assigned to either partisan news source continue to have lower trust in the media writ large. We additionally find that HuffPost liberalized respondents’ views on immigration policy during the campaign (CACE: 0:50 SD; p < 0:05), consistent with topic models demonstrating the outlet’s emphasis on the humanitarian dimension of the issue during this period. The combination of experimentation and computational social science techniques helps shed light on the long-term corrosive consequences of exposure to partisan news as well as its ability to influence views on highly salient topics.

Do Online Voter Guides Empower Citizens? Evidence from a Field Experiment with Digital Trace Data

Public Opinion Quarterly (2021)

Authors

• Simon Munzert, Hertie School
• Pablo Barberá, University of Southern California
• Andrew M. Guess, Princeton University
• JungHwan Yang, University of Illinois at Urbana-Champaign

Voting Advice Applications (VAAs), which provide citizens with information on the party that best represents their political preferences, are often cited as evidence of the empowering capabilities of digital tools. Aside from the informational benefits of these voter guides, observational studies have suggested a strong effect on political participation and vote choice. However, existing impact evaluations have been limited by a reliance on convenience samples, lack of random assignment, or both. This raises questions about self-selection and the precise mechanisms underlying how voters learn about politics. Here, we provide evidence from a field experiment with survey outcomes conducted with a sample of over 1,000 German citizens in the 2017 federal election campaign. Using linked panel survey and digital trace data combined with a randomized encouragement to complete a VAA, we are able to assess respondents’ compliance with treatment as well as to observe how the use of this tool affects political behavior, attitudes, media consumption, political knowledge, and even social media activity. Our findings reveal that the overwhelming consensus in favor of positive effects on turnout and vote choice should be treated with caution, as we find no such effects. Rather, we show that the actual virtue of VAAs in a complex online information environment is in increasing knowledge about parties’ positions on issues — exactly the kind of information these tools were designed to provide.

Media Exposure and Opinion Formation in an Age of Information Overload

Authors

• Simon Munzert, Hertie School
• Pablo Barberá, University of Southern California
• Andrew M. Guess, Princeton University
• JungHwan Yang, University of Illinois at Urbana-Champaign

Project white paper

We present the outline of a major online panel tracking study that aims to investigate media exposure and opinion formation in an age of information overload. The overarching project asks if online sources and social media platforms exacerbate polarization, and inequality in political knowledge and behavior, and what role weak ties play in moderating citizens’ information diets. To that end, we study how offline events are translated into media coverage, why people decide to consume or avoid coverage, and how these decisions affect attitudes and behavior. We combine passive metering technology to capture the online media consumption of a representative sample of individuals in Germany and the United States, draw on machine learning and natural language processing methods to estimate the topic and ideological slant of each separate piece of content consumed, and in parallel directly survey the panelists at regular intervals to monitor changes in issue attention, opinion, and political knowledge.

Meta-Analysis of the Effects of Voting Advice Applications

Political Communication (2021)

Authors

• Simon Munzert, Hertie School
• Sebastian Ramirez-Ruiz, Hertie School

We review the influence of voting advice applications (VAAs) on three core outcomes: turnout, vote choice, and issue knowledge. In a meta-analysis of 55 effects reported in 22 studies, comprising 73,673 participants in 9 countries, we find strong evidence for positive effects of VAA usage on reported turnout (OR= 1.87; 95% CI=[1.50, 2.33]) and vote choice (OR= 1.44; 95% CI=[1.16, 1.78]) as well as modest evidence on knowledge increase (partial correlation= 0.09; 95% CI=[-0.01, 0.18]). At the same time, we observe large heterogeneity in effect sizes, for which study design turns out to be a key driver. Effects are substantively weaker in causally more rigorous experimental studies. We highlight the need for more well-powered experimental research as well as studies focusing on the acquisition of knowledge in VAA usage.

Citizents' Preferences for Online Hate Speech Regulation in the United States and Germany

Working Paper

Authors

• Simon Munzert, Hertie School
• Richard Traunmüller, University of Mannheim
• Pablo Barberá, University of Southern California
• Andrew M. Guess, Princeton University
• JungHwan Yang, University of Illinois at Urbana-Champaign

To openly express one’s views is a fundamental right in any liberal democracy. However, in the age of social media, the questions of what is allowed to say and how public discourse should be regulated are ever more contested. We present a pre-registered study to analyze citizens? preferences for online hate speech regulation. We construct vignettes in forms of social media posts, mimicking actual cases of hate speech to bolster external validity, that vary along key dimensions of hate speech regulation. Respondents are asked to judge the posts with regards to perceived offensiveness and hatefulness as well as to actions that should be taken by the platform providers and other consequences the sender of hate speech should face. The experiment is embedded in nationally representative online panels in the US and Germany, which allows us to analyze context- as well as individual-level determinants for online hate speech regulation. The results indicate both convergence and divergence in how people in different contexts think about hate speech regulation, which has implications for its public legitimacy.

Trade Attitudes in the Wild

Working Paper

Authors

• Joshua D. Kertzer, Harvard University
• Pablo Barberá, University of Southern California
• Andrew M. Guess, Princeton University
• Simon Munzert, Hertie School
• JungHwan Yang, University of Illinois at Urbana-Champaign
• Andi Zhou, Harvard University

One of the central models in the study of international political economy holds that actors' preferences about economic issues like trade are a function of their economic interests as represented by their position in the global economy. Recent empirical work investigating the relationship between economic interests and trade attitudes, however, has found mixed results, leading to a new wave of experimental studies that point to the role of information in explaining why economic interests fail to predict economic preferences. But what kinds of information about trade are citizens exposed to in the real world, and what effect does it have on how they think about trade? This study combines survey data from an original 13 month national panel survey in the United States with individual-level behavioral measures of media consumption derived from web tracking data, to explore what news about trade Americans are exposed to in a naturalistic setting, and how it shapes their trade preferences. We find that most Americans are exposed to relatively little news about trade, but that the kind of trade news Americans are exposed to in the real world does not magnify the effects of economic interests; instead, we find some evidence that trade news affects trade preferences through sociotropic rather than pocketbook pathways, as Americans become more supportive of trade the more positive stories about trade they see.

The Effect of Hate Speech Regulation on Preference Falsification. Experimental Evidence from the US and Germany

Working Paper

Authors

• Richard Traunmüller, University of Mannheim
• Simon Munzert, Hertie School
• Andrew M. Guess, Princeton University
• Pablo Barberá, University of Southern California
• JungHwan Yang, University of Illinois at Urbana-Champaign

Concerns over the civility of public discourse and the spread of hateful messages have prompted governments into action around the world. Yet, whether and how to restrict speech that is considered offensive or promotes hate toward particular groups remains controversial. Empirically, claims about the expected effectiveness and likely consequences of regulatory intervention are largely untested. We present results from a pre-registered experimental study implemented in the United States and Germany, which tests two unintended consequences of hate speech law on self-censorship of citizens. Our expectations are twofold: First, we hypothesize that restricting the expression of obnoxious ideas merely drives them underground. Second, we expect a chilling effect on public discourse induced by hate speech regulation. The experimental findings run counter to our expectations and provide surprising new insights on the freedom of speech norm and the consequences of restricting hate speech.

Cheating in Online Assessments of Political Knowledge: Evidence from Survey and Digital Trace Data

Working Paper

Authors

• Simon Munzert, Hertie School
• Sebastian Ramirez-Ruiz, Hertie School
• Pablo Barberá, University of Southern California
• Andrew M. Guess, Princeton University
• JungHwan Yang, University of Illinois at Urbana-Champaign

Survey measures of individuals' political knowledge feature frequently in social science research. As self-administered online surveys grow in popularity, the validity of these measures has been called into question. In this note, we provide first-hand evidence of cheating in online assessments of political knowledge. We combine survey responses with web tracking data of a German (N = 685) and a US (N = 409) online panel to assess whether people turn to external sources for answers. The observed item-level prevalence rates of cheating vary depending on question type and difficulty from 0 to 12%. We find that 23% of the respondents from the German panel engage in cheating at least once across waves. On the contrary, less than 1% of the US respondents, who were discouraged from looking up external information with a commitment pledge, could be caught red-handed. We find robust respondent- and item-level characteristics associated with cheating. However, instances of cheating are rare events, as such, they are difficult to classify in the absence of tracking data. Even so, our analyses comparing naive and cheating-corrected measures of political knowledge provide little evidence that cheating distorts empirical models.

The False Consensus Effect Explains Why Citizens Overestimate the Prevalence of Online Echo Chambers

Working Paper

Authors

• Philipp Kling, University of Konstanz
• Karsten Donnay, University of Zürich
• Pablo Barberá, University of Southern California
• Andrew M. Guess, Princeton University
• Simon Munzert, Hertie School
• JungHwan Yang, University of Illinois at Urbana-Champaign

A frequently discussed phenomenon concerning online media consumption are so-called echo chambers: are social media users increasingly exposed only to information that reinforces their political views? While much research has been devoted to the prevalence of echo chambers and the potential effects of counter-attitudinal exposure, there is a research gap when it comes to the perceptions of the users themselves. How do citizens view their online information environments and are they aware of how politically slanted those environments are? By evaluating data from a representative survey of U.S. online adults linked to their Twitter data, here we offer evidence of a "false consensus" effect: citizens overestimate the proportion of people in their Twitter network that share the same ideological views as their. We observe this trend for all users, although its magnitude is lower for those with higher levels of political knowledge. We argue that this lack of awareness of biased information flows is a key piece of evidence that resolves the puzzle of why the concept of echo chambers has received extensive attention in academic and media circles despite the fact that empirical evidence shows that social media platforms increase exposure to diverse views.

(Mis)representing Ideology on Twitter: How Social Influence Shapes Online Political Expression

Working Paper

Authors

• William Small Schulz, Princeton University
• Andrew M. Guess, Princeton University
• Pablo Barberá, University of Southern California
• Simon Munzert, Hertie School
• JungHwan Yang, University of Illinois at Urbana-Champaign
• Alexander Gottlieb, Dartmouth College
• Adam Hughes, Pew Research Center
• Emma Remy, Pew Research Center
• Sono Shah, Pew Research Center
• Aaron Smith, Pew Research Center

Expressing opinions on social media has become a standard form of participation in the political process, but we know little about the factors that shape it. In this paper, we investigate the role of social context. Decades after the development of the canonical “Spiral of Silence” model, the public sphere has radically shifted toward a networked space mediated by social platforms. We articulate a theory of conformity in social media expression and test it by analyzing unique datasets linking U.S. survey respondents to their public Twitter accounts. To measure political expression, we develop and validate a supervised classifier of tweet-level ideology and apply it to respondents’ tweets and the tweets of people they follow. We find that the ideology of Twitter followees’ tweets is predictive of respondents’ own expressed ideology on Twitter, even after holding constant self-reported ideological predispositions. Our findings offer the first real-world evidence of social conformity effects on social media and demonstrate a powerful methodological approach for studying these dynamics.

News from Afar: Foreign Media and U.S. Public Opinion

Working Paper

Authors

• Naima Green-Riley, Harvard University
• Joshua D. Kertzer, Harvard University
• Andrew M. Guess, Princeton University
• Pablo Barberá, University of Southern California
• Simon Munzert, Hertie School
• JungHwan Yang, University of Illinois at Urbana-Champaign

Concerns over foreign in uence in American politics through state-owned news media like RT, Sputnik, and Al Jazeera have made headlines in recent years, at the same time as IR scholars have become increasingly interested in the effects of foreign voices on public opinion, as well as in testing the effects of public diplomacy efforts more generally. Worries over the potentially pernicious transnational effects of foreign media rest on at least two assumptions: that these foreign news sources successfully reach significant audiences in other countries, and that individuals who consume foreign news espouse systematically different views. We seek to test both of these assumptions empirically, integrating a nationally representative panel survey of Americans with behavioral individual-level measures of media consumption to paint a clearer picture of foreign news media consumption in the United States. The results have important implications for studies of public opinion about foreign policy, public diplomacy, and the fields of media and political communication more generally, and offer a new behavioral method for the study of public diplomacy.