- home | research
Toxic Content and User Engagement on Social Media: Evidence from a Field Experiment
With Rafael Jiménez-Durán, Mateusz Stalinski, and Jesse McCrosky
As much as forty percent of social media users have been harassed online, but there is scarce causal evidence of how toxic content impacts user engagement and whether it is contagious. In a pre-registered field experiment, we recruited participants to install a browser extension, and randomly assigned them to either a treatment group where the extension automatically hides toxic text content on Facebook, Twitter, and YouTube, or to a control group without hiding. As the first stage, 6.6% of the content displayed to users was classified as toxic by the extension relying on state-of-the-art toxicity detection tools, and duly hidden in the treatment group during a six-week long period. Lowering exposure to toxicity reduced content consumption on Facebook by 23% relative to the mean – beyond the mechanical effect of our intervention. We also report a 9.2% drop in ad consumption on Twitter (relative to the mean), where this metric is available. Additionally, the intervention reduced the average toxicity of content posted by users on Facebook and Twitter, evidence of toxicity being contagious. Taken together, our results suggest a trade-off faced by platforms: they can curb users’ toxicity at the expense of their content consumption.
Do Social Media Ads Matter for Political Behavior? A Field Experiment
With Mateusz Stalinski
Journal of Public Economics, 214 (2022): 104735
We exploit Facebook’s introduction of a filter hiding ads from the feed as a unique opportunity to study the effects of online ads on political behavior. In a pre-registered experiment, we randomly assigned participants to hide political ads (treatment) or alcohol ads (control) for several weeks preceding the 2020 US elections. We report an insignificant intent-to-treat effect of political ads on turnout (2.3 pp.), but we cannot rule out a sizable positive effect, with 95% confidence interval of [-2.8, 7.4]. The result may mask important heterogeneity, with political ads making Democrats slightly more motivated to vote and Republicans – substantially less. We explore the reasons for this effect, such as natural variation in ad content: the majority of Facebook ads on users’ feeds skewed Democratic. Lastly, the effect on measures of affective polarization and informedness was negligible.
To the Depths of the Sunk Cost: Experiments Revisiting the Elusive Effect
With Sota Ichiba and Mateusz Stalinski
Despite being often discussed both in practice and academic circles, the sunk cost effect remains empirically elusive. Our model based on reference point dependence suggests that the traditional way of testing it – by assigning discounts – may not produce the desired effect. Instead, we evaluate it across the gain-loss divide by randomizing the price (low, medium, or high) of a ticket to enter a real-effort task and observing its effect on playtime. Despite a strong intervention – we vary the sunk cost by $2 for a 14-minute task – and the sample size of N=1,806, we find only a small effect (0.09 SD or 1.1 minutes). We further explore the economic applications of the sunk cost effect in a field experiment on YouTube in which we randomize whether the time until a pre-video ad becomes skippable is shortened (0 s), default (5 s), or extended (10 s). We report two results. First, the intervention had an insignificant effect on video engagement (the time spent following the ad segment). Second, we detect a sizeable negative effect on the extensive margin – more users left before the video started in the extended treatment (5.3 pp. difference relative to the shortened treatment). Taking the results of both studies together, we offer a cautionary tale that applying even the most intuitive behavioral effects in policy settings might prove challenging.