Motivation to study the retelling of news

https://www.parsingscience.org/wp-content/uploads/2021/03/ParsingScience095-Shiri_Melumad.mp3#t=02:24,05:32

Melumad: The reason I started this paper is because I was interested in sort of a slightly different question. So initially, I was interested in how it is we end up with fake news. But as the project developed, it sort of fairly quickly became about a broader question, which I personally think is more interesting, which is, how do original news stories become distorted as they’re sequentially retold across people? So just to sort of concretize this, you can think of the children’s game of telephone. So let’s say I read a New York Times article, and I decided to tweet about it. So imagine, you know, I’m tweeting about it, I include a brief synopsis of what I read, and my take on it, right? And then let’s say you happen to come across my tweet. You think it’s pretty interesting. And you decide to retell what you read in my tweet to, let’s say, a friend of yours without actually having read the original article itself. And so on, right? So I think all of us can sort of relate to this idea. I think this happens all the time. And so basically, what I was interested in is whether this New York Times article would undergo any sort of systematic distortions with each new retelling of it. So that that’s sort of the broad story.

And what we find in the paper is, I think, fairly disturbing. So we find that with each new retelling of an original story, summaries contain fewer and fewer original details. That’s not particularly surprising. What is, I think, kind of scary is that with each new retelling, it also becomes increasingly more opinionated and increasingly negative. We refer to this as a shift towards “disagreeable personalization.” And the reason this happens is that … first of all, in general, when I’m sharing information with you, it’s typically because I assume you don’t already know this information, right? And so because of this, I’m going to feel naturally compelled to also provide guidance on whatever I’m relaying. So I relay not just the subset of facts, I think are the most important from the story, but also my interpretations and my opinions on it. And then the increased negativity part is also related to this. When I’m providing my interpretations of information, I also naturally want to do this in a persuasive manner. And so this is why I’ll also tend to emphasize negativity in my retelling, as a means of grabbing my audience’s attention. That’s related to a very well established phenomenon called the negativity bias: the idea that generally because negative information is rarer than positive information, it tends to stick out to us more and grab our attention more.

[ Back to topics ]

Why withhold details, personalize & accentuate negativity?

https://www.parsingscience.org/wp-content/uploads/2021/03/ParsingScience095-Shiri_Melumad.mp3#t=05:32,08:25

Leigh: Shiri’s work is situated within marketing research. And from business to politics. marketing focuses on changing people’s attitudes, beliefs, or behavior towards some product, event, idea, object a person. So Ryan and I were curious about what motivates people to persuade others by way of providing fewer details and more personal opinions and interpretations, and with a slant towards negativity.

Melumad: What sort of particularly cool about these findings is that these are all experimental results, meaning participants had no clear incentive to inject their personal opinions or to emphasize negativity when writing summaries of either a story or have a prior summary. Which speaks to, I think, sort of how powerful of an effect this is. And in contrast to conspiracy theories, or fake news, this distortion seems to be coming from a benevolent place, right? So again, the reason it’s happening is because as a reteller, I’m concerned about my audience, sort of, getting the key takeaways, and I want to help guide them on it. You know, I think theoretically, and with empirical backing for this, this is happening because of this natural inclination to want to sort of persuade whoever is reading my retelling or listening to my retelling of the interpretations I’m providing within the retelling. And in a number of the studies, we had the participants, think of a friend and name the friend. In that sense, we were trying to sort of personalize the audience as much as we could within the paradigm. Because I do really believe that if I’m speaking to sort of a nebulous audience, rather than like an individual who I can visualize, I’m going to express myself much differently. There’s work that’s consistent with this idea on broadcasting and narrowcasting, things like this. I think part of what might explain the content gorgeousness that we see on things like Twitter, is that often it feels like we’re broadcasting to sort of, again, like a nebulous audience: I’m not thinking of a single individual who be reading my tweet. Part of it is probably just norms associated with the given platform. But I also really do think part of it is the degree to which I can visualize a single person who would be listening to what I’m saying versus not. And so yes, I do think negativity will be more likely to arise in situations where there’s anonymity, whether my handle is anonymous, and/or my audience feels large enough that I wouldn’t be singling out a single person. And if you think about analogs of this in the real world, let’s assume I do that when I not just summarize a New York Times article, but I include the link. “So here’s all the information audience!” Most people don’t click on the link, right? Their takeaway of the article is based on whatever comment you wrote at the top of the link.

[ Back to topics ]

Example of the cycle of disagreeable personalization

https://www.parsingscience.org/wp-content/uploads/2021/03/ParsingScience095-Shiri_Melumad.mp3#t=08:25,11:58

Watkins: From the impartial reporting of facts to those characterize as decreasingly fact-laden, but increasingly opinionated and negative, Shiri calls the systematic devolution of original news stories over successive retellings, “disagreeable personalization.” Now, you might be wondering just how much information is transformed as it travels from the first retelling to the second and third. We’ll find out after this short break.

[ad: Figshare]

Watkins: Here again is Shiri Melumad.

Melumad: So this was when participants were given a political scandal story. There’s this mayor named Tom Ludovic, and he was basically accused of being part of a cult. And we left it ambiguous the extent to which the politician was actually guilty of what he was being accused of. I’ll read you an example summary from wave one and then I can read you an example from wave two, and then from wave three. And the examples build on each other. So the summary from wave two is going to be a synthesis of the wave one summary that I’m about to read. Okay, from wave one … Corgi is the town that the mayor resides and works in, and so: “Corgi Mayor Tom Ludovic is possibly being removed from office. Ludovic longtime friend, Pastor Thomas Harris has been alleged to start a cult called the Church of Eternal Salvation. Harris has been accused of forcing businesses to donate his foundations. Mayor Ludovic got entangled because he supposedly instructed his employees to collect money for the cult like foundation. Even though there are allegations of the two men being connected with the sacrificial and cult rituals, Ludovic states that they have not spoken in years.” So you might note that the summary has a fair amount of detail that seemed factual from the story. There’s not much by way of the writer’s opinion in this, right? Seems like a pretty factual synthesis.

Let’s listen to a summary of that summary: “Ludovic, the mayor of Corgi is alleged to have ties with this local cult group. Group’s called Church of Eternal Salvation. Supposedly, he and the pastor who started the group go way back and Ludovic has even had his official collect funds for the group. He claims, though, he hasn’t spoken to the pastor of the church in years. Be interesting to see what the investigation turns up.” So there’s sort of more of a skeptical tone here. There’s a little bit more interpretation included here. There’s fewer facts than in the prior wave.

Now let’s read a summary from the third wave: “This is strange, there’s evidence that the mayor of Corgi may have ties to a cult called the Church of Eternal Salvation. He may have even started in self along with the pastor and use his officials to collect funds for them. He claims to have cut ties with them, but it’s still pretty suspicious. No wonder people don’t trust politicians.” So by the third wave, we see it’s quite negative. It’s quite opinionated, and it’s briefer, there’s really not that much qualifying language. And at no point are any of the participants saying, “I haven’t read the original story, but …”

[ Back to topics ]

Is “agreeable personalization” possible?

https://www.parsingscience.org/wp-content/uploads/2021/03/ParsingScience095-Shiri_Melumad.mp3#t=11:58,15:03

Leigh: Because people often share information for self enhancing reasons, they’re more likely to retell a positive story when talking about themselves, and a negative one when talking about someone else. But if retailers use negativity to grab the attention of their audience, because it tends to be more salient, this would predict that a bias towards positivity might arise in settings where positive information is unusual or unexpected. Because the privileging of negativity stems in part from its comparative rarity, we asked Shiri to tell us more about what kind of factors might lead to “agreeable personalization.”

Melumad: So in study two the goal was sort of to manipulate what we thought the mechanism was, which is “felt relative knowledge.” Now, participants were explicitly being asked to write a summary of an ostensibly real science daily write-up. They were either told to write the summary for a friend who’s generally less knowledgeable on this topic than they are, or a friend who’s generally more knowledgeable on the topic of product health risks. And what we find is that – sort of consistent with the theoretical model – when I am summarizing, let’s say, a scientific article for someone who I think is less knowledgeable than me, then I really take on this role as like a guide on the information I’m relaying. We see greater injection of the writer’s opinions and interpretations of the facts they’ve chosen to relay. And greater emphasis on negativity. And negativity is actually manifested in two ways. It’s both the selection of the type of facts relayed, but also the nature of the opinions they express. In contrast, when I’m summarizing, you know, a scientific article to an expert on the topic, I will have very little reason to provide guidance on whatever facts I’m relaying right, or to emphasize negativity to sort of convince them to listen to me, right? Because they’re already inherently going to be interested. And there’s a good chance they already know it, frankly. There we see sort of a mitigation of the bias. So the reason this disagreeableness or negativity arises and retellings is as a means of convincing my audience to pay attention to what I’m saying. But part of it too, is that the negative information – when the story just sticks out to me naturally – because of the negativity bias, because negative information. Even though we might think it’s really common in our day to day, it actually is relatively rare relative to positive information. And so it tends to stick out to us more. And so the question was, “Okay, how about if we make positive information, sort of the more salient type of information? Are we able to flip the effect into an agreeable personalization bias?” And indeed, we’re able to. But an agreeable person lesion pious isn’t necessarily better than a disagreeable personalization one. So I guess it’s a glimmer of hope. But again, it’s not always the case that being particularly positive about something is necessarily better. So it really depends on the context. At least it is nice from a scientific perspective to see that we could flip the effect in that direction.

[ Back to topics ]

Disagreeable personalization of SciComm

https://www.parsingscience.org/wp-content/uploads/2021/03/ParsingScience095-Shiri_Melumad.mp3#t=15:03,16:48

Watkins: In Episode 63 of Parsing Science, we spoke with Susan Gelman about her research into how psychologists routinely use generic language in the titles and abstracts of their articles in order to imply that their findings are more generalizable than they may actually be. So if researchers may be promoting their research through such linguistic devices, Doug and I wondered what Shiri and her colleagues’ study might tell us about the dynamics of distortion stemming from retellings of scientific papers.

Melumad: Frankly, I think some of the scariest implications fall in the domain of SciComm, right? So in some of my studies, we use this ostensibly real article about laurel sulfate, this ingredient in shampoos. And when given the option to read the original article, before they summarized a prior synopsis of it, most people chose to not read the original article. And even the subset who did read the original article, it did not get rid of the bias in their summaries. It tempered it, but it didn’t remediate the bias. And so basically, we find that this bias is robust to a lot of source articles. Whether it’s a scientific article about sort of health risks associated with a product, we see that over time, it becomes increasingly more personalized writing, increasingly more self referential. And that’s I think, territory that’s particularly scary, right? Becoming more and more opinionated about something like product health risks, the less they know about product health risks. And I think that happens all the time. I also think, unfortunately, it’s the type of thing where people are particularly going to be less likely to read an original, you know, technical report on some scientific findings.

[ Back to topics ]

Overcoming disagreeable personalization

https://www.parsingscience.org/wp-content/uploads/2021/03/ParsingScience095-Shiri_Melumad.mp3#t=16:48,18:55

Leigh: From the nudges of behavioral economists to training psychologists to receive interpretation bias modification, an increasing number of academic and professional disciplines are seeking out means of mitigating our cognitive biases. But it’s hard. So we asked Shiri what her findings might suggest about possible avenues for overcoming the disagreeable personalization of information brought about by our negativity bias.

Melumad: I find myself qualifying any retelling more than I ever have before. I begin and end with a qualifying statement. But I don’t even know how helpful that’s going to be ultimately, right? Because they probably are going to ignore the qualifier. The next time they retell what I said. So, I don’t mean to sound so pessimistic, but I wish that my findings allowed me to be more positive about this. And so I have to say, very disappointingly, it is very hard to kill this bias. We have a study where we randomly assign participants into sort of a different goal in summarizing. So one is, you know, to be opinionated. One is to be entertaining. And one is to be as accurate as possible in their retelling. And this is not just injection of opinions, but this penchant for negativity arises regardless of what the instructed goal is. So I think in some ways, we should really be talking about this from the perspective of the creators of the original content. Like, let’s start there. If we can somehow incentivize, let’s say, writers to avoid sensationalizing certain key pieces of information – or like to avoid over claiming that certain information is shocking or unprecedented – this should in turn reduce readers’ motivation to insert their own interpretations and negativity. So I think it has to on some level start there. But what other advice would I want to give retailers? Include the link to the article, but I show that most people aren’t going to click on that link. I show that when people are instructed to be as accurate as possible, this bias still arises. And so I really think our best shot has to start with the originators of the of the article or of the report.

[ Back to topics ]

Disagreeable personalization in differing contexts

https://www.parsingscience.org/wp-content/uploads/2021/03/ParsingScience095-Shiri_Melumad.mp3#t=18:55,20:53

Watkins: In retelling news stories, Shiri’s research suggests that we’re naturally inclined to accentuate the negative, to add our own color commentary to stories, and to skip over – or even not seek out – some of the basic facts. But as she explains, we might also differ in how much we do this based on the audience to whom we’re sharing this new to them information.

Melumad: What my findings suggest is that there are certain types of networks or communities in which this effect is going to be more versus less likely. So we should probably see this arise for people who consider themselves to be, let’s say, fairly knowledgeable, and they’re communicating to a broader network where there’s more heterogeneity in the knowledge on that topic. The bias should be particularly likely there. So there’s going to be enough people who presumably know less than me. On the other end of the spectrum, they should be less likely in communities or networks that are more homogenous. Like for example, brand communities, right? Where everyone is fairly knowledgeable. And so you should see less of this bias. That’s just one leg of this. Once we’re dealing in a space where I care about how someone would react to my post. If I’m, let’s say, posting to my echo chamber of, let’s say, very conservative users, then I know they’re going to agree with me. In that case, sure, why not double down on my negative opinions on this article? Because I know most likely I’m going to get retweets and positive feedback. But if there’s a chance for me to be criticized by, let’s say, you on your anonymous Twitter, that should elicit some sort of self-presentational concerns that are probably less likely to arise when I know people are going to be supportive of me. So yeah, I think it’s a really important question. And then how does the back and forth amongst users alter this? There’s a lot of sort of unanswered questions to explore, which makes this research pretty exciting to me.

[ Back to topics ]

Supplemental studies

https://www.parsingscience.org/wp-content/uploads/2021/03/ParsingScience095-Shiri_Melumad.mp3#t=20:53,22;34

Leigh: For this paper, Shiri and her colleagues carried out a total of 10 experiments, four of which were included in detail in the accepted manuscript, with the other six included as supplements to the article. As Ryan and I are often coach our own students in writing their own studies up for publication, we were interested in learning how she and her co-authors decided on which studies to expand upon in the main body of the article and which to move into the supplements.

Melumad: Generally speaking, the review process will often shape a paper in a very specific way. And this was in large part shaped by the review process. It underwent a number of rounds of reviews. And the four studies that earned the main text largely have to do with what the reviewers either allowed us or encouraged us to keep in the main text. And, you know, we ran so many studies that it’s not right to not report them if we have the chance to do so as supplemental studies. And it really speaks to the robustness of the effect to be able to show it across so many different contexts. And I do think that paper came out a much better paper on the other side of the review process, I will say that. But yeah, I can’t say that we were wholly the arbiters of the decision of which study stayed in versus not. And, in fact, the story of the paper that I hadn’t really come up with the felt relative knowledge idea up front. So that was not part of the original paper. I got pushed back on how the theory wasn’t sort of thought through enough. And so I really sat down and thought about what could be going on, and it worked out really well. So another sort of takeaway for students is to be open to discovery. Because, again, this paper was meant to be about fake news at the beginning of it.

[ Back to topics ]

What we can learn from the methods of marketing research

https://www.parsingscience.org/wp-content/uploads/2021/03/ParsingScience095-Shiri_Melumad.mp3#t=22:34,24:47

Watkins: As Shiri’s work illustrates scholarship, exploring the theories and hypotheses of marketing connects to a number of disciplines such as psychology, and economics. So, in wrapping up our conversation, we asked her what she thinks listeners from other disciplines might take away from the techniques she applies when carrying out her research.

Melumad: Most of my work is experimental research. This, you know, allows me to manipulate whatever factor is of interest to me, holding as many other variables constant as possible, so that I can isolate sort of the independent variable of interest to me, and this allows one to establish causality. But I always tried to compliment my experimental results with field data when I can. The type of research that I’m interested in is the type of research that hopefully most people outside of my field would be interested in. I care about research with real world applications. So that’s why I field data, when I can find relevant data sources, is so important in my research. In a lot of my work, I look at modality effects: so how the way people express themselves on their phone differs from their computer, for example. And you know, sources like Twitter – where if you, you know, scraped tweets, you’re able to identify whether the tweet was written on a phone versus PC – that really allows me to establish the generalizability of my experimental findings. So of course, the downside of just relying on field data is that it’s correlational. So that’s what the experimental results allowed me to do is: “Okay, so we’ve established this, let’s say in the field on Twitter. Now let’s bring it to the lab where I can control extraneous variables and just try to isolate what I think is going on.” And so I think this – I’m biased – but I think that this is a very valuable approach to research that hopefully people in many other fields would find useful.

[ Back to topics ]

Links to manuscript, bonus audio and other materials

Leigh: That was Shiri Melumad discussing her article, The dynamics of distortion: how successive summarization alters the retelling of news,” which he published with Robert Meyer and Yoon Duk Kim, on January 7, 2021 in the Journal of Marketing Research. You’ll find a link to their paper at parsingscience.org/e95, along with transcripts, bonus audio clips, and other materials that we discussed during the episode.

Watkins: If you like what you’ve been hearing on the show, then head over to Apple podcasts, Google podcasts, or wherever else you might get your podcasts and subscribe to Parsing Science. And if you do already subscribe, consider leaving a review on iTunes. It’s a great way not only for others to learn about the show, but also a great way to help spread the work of the scientists on it.

Preview of episode 96

Leigh: Next time, in Episode 96 of Parsing Science, we’ll talk with Jean-Baptiste “JB” Lecca from the University of Lethbridge about his field research observing interactions among macaques at a Hindu temple in Bali, where the monkeys have learned to rob tourists of everything from eyeglasses to flip flops, and then barter their return to temple staff in exchange for food.

JB Lecca: You have to picture these events as being highly emotionally loaded. You’re not necessarily expecting to have, literally, a monkey climbing on your shoulders and snatching your sunglasses, or eyeglasses, or cell phones.

Leigh: We hope that you’ll join us again.

[ Back to topics ]