Harvard Experiment Finds Large Effects From Small News Outlets

By Jason Alcorn

November 20, 2017 | Crosspost with "Information Clearing House" - 

Even small publishers have a large effect on the national discourse, according to a new paper published in Science on the effects of news. “Exposure to the news media,” the study states, “causes Americans to take public stands on specific issues, join national policy conversations, and express themselves publicly.”

The research aims to quantify the effect of news media. Put in terms that are increasingly common when talking about journalism: What is the impact of news organizations?

The act of publishing news is influential in still-surprising and robust ways.

The study by Harvard professor Gary King and collaborators found that a few, mostly small news outlets publishing simultaneously in a broad area of public policy concern increased the volume of conversation on social media by 19 percent the day after publication. Over a full week, the volume was increased 63 percent relative to the average day’s volume. The number of unique authors increased as well, and the composition of opinion changed in the direction of the published articles.

News outlets it appears, even in the face of social media algorithms, the echo chambers of political polarization and a barrage of digital information, have a profound and measurable effect on national discourse.

Independent outlets have outsized impact #indymediamatters #tmcmedia https://t.co/CVCNRCGyzY

— Jo Ellen Kaiser (@jgksf) November 9, 2017

How the experiment was conducted

One of the challenges with research on media impact is the lack of control researchers have over the publication of news itself. You can’t exactly randomize the news. The need to maintain editorial independence and the timeliness of reporting means there have been relatively few large-scale, controlled experiments attempting to experimentally measure the effect of news media.

The research team found a way around this by actually collaborating with news outlets, using the first three years of the five-year study to build trust. “Frankly, when we began talks with the researchers in 2012, what they required seemed impossible,” Media Consortium executive director Jo Ellen Green Kaiser wrote on MetricShift in 2016. However, the researchers began attending industry conferences and worked with the Media Consortium to test the experiment with a small group of participants.

In the end, the study recruited 48 news outlets who agreed, in small groups, to publish at the same time on a broad policy area agreed-to beforehand and typical of the the outlets regular coverage. The 11 policy areas were race, immigration, jobs, abortion, climate, food policy, water, education policy, refugees, domestic energy production and reproductive rights. The media outlets retained full control over what was published, as well as the option to opt-out at any time. (A list of outlets that participated fully in the experiment is available in an appendix to the report.)

For their part, the researchers picked a two-week publication window when they expected a slow news cycle and randomized the publication date to either the first week or the second week. They then were able to measure and compare changes in the volume and composition of conversation on Twitter between the publication week and the control week.

The experiment was run 35 times, and this group-publishing was found to have increased conversation about the general topic by 63 percent over a week, relative to the average day’s volume.

The organizations were mostly small publications, with a median 50,000 subscribers. Some members of the Media Consortium who participated were Truthout, In These Times, Bitch Media, The Progressive, Earth Island Journal, Feministing, Generation Progress, Ms. Magazine, Yes! magazine and Making Contact.

What do the findings mean?

[dropcap]F[/dropcap]or the Media Consortium, this research shows that “even small independent news outlets can have a dramatic effect on the content of national conversation,” Kaiser wrote in a blog post.

Actually, she says, independent media has some built in advantages when it comes to having a impact on the national discourse. Independent outlets boast loyal communities of readers and supporters who are eager to push national conversations on social media, a willingness to collaborate openly that amplifies the effect of news, and news coverage that can face down serious topics because there’s less imperative to chase pageviews.

Defining impact as influence on a national conversation is new, with trustworthy measurement having been a big hurdle to overcome.

There are some limitations to the study. Kathleen Hall Jamieson, director of the Annenberg Public Policy Center at the University of Pennsylvania, told the New York Times that without the researchers disclosing more information (names of individual stories and publishers were withheld for reasons of editorial integrity), there would be questions about its general applicability. Is the volume of conversation on Twitter a true proxy for the national conversation, for example, and how meaningful is the discourse there?

Nevertheless, this research may shift the discussion about media impact. “In fact, too often, philanthropic foundations refuse to support these outlets because they are ‘too small’ and ‘don’t have enough impact,’” Kaiser wrote.

After sending 153,861 tweets, I definitely *want* to believe that journalists at small outlets have "substantial power to affect national conversations about politics & policy" https://t.co/XpOLZTtmws Highlights need for diversity in media sources & amplifying trusted journalism. pic.twitter.com/bGCC01Lr4X

— Alex Howard (@digiphile) November 10, 2017

A more expansive definition of impact

[dropcap]W[/dropcap]e are in an age of digital media where reader-supported journalism and nonprofit news will increasingly provide a majority of public interest reporting. Against that backdrop, this research is a step toward a deeper understanding.

Until now, impact has typically been measured along a vector. In that formation, acts of journalism lead to responses from the public and ultimately a reaction in the form of individual or social change. Impact trackers like the ones used by Gannett, Chalkbeat, and the Center for Investigative Reporting measure vector-based impact quite well.

This research raises the prospect that whether or not we measure it every time, the act of publishing news is influential in still-surprising and robust ways. “Our results should remind us of the importance of the ongoing and interconnected national conversation Americans have around major issues of public policy,” write the study’s authors. “This conversation is a fundamental characteristic of modern large-scale government.”

If mostly small publishers working in small collaborations can influence conversation so significantly, what of non-profit news collaborations, local newspapers, and television news? What of collaborations between all three?

Correction: Mother Jones is a member of The Media Consortium but did not participate in the experiment. The article has been updated.