Article published In:
The Agenda Setting JournalVol. 5:2 (2021) ► pp.156–176
A world of two agendas
Agenda setting sampling
This article studies the efficiency of different samples for content analysis of news in media effects studies by comparing the agenda-setting effect of a classic sample with the effect of a sample drawn based on audiences’ self-reported media habits. Contrary to the belief that exposure to sampled media content is necessary for observation of media effects, samples drawn based on overall readership/viewership of the media are more efficient than samples based on audiences’ actual consumption habits. A traditional media sample yields a stronger agenda-setting effect compared to a sample drawn based on self-reported media habits. But correlations between the two media samples are also strong. The findings suggest that a broad intermedia agenda-setting process makes it possible for researchers to draw a traditional sample that is representative of the issues salient to audiences regardless of their level of exposure to the sampled media. In other words, even in a demassified media environment, traditional samples are still the best option for media effects researchers.
Keywords: agenda setting, sampling, media effects, content analysis, audience, exposure
Article outline
- Theoretical framework
- Method
- Survey sample
- News media sample
- Classic sample
- Audience sample
- Concepts and measures
- Coding
- Analysis and results
- Discussion and conclusion
- Notes
-
References
References
Althaus, S. L., & Tewksbury, D.
(2002) Agenda
setting and the “new” news: Patterns of issue importance among readers of the paper and online versions of the New York
Times. Communication
Research,
29
(2), 180–207. 
Ariens, C.
Atwater, T., Salwen, M. B., & Anderson, R. B.
(1985) Media
Agenda-Setting with Environmental Issues. Journalism
Quarterly,
62
(2), 393–397.
Bennett, W. L., & Iyengar, S.
(2008) A
new era of minimal effects? The changing foundations of political communication. Journal of
Communication,
58
(4), 707–731.
Berinsky, A. J., Huber, G. A., & Lenz, G. S.
(2012) Evaluating
Online Labor Markets for Experimental Research: Amazon.com’s Mechanical Turk. Political
Analysis,
20
(03), 351–368.
Blondheim, M.
(1994) News
over the wires: The telegraph and the flow of public information in America,
1844–1897. Harvard University Press.
Breed, W.
(1955) Newspaper
‘opinion leaders’ and processes of standardization. Journalism
Quarterly,
32
(3), 277–328.
Brosius, H.-B., & Kepplinger, H. M.
(1992) Linear
and nonlinear models of agenda-setting in television. Journal of Broadcasting & Electronic
Media,
36
(1), 5–23.
Brosius, H.-B., & Weimann, G.
(1996) Who
sets the agenda: Agenda-setting as a two-step flow. Communication
Research,
23
(5), 561–580.
Camaj, L., & Weaver, D. H.
(2013) Need
for orientation and attribute agenda-setting during a U.S. election campaign. International
Journal of
Communication,
7
1, 1442–1463.
Chaffee, S. H., & Metzger, M. J.
(2001) The
end of mass communication? Mass Communication and
Society,
4
(4), 365–379.
Chmielewski, M., & Kucker, S. C.
(2020) An
MTurk Crisis? Shifts in Data Quality and the Impact on Study Results. Social Psychological and
Personality
Science,
11
(4), 464–473.
Coleman, R., & Banning, S.
(2006) Network
TV news’ affective framing of the presidential candidates: Evidence for a second-level agenda-setting effect through visual
framing. Journalism & Mass Communication
Quarterly,
83
(2), 313–328.
Coleman, R., & McCombs, M. E.
(2007) The
young and agenda-less? Exploring age-related differences in agenda setting on the youngest generation, baby boomers, and the
civic generation. Journalism & Mass Communication
Quarterly,
84
(3), 495–508.
Conway, B. A.
(2013) Addressing
the “medical malady”: Second-level agenda setting and public approval of
“Obamacare.” International Journal of Public Opinion
Research,
25
(4), 535–546.
Conway, B. A., Kenski, K., & Wang, D.
(2015) The
rise of Twitter in the political campaign: Searching for intermedia agenda-setting effects in the presidential
primary. Journal of Computer-Mediated
Communication,
20
(4), 363–380.
Cowart, H.
(2020) What
to think about: The applicability of agenda-settings in a social media context. The Agenda
Setting
Journal,
4
(2), 195–218.
Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M.
(2013) Evaluating
Amazon’s Mechanical Turk as a Tool for Experimental Behavioral Research. PLoS
ONE,
8
(3), e57410.
Dearing, J. W., & Rogers, E. M.
(1996) Agenda-setting. Sage.
Dillman Carpentier, F. R.
(2014) Agenda
setting and priming effects based on information presentation: Revisiting accessibility as a mechanism explaining agenda
setting and priming. Mass Communication and
Society,
17
(4), 531–552.
Djerf-Pierre, M., & Shehata, A.
(2017) Still
an agenda setter: Traditional news media and public opinion during the transition from low to high choice media
environments. Journal of
Communication,
67
(5), 733–757.
Funkhouser, G. R.
(1973) The
issues of the sixties: An exploratory study in the dynamics of public opinion. Public Opinion
Quarterly,
37
(1), 62.
Golan, G., & Wanta, W.
(2001) Second-level
agenda setting in the New Hampshire primary: A comparison of coverage in three newspapers and public perceptions of
candidates. Journalism & Mass Communication
Quarterly,
78
(2), 247–259.
Gold, D., & Simmons, J. L.
(1965) News
selection patterns among Iowa dailies. Public Opinion
Quarterly,
29
(3), 425–430.
Hester, J. B., & Dougall, E.
(2007) The
efficiency of constructed week sampling for content analysis of online news. Journalism &
Mass Communication
Quarterly,
84
(4), 811–824.
Hester, J. B., & Gibson, R.
(2003) The
economy and second-level agenda setting: A time-series analysis of economic news and public opinion about the
economy. Journalism & Mass Communication
Quarterly,
80
(1), 73–90.
(2007) The
agenda-setting function of national versus local media: A time-series analysis for the issue of same-sex
marriage. Mass Communication and
Society,
10
(3), 299–317.
Horton, J. J., Rand, D. G., & Zeckhauser, R. J.
(2011) The
online laboratory: Conducting experiments in a real labor market. Experimental
Economics,
14
(3), 399–425.
Horvit, B., Gade, P., & Lance, E. A.
(2013) News
wire greatest predictor of papers’ international news. Newspaper Research
Journal,
34
(1), 89–103.
Iyengar, S., & Kinder, D. R.
(1987) News
that matters: Television and American opinion. University of Chicago Press.
Johnson, T. J.
(Ed.) (2014) Agenda
setting in a 2.0 world. Routledge.
Kim, S. H., Scheufele, D. A., & Shanahan, J.
(2002) Think about it this way: Attribute agenda-setting function of the press and the public’s evaluation of a local issue. Journalism & mass communication quarterly 79(1), 7–25.
Kim, Y., Kim, Y., & Zhou, S.
(2017) Theoretical
and methodological trends of agenda-setting theory: A thematic analysis of the last four decades of
research. The Agenda Setting
Journal,
1
(1), 5–22.
Kosicki, G. M.
(1993) Problems
and opportunities in agenda-setting research. Journal of
Communication,
43
(2), 100–127.
Ku, G., Kaid, L. L., & Pfau, M.
(2003) The
impact of web site campaigning on traditional news media and public information
processing. Journalism & Mass Communication
Quarterly,
80
(3), 528–547.
Lacy, S., Riffe, D., Stoddard, S., Martin, H., & Chang, K.-K.
(2001) Sample
size for newspaper content analysis in multi-year studies. Journalism & Mass Communication
Quarterly,
78
(4), 836–845.
Lee, B., Lancendorfer, K. M., & Lee, K. J.
(2005) Agenda-setting
and the internet: The intermedia influence of internet bulletin boards on newspaper coverage of the 2000 general election in
South Korea. Asian Journal of
Communication,
15
(1), 57–71.
Leetaru, K. H.
(2015) Mining
libraries: Lessons learned from 20 years of massive computing on the world’s
information. Information Services &
Use,
35
(1–2), 31–50.
Legendre, P.
(2005) Species
associations: The Kendall coefficient of concordance revisited. Journal of Agricultural,
Biological, and Environmental
Statistics,
10
(2), 226–245.
Lim, J.
(2006) A
cross-lagged analysis of agenda setting among online news media. Journalism & Mass
Communication
Quarterly,
83
(2), 298–312.
(2011) First-level
and second-level intermedia agenda-setting among major news websites. Asian Journal of
Communication,
21
(2), 167–185.
Lopez-Escobar, E., Llamas, J. P., & McCombs, M. E.
(1998) Agenda
setting and community consensus: First and second level effects. International Journal of
Public Opinion
Research,
10
(4), 335–348.
McCombs, M. E.
(2005) A
Look at Agenda-setting: Past, present and future. Journalism
Studies,
6
(4), 543–557.
(2014) Setting
the agenda: The mass media and public opinion (2nd
ed.). Polity.
McCombs, M. E., Llamas, J. P., Lopez-Escobar, E., & Rey, F.
(1997) Candidate
images in Spanish elections: Second-level agenda-setting effects. Journalism & Mass
Communication
Quarterly,
74
(4), 703–717.
McCombs, M. E., & Shaw, D. L.
(1972) The
agenda-setting function of mass media. Public Opinion
Quarterly,
36
(2), 176.
(1993) The
evolution of agenda-setting research: Twenty-five years in the marketplace of ideas. Journal of
Communication,
43
(2), 58–67.
Megwa, E. R., & Brenner, D. J.
(1988) Toward
a paradigm of media agenda-setting effect: Agenda-setting as a process. Howard Journal of
Communications,
1
(1), 39–56.
Minooie, M.
(2019) Agendamelding:
How audiences meld agendas in Iran. The Agenda Setting
Journal,
3
(2), 139–164.
Misachi, J.
(2017, August 1). The
10 Most Popular Daily Newspapers in The United States [News]. World Atlas. [URL]
Moeller, J., Trilling, D., Helberger, N., Irion, K., & De Vreese, C.
(2016) Shrinking
core? Exploring the differential agenda setting power of traditional and personalized news
media. Info,
18
(6), 26–41.
Paolacci, G., & Chandler, J.
(2014) Inside
the Turk: Understanding Mechanical Turk as a Participant Pool. Current Directions in
Psychological
Science,
23
(3), 184–188.
Ragas, M. W., & Roberts, M. S.
(2009) Agenda
setting and agenda melding in an age of horizontal and vertical media: A new theoretical lens for virtual brand
communities. Journalism & Mass Communication
Quarterly,
86
(1), 45–64.
Reese, S. D., & Danielian, L.
(1989) Intermedia
influence and the drug issue: Converging on cocaine. In P. J. Shoemaker (Ed.), Communication
campaigns about drugs: Government, media, and the
public (pp. 29–46). L. Erlbaum Associates.
Riffe, D., Lacy, S., & Drager, M. W.
(1996) Sample
Size in Content Analysis of Weekly News Magazines. Journalism & Mass Communication
Quarterly,
73
(3), 635–644.
Riffe, D., Lacy, S., & Fico, F.
(2014) Analyzing
media messages: Using quantitative content analysis in research (Third
edition). Routledge/Taylor & Francis Group.
Riffe, D., Lacy, S., Nagovan, J., & Burkum, L.
(1996) The
Effectiveness of Simple and Stratified Random Sampling in Broadcast News Content
Analysis. Journalism & Mass Communication
Quarterly,
73
(1), 159–168.
Rogers, E. M., & Chang, S. B.
(1991) Media
coverage of technology issues: Ethiopian drought of 1984, AIDS, challenger, and
Chernobyl. In L. Wilkins & P. Patterson (Eds.), Risky
business: Communicating issues of science, risk, and public
policy (pp. 75–96). Greenwood Press.
Rogers, E. M., Dearing, J. W., & Bregman, D.
(1993) The
anatomy of agenda-setting research. Journal of
Communication,
43
(2), 68–84.
Sayre, B., Bode, L., Shah, D., Wilcox, D., & Shah, C.
(2010) Agenda
setting in a digital age: Tracking attention to California Proposition 8 in social media, online news and conventional
news. Policy &
Internet,
2
(2), 7–32.
Schoenbach, K., & Semetko, H. A.
(1992) Agenda-Setting,
Agenda-Reinforcing or Agenda-Deflating? A Study of the 1990 German National
Election. Journalism
Quarterly,
69
(4), 837–846.
Semetko, H. A., Blumler, J. G., Gurevitch, M., & Weaver, D. H.
(1991) The
formation of campaign agendas: A comparative analysis of party and media roles in recent American and British
elections. L. Erlbaum Associates.
Shank, D. B.
(2016) Using
Crowdsourcing Websites for Sociological Research: The Case of Amazon Mechanical Turk. The
American
Sociologist,
47
(1), 47–55.
Shaw, D. L., & McCombs, M. E.
(1977) The
emergence of American political issues: The agenda-setting function of the press. West Pub. Co.
Shaw, D. L., Minooie, M., Aikat, D., & Vargo, C. J.
(2019) Agendamelding:
News, social media, audiences, and civic community. Peter Lang.
Shaw, D. L., & Weaver, D. H.
(2014) Media
agenda-setting and audience agendamelding. In M. E. McCombs (Ed.), Setting
the agenda: The mass media and public opinion (2nd
ed., pp. 145–150). Polity.
Shehata, A., & Strömbäck, J.
(2013) Not
(yet) a new era of minimal effects: A study of agenda setting at the aggregate and individual
levels. The International Journal of
Press/Politics,
18
(2), 234–255.
Tan, Y., & Weaver, D. H.
(2013) Agenda
diversity and agenda setting from 1956 to 2004: What are the trends over time? Journalism
Studies,
14
(6), 773–789.
Towner, T. L., & Muñoz, C. L.
(2018) Picture
perfect? The role of Instagram in issue agenda setting during the 2016 presidential primary
campaign. Social Science Computer
Review,
36
(4), 484–499.
Vargo, C. J., Guo, L., & Amazeen, M. A.
(2017) The
agenda-setting power of fake news: A big data analysis of the online media landscape from 2014 to
2016. New Media &
Society, 146144481771208.
Vargo, C. J., Guo, L., McCombs, M. E., & Shaw, D. L.
(2014) Network
issue agendas on Twitter during the 2012 U.S. Presidential Election: Network issue agendas on
Twitter. Journal of
Communication,
64
(2), 296–316.
Weaver, D. H., Graber, D. A., McCombs, M. E., & Eyal, C. H.
(1981) Media
agenda-setting in a presidential election: Issues, images, and
interest. Praeger.
Weaver, D. H., Wojdynski, B., McKeever, R., & Shaw, D. L.
(2010) Vertical
and or versus? Horizontal communities: Need for orientation, media use and agenda
melding. Proceedings of the Annual Convention of the World Association for Public Opinion
Research. WAPOR, Chicago, IL.
Weimann, G., & Brosius, H.-B.
(2017) Redirecting
the agenda: Agenda-setting in the online Era. The Agenda Setting
Journal,
1
(1), 63–101.
Whitney, D. C., & Becker, L. B.
(1982) ‘Keeping
the gates’ for gatekeepers: The effects of wire news. Journalism
Quarterly,
59
(1), 60–65.