Cover not available
Article published In:
The Agenda Setting Journal
Vol. 5:2 (2021) ► pp.156–176

A world of two agendas

Agenda setting sampling

 | Kennesaw State University
This article studies the efficiency of different samples for content analysis of news in media effects studies by comparing the agenda-setting effect of a classic sample with the effect of a sample drawn based on audiences’ self-reported media habits. Contrary to the belief that exposure to sampled media content is necessary for observation of media effects, samples drawn based on overall readership/viewership of the media are more efficient than samples based on audiences’ actual consumption habits. A traditional media sample yields a stronger agenda-setting effect compared to a sample drawn based on self-reported media habits. But correlations between the two media samples are also strong. The findings suggest that a broad intermedia agenda-setting process makes it possible for researchers to draw a traditional sample that is representative of the issues salient to audiences regardless of their level of exposure to the sampled media. In other words, even in a demassified media environment, traditional samples are still the best option for media effects researchers.
Keywords: agenda setting, sampling, media effects, content analysis, audience, exposure
Article outline
  • Theoretical framework
  • Method
    • Survey sample
    • News media sample
      • Classic sample
      • Audience sample
      • Concepts and measures
      • Coding
  • Analysis and results
  • Discussion and conclusion
  • Notes
  • References
Published online: 21 December 2021
DOI logo
https://doi.org/10.1075/asj.21001.min
References (75)
References
Althaus, S. L., & Tewksbury, D. (2002). Agenda setting and the “new” news: Patterns of issue importance among readers of the paper and online versions of the New York Times. Communication Research, 29 (2), 180–207. DOI logoGoogle Scholar
Ariens, C. (2018, March 27). Evening News Ratings: Week of March 19 [News]. Ad Week. [URL]
Atwater, T., Salwen, M. B., & Anderson, R. B. (1985). Media Agenda-Setting with Environmental Issues. Journalism Quarterly, 62 (2), 393–397. DOI logoGoogle Scholar
Bennett, W. L., & Iyengar, S. (2008). A new era of minimal effects? The changing foundations of political communication. Journal of Communication, 58 (4), 707–731. DOI logoGoogle Scholar
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012). Evaluating Online Labor Markets for Experimental Research: Amazon.com’s Mechanical Turk. Political Analysis, 20 (03), 351–368. DOI logoGoogle Scholar
Blondheim, M. (1994). News over the wires: The telegraph and the flow of public information in America, 1844–1897. Harvard University Press.Google Scholar
Breed, W. (1955). Newspaper ‘opinion leaders’ and processes of standardization. Journalism Quarterly, 32 (3), 277–328. DOI logoGoogle Scholar
Brosius, H.-B., & Kepplinger, H. M. (1992). Linear and nonlinear models of agenda-setting in television. Journal of Broadcasting & Electronic Media, 36 (1), 5–23. DOI logoGoogle Scholar
Brosius, H.-B., & Weimann, G. (1996). Who sets the agenda: Agenda-setting as a two-step flow. Communication Research, 23 (5), 561–580. DOI logoGoogle Scholar
Camaj, L., & Weaver, D. H. (2013). Need for orientation and attribute agenda-setting during a U.S. election campaign. International Journal of Communication, 7 1, 1442–1463.Google Scholar
Chaffee, S. H., & Metzger, M. J. (2001). The end of mass communication? Mass Communication and Society, 4 (4), 365–379. DOI logoGoogle Scholar
Chmielewski, M., & Kucker, S. C. (2020). An MTurk Crisis? Shifts in Data Quality and the Impact on Study Results. Social Psychological and Personality Science, 11 (4), 464–473. DOI logoGoogle Scholar
Coleman, R., & Banning, S. (2006). Network TV news’ affective framing of the presidential candidates: Evidence for a second-level agenda-setting effect through visual framing. Journalism & Mass Communication Quarterly, 83 (2), 313–328. DOI logoGoogle Scholar
Coleman, R., & McCombs, M. E. (2007). The young and agenda-less? Exploring age-related differences in agenda setting on the youngest generation, baby boomers, and the civic generation. Journalism & Mass Communication Quarterly, 84 (3), 495–508. DOI logoGoogle Scholar
Conway, B. A. (2013). Addressing the “medical malady”: Second-level agenda setting and public approval of “Obamacare.” International Journal of Public Opinion Research, 25 (4), 535–546. DOI logoGoogle Scholar
Conway, B. A., Kenski, K., & Wang, D. (2015). The rise of Twitter in the political campaign: Searching for intermedia agenda-setting effects in the presidential primary. Journal of Computer-Mediated Communication, 20 (4), 363–380. DOI logoGoogle Scholar
Cowart, H. (2020). What to think about: The applicability of agenda-settings in a social media context. The Agenda Setting Journal, 4 (2), 195–218. DOI logoGoogle Scholar
Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M. (2013). Evaluating Amazon’s Mechanical Turk as a Tool for Experimental Behavioral Research. PLoS ONE, 8 (3), e57410. DOI logoGoogle Scholar
Dearing, J. W., & Rogers, E. M. (1996). Agenda-setting. Sage.Google Scholar
Dillman Carpentier, F. R. (2014). Agenda setting and priming effects based on information presentation: Revisiting accessibility as a mechanism explaining agenda setting and priming. Mass Communication and Society, 17 (4), 531–552. DOI logoGoogle Scholar
Djerf-Pierre, M., & Shehata, A. (2017). Still an agenda setter: Traditional news media and public opinion during the transition from low to high choice media environments. Journal of Communication, 67 (5), 733–757. DOI logoGoogle Scholar
Funkhouser, G. R. (1973). The issues of the sixties: An exploratory study in the dynamics of public opinion. Public Opinion Quarterly, 37 (1), 62. DOI logoGoogle Scholar
Golan, G., & Wanta, W. (2001). Second-level agenda setting in the New Hampshire primary: A comparison of coverage in three newspapers and public perceptions of candidates. Journalism & Mass Communication Quarterly, 78 (2), 247–259. DOI logoGoogle Scholar
Gold, D., & Simmons, J. L. (1965). News selection patterns among Iowa dailies. Public Opinion Quarterly, 29 (3), 425–430. DOI logoGoogle Scholar
Hester, J. B., & Dougall, E. (2007). The efficiency of constructed week sampling for content analysis of online news. Journalism & Mass Communication Quarterly, 84 (4), 811–824. DOI logoGoogle Scholar
Hester, J. B., & Gibson, R. (2003). The economy and second-level agenda setting: A time-series analysis of economic news and public opinion about the economy. Journalism & Mass Communication Quarterly, 80 (1), 73–90. DOI logoGoogle Scholar
(2007). The agenda-setting function of national versus local media: A time-series analysis for the issue of same-sex marriage. Mass Communication and Society, 10 (3), 299–317. DOI logoGoogle Scholar
Horton, J. J., Rand, D. G., & Zeckhauser, R. J. (2011). The online laboratory: Conducting experiments in a real labor market. Experimental Economics, 14 (3), 399–425. DOI logoGoogle Scholar
Horvit, B., Gade, P., & Lance, E. A. (2013). News wire greatest predictor of papers’ international news. Newspaper Research Journal, 34 (1), 89–103. DOI logoGoogle Scholar
Iyengar, S., & Kinder, D. R. (1987). News that matters: Television and American opinion. University of Chicago Press.Google Scholar
Johnson, T. J. (Ed.). (2014). Agenda setting in a 2.0 world. Routledge.Google Scholar
Kim, S. H., Scheufele, D. A., & Shanahan, J. (2002). Think about it this way: Attribute agenda-setting function of the press and the public’s evaluation of a local issue. Journalism & mass communication quarterly, 79(1), 7–25. DOI logoGoogle Scholar
Kim, Y., Kim, Y., & Zhou, S. (2017). Theoretical and methodological trends of agenda-setting theory: A thematic analysis of the last four decades of research. The Agenda Setting Journal, 1 (1), 5–22. DOI logoGoogle Scholar
Kosicki, G. M. (1993). Problems and opportunities in agenda-setting research. Journal of Communication, 43 (2), 100–127. DOI logoGoogle Scholar
Ku, G., Kaid, L. L., & Pfau, M. (2003). The impact of web site campaigning on traditional news media and public information processing. Journalism & Mass Communication Quarterly, 80 (3), 528–547. DOI logoGoogle Scholar
Lacy, S., Riffe, D., Stoddard, S., Martin, H., & Chang, K.-K. (2001). Sample size for newspaper content analysis in multi-year studies. Journalism & Mass Communication Quarterly, 78 (4), 836–845. DOI logoGoogle Scholar
Lee, B., Lancendorfer, K. M., & Lee, K. J. (2005). Agenda-setting and the internet: The intermedia influence of internet bulletin boards on newspaper coverage of the 2000 general election in South Korea. Asian Journal of Communication, 15 (1), 57–71. DOI logoGoogle Scholar
Leetaru, K. H. (2015). Mining libraries: Lessons learned from 20 years of massive computing on the world’s information. Information Services & Use, 35 (1–2), 31–50. DOI logoGoogle Scholar
Legendre, P. (2005). Species associations: The Kendall coefficient of concordance revisited. Journal of Agricultural, Biological, and Environmental Statistics, 10 (2), 226–245. DOI logoGoogle Scholar
Lim, J. (2006). A cross-lagged analysis of agenda setting among online news media. Journalism & Mass Communication Quarterly, 83 (2), 298–312. DOI logoGoogle Scholar
(2011). First-level and second-level intermedia agenda-setting among major news websites. Asian Journal of Communication, 21 (2), 167–185. DOI logoGoogle Scholar
Lopez-Escobar, E., Llamas, J. P., & McCombs, M. E. (1998). Agenda setting and community consensus: First and second level effects. International Journal of Public Opinion Research, 10 (4), 335–348. DOI logoGoogle Scholar
McCombs, M. E. (2005). A Look at Agenda-setting: Past, present and future. Journalism Studies, 6 (4), 543–557. DOI logoGoogle Scholar
(2014). Setting the agenda: The mass media and public opinion (2nd ed.). Polity.Google Scholar
McCombs, M. E., Llamas, J. P., Lopez-Escobar, E., & Rey, F. (1997). Candidate images in Spanish elections: Second-level agenda-setting effects. Journalism & Mass Communication Quarterly, 74 (4), 703–717. DOI logoGoogle Scholar
McCombs, M. E., & Shaw, D. L. (1972). The agenda-setting function of mass media. Public Opinion Quarterly, 36 (2), 176. DOI logoGoogle Scholar
(1993). The evolution of agenda-setting research: Twenty-five years in the marketplace of ideas. Journal of Communication, 43 (2), 58–67. DOI logoGoogle Scholar
Megwa, E. R., & Brenner, D. J. (1988). Toward a paradigm of media agenda-setting effect: Agenda-setting as a process. Howard Journal of Communications, 1 (1), 39–56. DOI logoGoogle Scholar
Minooie, M. (2019). Agendamelding: How audiences meld agendas in Iran. The Agenda Setting Journal, 3 (2), 139–164. DOI logoGoogle Scholar
Misachi, J. (2017, August 1). The 10 Most Popular Daily Newspapers in The United States [News]. World Atlas. [URL]
Moeller, J., Trilling, D., Helberger, N., Irion, K., & De Vreese, C. (2016). Shrinking core? Exploring the differential agenda setting power of traditional and personalized news media. Info, 18 (6), 26–41. DOI logoGoogle Scholar
Paolacci, G., & Chandler, J. (2014). Inside the Turk: Understanding Mechanical Turk as a Participant Pool. Current Directions in Psychological Science, 23 (3), 184–188. DOI logoGoogle Scholar
Ragas, M. W., & Roberts, M. S. (2009). Agenda setting and agenda melding in an age of horizontal and vertical media: A new theoretical lens for virtual brand communities. Journalism & Mass Communication Quarterly, 86 (1), 45–64. DOI logoGoogle Scholar
Reese, S. D., & Danielian, L. (1989). Intermedia influence and the drug issue: Converging on cocaine. In P. J. Shoemaker (Ed.), Communication campaigns about drugs: Government, media, and the public (pp. 29–46). L. Erlbaum Associates.Google Scholar
Riffe, D., Lacy, S., & Drager, M. W. (1996). Sample Size in Content Analysis of Weekly News Magazines. Journalism & Mass Communication Quarterly, 73 (3), 635–644. DOI logoGoogle Scholar
Riffe, D., Lacy, S., & Fico, F. (2014). Analyzing media messages: Using quantitative content analysis in research (Third edition). Routledge/Taylor & Francis Group. DOI logoGoogle Scholar
Riffe, D., Lacy, S., Nagovan, J., & Burkum, L. (1996). The Effectiveness of Simple and Stratified Random Sampling in Broadcast News Content Analysis. Journalism & Mass Communication Quarterly, 73 (1), 159–168. DOI logoGoogle Scholar
Rogers, E. M., & Chang, S. B. (1991). Media coverage of technology issues: Ethiopian drought of 1984, AIDS, challenger, and Chernobyl. In L. Wilkins & P. Patterson (Eds.), Risky business: Communicating issues of science, risk, and public policy (pp. 75–96). Greenwood Press.Google Scholar
Rogers, E. M., Dearing, J. W., & Bregman, D. (1993). The anatomy of agenda-setting research. Journal of Communication, 43 (2), 68–84. DOI logoGoogle Scholar
Sayre, B., Bode, L., Shah, D., Wilcox, D., & Shah, C. (2010). Agenda setting in a digital age: Tracking attention to California Proposition 8 in social media, online news and conventional news. Policy & Internet, 2 (2), 7–32. DOI logoGoogle Scholar
Schoenbach, K., & Semetko, H. A. (1992). Agenda-Setting, Agenda-Reinforcing or Agenda-Deflating? A Study of the 1990 German National Election. Journalism Quarterly, 69 (4), 837–846. DOI logoGoogle Scholar
Semetko, H. A., Blumler, J. G., Gurevitch, M., & Weaver, D. H. (1991). The formation of campaign agendas: A comparative analysis of party and media roles in recent American and British elections. L. Erlbaum Associates.Google Scholar
Shank, D. B. (2016). Using Crowdsourcing Websites for Sociological Research: The Case of Amazon Mechanical Turk. The American Sociologist, 47 (1), 47–55. DOI logoGoogle Scholar
Shaw, D. L., & McCombs, M. E. (1977). The emergence of American political issues: The agenda-setting function of the press. West Pub. Co.Google Scholar
Shaw, D. L., Minooie, M., Aikat, D., & Vargo, C. J. (2019). Agendamelding: News, social media, audiences, and civic community. Peter Lang. DOI logoGoogle Scholar
Shaw, D. L., & Weaver, D. H. (2014). Media agenda-setting and audience agendamelding. In M. E. McCombs (Ed.), Setting the agenda: The mass media and public opinion (2nd ed., pp. 145–150). Polity.Google Scholar
Shehata, A., & Strömbäck, J. (2013). Not (yet) a new era of minimal effects: A study of agenda setting at the aggregate and individual levels. The International Journal of Press/Politics, 18 (2), 234–255. DOI logoGoogle Scholar
Tan, Y., & Weaver, D. H. (2013). Agenda diversity and agenda setting from 1956 to 2004: What are the trends over time? Journalism Studies, 14 (6), 773–789. DOI logoGoogle Scholar
Towner, T. L., & Muñoz, C. L. (2018). Picture perfect? The role of Instagram in issue agenda setting during the 2016 presidential primary campaign. Social Science Computer Review, 36 (4), 484–499. DOI logoGoogle Scholar
Vargo, C. J., Guo, L., & Amazeen, M. A. (2017). The agenda-setting power of fake news: A big data analysis of the online media landscape from 2014 to 2016. New Media & Society, 146144481771208. DOI logoGoogle Scholar
Vargo, C. J., Guo, L., McCombs, M. E., & Shaw, D. L. (2014). Network issue agendas on Twitter during the 2012 U.S. Presidential Election: Network issue agendas on Twitter. Journal of Communication, 64 (2), 296–316. DOI logoGoogle Scholar
Weaver, D. H., Graber, D. A., McCombs, M. E., & Eyal, C. H. (1981). Media agenda-setting in a presidential election: Issues, images, and interest. Praeger.Google Scholar
Weaver, D. H., Wojdynski, B., McKeever, R., & Shaw, D. L. (2010). Vertical and or versus? Horizontal communities: Need for orientation, media use and agenda melding. Proceedings of the Annual Convention of the World Association for Public Opinion Research. WAPOR, Chicago, IL.Google Scholar
Weimann, G., & Brosius, H.-B. (2017). Redirecting the agenda: Agenda-setting in the online Era. The Agenda Setting Journal, 1 (1), 63–101. DOI logoGoogle Scholar
Whitney, D. C., & Becker, L. B. (1982). ‘Keeping the gates’ for gatekeepers: The effects of wire news. Journalism Quarterly, 59 (1), 60–65. DOI logoGoogle Scholar