Cover not available
Part of
Polylogues on The Mental Lexicon: An exploration of fundamental issues and directions
Edited by Gary Libben, Gonia Jarema and Victor Kuperman
[Not in series 238] 2021
► pp. 17–44

Chapter 2
Meta-megastudies

| National Chung Cheng University
Article outline
  • Thesis
  • Commentaries on Myers thesis
    • Ray Jackendoff commentary on Myers thesis
    • Dorit Ravid commentary on Myers thesis
    • Russel Richie commentary on Myers thesis
    • Benjamin Tucker commentary on Myers thesis
    • Chris Westbury commentary on Myers
  • The article
    • The need for meta-megastudies
    • The challenges of meta-megastudies
    • Confounded variables
    • Typological language sampling
    • Infrastructure for meta-megastudies
    • Conclusions
  • References
Published online: 13 October 2021
DOI logo
https://doi.org/10.1075/z.238.02mye
References (104)
References
Adelman, J. S., Johnson, R. L., McCormick, S. F., McKague, M., Kinoshita, S., Bowers, J. S., Perry, J. R., Lupker, S. J., Forster, K. I., Cortese, M. J., Scaltritti, M., Aschenbrenner, A. J., Coane, J. H., White, L., Yap, M. J., Davis, C., Kim, J., & C. J. Davis. (2014). A behavioral database for masked form priming. Behavior Research Methods, 46(4), 1052–1067. DOI logoGoogle Scholar
Alario, F. X., Perre, L., Castel, C., & Ziegler, J. C. (2007). The role of orthography in speech production revisited. Cognition, 102(3), 464–475. DOI logoGoogle Scholar
Aronoff, M., Meir, I., & Sandler, W. (2005). The paradox of sign language morphology. Language, 81(2), 301–344. DOI logoGoogle Scholar
Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59(4), 390–412. DOI logoGoogle Scholar
Bader, M., & Häussler, J. (2010). Toward a model of grammaticality judgments. Journal of Linguistics, 46 (2), 273–330. DOI logoGoogle Scholar
Bailey, T. M., & Hahn, U. (2001). Determinants of wordlikeness: Phonotactics or lexical neighborhoods? Journal of Memory & Language, 44, 569–591. DOI logoGoogle Scholar
Balota, D. A., Yap, M. J., Cortese, M. J., Hutchison, K. A., Kessler, B., Loftis, B., Neely, J. H., Nelson, D. L., Simpson, G. B., & Treiman, R. (2007). The English Lexicon Project. Behavior Research Methods, 39, 445–459. DOI logoGoogle Scholar
Balota, D. A., Yap, M. J., Hutchison, K. A., & Cortese, M. J. (2012). Megastudies: What do millions (or so) of trials tell us about lexical processing? In J. S. Adelman (Ed). Visual word recognition, Vol. 1 (pp. 90–115). London: Psychology Press Psychology Press.Google Scholar
Bates, E., D’Amico, S., Jacobsen, T., Székely, A., Andonova, E., Devescovi, A., Herron, D., Lu, C-C., Pechmann, T., Pléh, C., Wicha, N., Federmeier, K., Gerdjikova, I., Gutierrez, G., Hung, D., Hsu, J., Iyer, G., Kohnert, K., Mehotcheva, T., Orozco-Figueroa, A., Tzeng, A., & Tzeng, O. (2003). Timed picture naming in seven languages. Psychonomic Bulletin & Review, 10 (2), 344–380. DOI logoGoogle Scholar
Bates, E., Devescovi, A., & Wulfeck, B. (2001). Psycholinguistics: A cross-language perspective. Annual Review of Psychology, 52, 369–96. DOI logoGoogle Scholar
Benkı́, J. R. (2003). Quantitative evaluation of lexical status, word frequency, and neighborhood density as context effects in spoken word recognition. The Journal of the Acoustical Society of America, 113(3), 1689–1705. DOI logoGoogle Scholar
Bi, Y., Wei, T., Janssen, N., & Han, Z. (2009). The contribution of orthography to spoken word production: Evidence from Mandarin Chinese. Psychonomic Bulletin & Review, 16(3), 555–560. DOI logoGoogle Scholar
Bickel, B. (2015). Distributional typology: Statistical inquiries into the dynamics of linguistic diversity. In B. Heine & H. Narrog (Eds). The Oxford Handbook of Linguistic Analysis, 2nd edition (pp. 901–923). Oxford: Oxford University Press.Google Scholar
Bishop, S. (2014). Science exposed. Scientific American, 311(4), 46. DOI logoGoogle Scholar
Brown, C. H., Holman, E. W., & Wichmann, S. (2013). Sound correspondences in the world’s languages. Language, 89 (1), 4–29. DOI logoGoogle Scholar
Cai, Z.-Q. (Ed.) (2008). How to read Chinese poetry: A guided anthology. New York: Columbia University Press.Google Scholar
Caselli, N. K., & Cohen-Goldberg, A. M. (2014). Lexical access in sign language: A computational model. Frontiers in Psychology, 5, 428. DOI logoGoogle Scholar
Chen, J.-Y., Chen, T.-M., & Dell, G. S. (2002). Word-form encoding in Mandarin Chinese as assessed by the implicit priming task. Journal of Memory and Language, 46(4), 751–781. DOI logoGoogle Scholar
Chen, T.-M., Dell, G., & Chen, J.-Y. (2007). A cross-linguistic study of phonological units: Syllables emerge from the statistics of Mandarin Chinese, but not from the statistics of English. Chinese Journal of Psychology, 49(2), 137–144.Google Scholar
Chen, T.-Y., & Myers, J. (2021). Worldlikeness: A Web crowdsourcing platform for typological psycholinguistics. Linguistic Vanguard, 7(s1), 20190011. DOI logoGoogle Scholar
Clark, H. (1973). The language-as-fixed-effect fallacy: A critique of language statistics in psychological research. Journal of Verbal Learning and Verbal Behavior, 12, 335–359. DOI logoGoogle Scholar
Clauson, K. A., Polen, H. H., Boulos, M. N. K., & Dzenowagis, J. H. (2008). Scope, completeness, and accuracy of drug information in Wikipedia. Annals of Pharmacotherapy, 42(12), 1814–1821. DOI logoGoogle Scholar
Cohen-Goldberg, A. M. (2012). Phonological competition within the word: Evidence from the phoneme similarity effect in spoken production. Journal of Memory and Language, 67(1), 184–198. DOI logoGoogle Scholar
Coleman, T., & Greif, S. (2013). Discover Meteor. URL: [URL]
Costa, A., Alario, F. X., & Sebastián-Gallés, N. (2007). Cross-linguistic research on language production. In M. G. Gaskell (Ed.), The Oxford handbook of psycholinguistics (pp. 531–546). Oxford: Oxford University Press.Google Scholar
Cutler, A. (1981). Making up materials is a confounded nuisance, or: Will we be able to run any psycholinguistic experiments at all in 1990? Cognition, 10, 65–70. DOI logoGoogle Scholar
(1985). Cross-language psycholinguistics. Linguistics, 23, 659–667. DOI logoGoogle Scholar
Cysouw, M. (2005). Quantitative methods in typology. In R. Kohler, G. Altmann, & R. G. Piotrowski (Eds.) Quantitative Linguistik: Ein internationales Handbuch [Quantitative linguistics: An international handbook] (pp. 554–578). Berlin: Walter de Gruyter.Google Scholar
DeFrancis, J. (1989). Visible speech: The diverse oneness of writing systems. Honolulu: University of Hawaii Press.Google Scholar
Dryer, M. S. (1992). The Greenbergian word order correlations. Language, 68 (1), 81–138. DOI logoGoogle Scholar
(2011). The evidence for word order correlations. Linguistic Typology, 15(2), 335–380. DOI logoGoogle Scholar
Dufau, S., Duñabeitia, J. A., Moret-Tatay, C., McGonigal, A., Peeters, D., Alario, F. X., Balota, D. A., Brysbaert, M., Carreiras, M., Ferrand, L., Ktori, M., Perea, M., Rastle, K., Sasburg, O., Yap, M. J., Ziegler, J. C., & Grainger, J. (2011). Smart phone, smart science: How the use of smartphones can revolutionize research in cognitive science. PloS One, 6(9), e24974. DOI logoGoogle Scholar
Durrett, G., & DeNero, J. (2013). Supervised learning of complete morphological paradigms. Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pp. 1185–1195.
Erlewine, M. Y., & Kotek, H. (2016). A streamlined approach to online linguistic surveys. Natural Language & Linguistic Theory, 34(2), 481–495. DOI logoGoogle Scholar
Ernestus, M., & Cutler, A. (2015). BALDEY: A database of auditory lexical decisions. The Quarterly Journal of Experimental Psychology, 68(8), 1469–1488. DOI logoGoogle Scholar
Evans, N., & Levinson, S. C. (2009). The myth of language universals: Language diversity and its importance for cognitive science. Behavioral and Brain Sciences, 32 (5), 429–492. DOI logoGoogle Scholar
Feng, G., Miller, K., Shu, H., & Zhang, H. (2001). Rowed to recovery: the use of phonological and orthographic information in reading Chinese and English. Journal of Experimental Psychology: Learning, Memory, and Cognition, 27(4), 1079–1100.Google Scholar
Fenlon, J., Schembri, A., Rentelis, R., Vinson, D., & Cormier, K. (2014). Using conversational data to determine lexical frequency in British Sign Language: The influence of text type. Lingua, 143, 187–202. DOI logoGoogle Scholar
Ferrand, L., New, B., Brysbaert, M., Keuleers, E., Bonin, P., Méot, A., Augustinova, M., & Pallier, C. (2010). The French Lexicon Project: Lexical decision data for 38,840 French words and 38,840 pseudowords. Behavior Research Methods, 42(2), 488–496. DOI logoGoogle Scholar
Forster, K. I. (2000). The potential for experimenter bias effects in word recognition experiments. Memory & Cognition, 28(7), 1109–1115. DOI logoGoogle Scholar
Fox, R. A. (1984). Effect of lexical status on phonetic categorization. Journal of Experimental Psychology: Human Perception and Performance, 10(4), 526–540.Google Scholar
Fromkin, V. A. (1971). The non-anomalous nature of anomalous utterances. Language, 47(1), 27–52. DOI logoGoogle Scholar
Gandour, J., Xu, Y., Wong, D., Dzemidzic, M., Lowe, M., Li, X., & Tong, Y. (2003). Neural correlates of segmental and tonal information in speech perception. Human Brain Mapping, 20(4), 185–200. DOI logoGoogle Scholar
Graham, J., Nosek, B. A., Haidt, J., Iyer, R., Koleva, S., & Ditto, P. H. (2011). Mapping the moral domain. Journal of Personality and Social Psychology, 101(2), 366–385. DOI logoGoogle Scholar
Greenberg, J. H. (1963). Some universals of grammar with particular reference to the order of meaningful elements. In J. H. Greenberg (Ed.) Universals of language (pp. 73–113). Cambridge, MA: MIT Press.Google Scholar
Haspelmath, M., Dryer, M. S., Gil, D., & Comrie, B. (Eds.) (2005). The world atlas of language structure. Oxford: Oxford University Press.Google Scholar
Heylighen, F. (2007). Why is open access development so successful? Stigmergic organization and the economics of information. In Lutterbeck, B., Bärwolff, M., & Gehring, R. A. (Eds.) Open Source Jahrbuch 2007. Berlin: Technical University of Berlin.Google Scholar
Hutchison, K. A., Balota, D. A., Neely, J. H., Cortese, M. J., Cohen-Shikora, E. R., Tse, C. S., Yap, M. J., Bengson, J. J., Niemeyer, D., & Buchanan, E. (2013). The semantic priming project. Behavior Research Methods, 45(4), 1099–1114. DOI logoGoogle Scholar
Iacoponi, L., & Savy, R. (2011). Sylli: Automatic phonological syllabification for Italian. INTERSPEECH 2011, 641–644. DOI logoGoogle Scholar
Jaeger, T. F., & Norcliffe, E. J. (2009). The cross-linguistic study of sentence production. Language and Linguistics Compass, 3/4, 866–887. DOI logoGoogle Scholar
Keuleers, E., & Balota, D. A. (2015). Megastudies, crowdsourcing, and large datasets in psycholinguistics: An overview of recent developments. The Quarterly Journal of Experimental Psychology, 68 (8), 1457–1468. DOI logoGoogle Scholar
Keuleers, E., & Brysbaert, M. (2010). Wuggy: A multilingual pseudoword generator. Behavior Research Methods, 42(3), 627–633. DOI logoGoogle Scholar
Keuleers, E., Diependaele, K., & Brysbaert, M. (2010). Practice effects in large-scale visual word recognition studies: A lexical decision study on 14,000 Dutch mono-and disyllabic words and nonwords. Frontiers in Psychology, 1, 174. DOI logoGoogle Scholar
Keuleers, E., Lacey, P., Rastle, K., & Brysbaert, M. (2012). The British Lexicon Project: Lexical decision data for 28,730 monosyllabic and disyllabic English words. Behavior Research Methods, 44(1), 287–304. DOI logoGoogle Scholar
Keuleers, E., Stevens, M., Mandera, P., & Brysbaert, M. (2015). Word knowledge in the crowd: Measuring vocabulary size and word prevalence in a massive online experiment. The Quarterly Journal of Experimental Psychology, 68(8), 1665–1692. DOI logoGoogle Scholar
Kirby, J. P., & Yu, A. C. L. (2007). Lexical and phonotactic effects on wordlikeness judgments in Cantonese. Proceedings of the International Congress of Phonetic Sciences, 16, 1389–1392.Google Scholar
Klein, D., Zatorre, R. J., Milner, B., & Zhao, V. (2001). A cross-linguistic PET study of tone perception in Mandarin Chinese and English speakers. Neuroimage, 13(4), 646–653. DOI logoGoogle Scholar
Kupferberg, N., & Protus, B. M. (2011). Accuracy and completeness of drug information in Wikipedia: An assessment. Journal of the Medical Library Association, 99(4), 310–313. DOI logoGoogle Scholar
Lee, H.-H. (2016). A comparative study of the phonology of Taiwan Sign Language and Signed Chinese. Unpublished National Chung Cheng University Ph.D. thesis.Google Scholar
Lemhöfer, K., Dijkstra, T., Schriefers, H., Baayen, R. H., Grainger, J., & Zwitserlood, P. (2008). Native language influences on word recognition in a second language: A megastudy. Journal of Experimental Psychology: Learning, Memory, and Cognition, 34 (1), 12–31.Google Scholar
Levelt, W. J., Roelofs, A., & Meyer, A. S. (1999). A theory of lexical access in speech production. Behavioral and Brain Sciences, 22(01), 1–38. DOI logoGoogle Scholar
Lewis, M. P., Simons, G. F., Fennig, C. D. (Eds.). (2014). Ethnologue: Languages of the world, Seventeenth edition. Dallas, Texas: SIL International. Online version: [URL]
Light, T. (1977). The Cantonese final: An exercise in indigenous analysis. Journal of Chinese Linguistics, 5(1), 75–102.Google Scholar
Luce, P. A., & Large, N. R. (2001). Phonotactics, density, and entropy in spoken word recognition. Language & Cognitive Processes, 16(5/6), 565–581. DOI logoGoogle Scholar
Malins, J. G., & Joanisse, M. F. (2010). The roles of tonal and segmental information in Mandarin spoken word recognition: An eyetracking study. Journal of Memory and Language, 62(4), 407–420. DOI logoGoogle Scholar
Malmqvist, G. (1994). Chinese linguistics. In G. Lepschy (Ed.), History of linguistics: Volume I: The Eastern traditions of linguistics (pp. 1–24). London: Longman.Google Scholar
McBride-Chang, C., Tong, X., Shu, H., Wong, A. M. Y., Leung, K. W., & Tardif, T. (2008). Syllable, phoneme, and tone: Psycholinguistic units in early Chinese and English word recognition. Scientific Studies of Reading, 12(2), 171–194. DOI logoGoogle Scholar
Meier, R. P. (2002). Why different, why the same? Explaining effects and non-effects of modality upon linguistic structure in sign and speech. In R. P. Meier & K. Cormier (Eds.) Modality and structure in signed and spoken languages (pp. 1–25). Cambridge, UK: Cambridge University Press. DOI logoGoogle Scholar
Moscoso del Prado Martín, F., Kostić, A., & Baayen, R. H. (2004). Putting the bits together: An information theoretical perspective on morphological processing. Cognition, 94(1), 1–18. DOI logoGoogle Scholar
Myers, J. (2007). Generative morphology as psycholinguistics. In G. Jarema & G. Libben (Eds.), The mental lexicon: Core perspectives (pp. 105–128). Amsterdam: Elsevier. DOI logoGoogle Scholar
(2012). Chinese as a natural experiment. In G. Libben, G. Jarema, & C. Westbury (Eds.), Methodological and analytic frontiers in lexical research (pp. 155–169). Amsterdam: John Benjamins. DOI logoGoogle Scholar
(2015). Markedness and lexical typicality in Mandarin acceptability judgments. Language & Linguistics, 16 (6). DOI logoGoogle Scholar
Myers, J., & Chen, T-Y. (2016). The time course of sociolinguistic influences on wordlikeness judgments. In A. Botinis (Ed.), Proceedings of the 7th Tutorial and Research Workshop on Experimental Linguistics (pp. 119–122). International Speech Communication Association.Google Scholar
Myers, J., & Tsay, J. (2005). The processing of phonological acceptability judgments. Proceedings of Symposium on 90–92 NSC Projects (pp. 26–45). Taipei, Taiwan, May.
Nathan, D. (2013). Access and accessibility at ELAR, a social networking archive for endangered languages documentation. In M. Turin, C. Wheeler & E. Wilkinson (Eds.) Oral literature in the digital age: Archiving orality and connecting with communities. Cambridge, UK: Open Book Publishers.Google Scholar
Newmeyer, F. J. (2005). Possible and probable languages: A generative perspective on linguistic typology. Oxford: Oxford University Press. DOI logoGoogle Scholar
Nielsen, M. (2012). Reinventing discovery: The new era of networked science. Princeton, NJ: Princeton University Press.Google Scholar
Norris, D., & Cutler, A. (1988). The relative accessibility of phonemes and syllables. Perception & Psychophysics, 43(6), 541–550. DOI logoGoogle Scholar
O’Brien, R. M. (2007). A caution regarding rules of thumb for Variance Inflation Factors. Quality & Quantity, 41, 673–690. DOI logoGoogle Scholar
O’Seaghdha, P. G., Chen, J.-Y., & Chen, T.-M. (2010). Proximate units in word production: Phonological encoding begins with syllables in Mandarin Chinese but with segments in English. Cognition, 115, 282–302. DOI logoGoogle Scholar
(2013). Close but not proximate: The significance of phonological segments in speaking depends on their functional engagement. Proceedings of the National Academy of Sciences, 110(1), E3. DOI logoGoogle Scholar
Peirce, J. W., & MacAskill, M. R. (2018). Building experiments in PsychoPy. London: Sage.Google Scholar
Qu, Q., Damian, M. F., & Kazanina, N. (2012). Sound-sized segments are significant for Mandarin speakers. Proceedings of the National Academy of Sciences, 109(35), 14265–14270. DOI logoGoogle Scholar
(2013). Reply to O’Seaghdha et al.: Primary phonological planning units in Chinese are phonemically specified. Proceedings of the National Academy of Sciences, 110(1), E4. DOI logoGoogle Scholar
Rastle, K., McCormick, S. F., Bayliss, L., & Davis, C. J. (2011). Orthography influences the perception and production of speech. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37(6), 1588–1594.Google Scholar
Rentzepopoulos, P. A., & Kokkinakis, G. K. (1996). Efficient multilingual phoneme-to-grapheme conversion based on HMM. Computational Linguistics, 22(3), 351–376.Google Scholar
Rice, S., Libben, G., & Derwing, B. (2002). Morphological representation in an endangered, polysynthetic language. Brain and Language, 81(1), 473–486. DOI logoGoogle Scholar
Sampson, G. (2015). A Chinese phonological enigma. Journal of Chinese Linguistics, 43, 679–691. DOI logoGoogle Scholar
Sandler, W. (1999). Cliticization and prosodic words in a sign language. In T. A. Hall and U. Kleinhenz (Eds.) Studies on the phonological word (pp. 223–255). Amsterdam: John Benjamins. DOI logoGoogle Scholar
Sandler, W., & Lillo-Martin. (2006). Sign language and linguistic universals. Cambridge, UK: Cambridge University Press. DOI logoGoogle Scholar
Silvertown, J. (2009). A new dawn for citizen science. Trends in Ecology & Evolution, 24(9), 467–471. DOI logoGoogle Scholar
Singleton, J. L., & Newport, E. L. (2004). When learners surpass their models: The acquisition of American Sign Language from inconsistent input. Cognitive Psychology 49, 370–407. DOI logoGoogle Scholar
Sze, W. P., Liow, S. J. R., & Yap, M. J. (2014). The Chinese Lexicon Project: A repository of lexical decision behavioral responses for 2,500 Chinese characters. Behavior Research Methods, 46(1), 263–273. DOI logoGoogle Scholar
Szekely, A., Jacobsen, T., D’Amico, S., Devescovi, A., Andonova, E., Herron, D., Lu, C.-C., Pechmann, T., Pléh, C., Wicha, N., Federmeier, K., Gerdjikova, I., Gutierrez, G., Hung, D., Hsu, J., Iyer, G., Kohnert, K., Mehotcheva, T., Orozco-Figueroa, A., Tzeng, A., Tzeng, O., Arévalo, A., Vargha, A., Butler, A. C., Buffngton, R., & Bates, E. (2004). A new on-line resource for psycholinguistic studies. Journal of Memory and Language, 51(2), 247–250. DOI logoGoogle Scholar
Tse, C.-S., Yap, M. J., Chan, Y.-L., Sze, W.-P., Shaoul, C., & Lin, D. (forthcoming). The Chinese Lexicon Project: A megastudy of lexical decision performance for 25,000+ traditional Chinese two-character compound words. Behavior Research Methods. DOI logo
Tseng, C.-H., Huang, K.-Y., & Jeng, J.-Y. (1996). The role of the syllable in perceiving spoken Chinese. Proceedings of the National Science Council, Part C: Humanities and Social Sciences, 6 (1), 71–86.Google Scholar
Vittinghoff, E., & McCulloch, C. E. (2007). Relaxing the rule of ten events per variable in logistic and Cox regression. American Journal of Epidemiology, 165(6), 710–718. DOI logoGoogle Scholar
von Bastian, C. C., Locher, A., & Ruflin, M. (2013). Tatool: A Java-based open-source programming framework for psychological studies. Behavior Research Methods, 45(1), 108–115. DOI logoGoogle Scholar
Weskott, T., & Fanselow, G. (2011). On the informativity of different measures of linguistic acceptability. Language, 87 (2), 249–273. DOI logoGoogle Scholar
Wong, A. W.-K., & Chen, H.-C. (2008). Processing segmental and prosodic information in Cantonese word production. Journal of Experimental Psychology: Learning, Memory and Cognition, 34 (5), 1172–1190.Google Scholar
Yap, M. J., Liow, S. J. R., Jalil, S. B., & Faizal, S. S. B. (2010). The Malay Lexicon Project: A database of lexical statistics for 9,592 words. Behavior Research Methods, 42(4), 992–1003. DOI logoGoogle Scholar
Yu, L., & Shu, H. (2003). Hanyu yanyu chansheng de yuyin jiagong jizhi [Phonological processing mechanism in Chinese speech production]. Xinli Kexue, 26 (5), 818–822.Google Scholar
Yu, M., Mo, C., & Mo, L. (2014). The role of phoneme in Mandarin Chinese production: Evidence from ERPs. PloS one 9 (9), e106486. DOI logoGoogle Scholar
Yu, M., Mo, C., Li, Y., & Mo, L. (2015). Distinct representations of syllables and phonemes in Chinese production: Evidence from fMRI adaptation. Neuropsychologia, 77, 253–259. DOI logoGoogle Scholar