Cover not available
Article published In:
Study Abroad Research in Second Language Acquisition and International Education
Vol. 9:2 (2024) ► pp.217–244

Introducing regression discontinuity design to applied linguistics

The case of Japanese government study-abroad scholarships

ORCID logo | Waseda University
| Sophia University
| Keio University
| University of Melbourne
| Kansai University
This report presents regression discontinuity design (RDD) as a powerful analytical tool for use in applied linguistics showcased through our study of the impact of Japanese government study-abroad (SA) scholarships. RDD enables the estimation of causal effects in scenarios where a true experiment is not feasible by exploiting a naturally occurring cutoff point for treatment assignment. Because RDD may be novel to most readers in applied linguistics, this report provides a detailed step-by-step explanation of the standard RDD procedures our study exemplifies. The results section is crafted to reflect the conventional presentation style of RDD findings. Furthermore, the report’s concluding segment suggests scenarios within SA-related language learning research that could benefit from RDD application by enhancing the nuanced and precise interpretation of data in applied linguistics contexts.
Keywords: Regression discontinuity design, study-abroad scholarships, between-group design, within-group design, high school students
Article outline
  • 1.Introduction
  • 2.Past studies of impacts of SA experiences with special focus on methodology
  • 3.Regression discontinuity design (RDD)
    • 3.1Background
    • 3.2Standard RDD procedure
      • 3.2.1Step 1: Defining the assignment variable and cutoff
      • 3.2.2Step 2: Identifying the control group and ensuring continuity
      • 3.2.3Step 3: Choosing the bandwidth
      • 3.2.4Step 4: Graphical analysis and subsequent statistical testing
      • 3.2.5Step 5: Assessing the density of the assignment variable
      • 3.2.6Step 6: Determining robustness through multiple bandwidths
      • 3.2.7Step 7: Supplementing RDD with other statistical methods
      • 3.2.8Step 8: Interpreting reduced-form models
      • 3.2.9Step 9: Stage 1 for conducting the instrumental variable (IV) approach to estimate local average treatment effect (LATE)
      • 3.2.10Step 10: IV approach Stage 2 to estimating LATE
      • 3.2.11Step 11: Analyzing heterogeneous treatment effects
  • 4.Illustrative study: Effects of SA scholarships
    • 4.1Background
    • 4.2Outcome (dependent) variables
    • 4.3Research questions
    • 4.4Method
      • 4.4.1Participants
      • 4.4.2Outcome (dependent) variables
        • 4.4.2.1Perceived (Communication) Competence in the SA country’s main language
        • 4.4.2.2International Posture
      • 4.4.3Data collection procedure
      • 4.4.4Analyses
    • 4.5Results
    • 4.5.1Numbers of SA months between pre-SA and post-SA surveys
    • 4.5.2Perceived Competence in the SA country’s main language
    • 4.5.3International Posture
    • 4.5.4Regression results: Reduced-form models, IV-LATE, and heterogeneity of treatment effects
  • 4.6Discussion
  • 5.Conclusion and suggestions for possible application of RDD to applied linguistics studies
  • Notes
  • References
Published online: 13 May 2024
DOI logo
https://doi.org/10.1075/sar.21020.sas
References (37)
References
Angrist, J. D., & Pischke, J. S. (2010). The credibility revolution in empirical economics: How better research design is taking the con out of econometrics. Journal of Economic Perspectives, 24 (2), 3–30. DOI logoGoogle Scholar
Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford University Press.Google Scholar
Calonico, S., Cattaneo, M. D., & Titiunik, R. (2014). Robust nonparametric confidence intervals for regression-discontinuity designs. Econometrica, 82 ( 4 ), 2295–2326. DOI logoGoogle Scholar
Cattaneo, M. D., Idrobo, N., & Titiunik, R. (2019). A practical introduction to regression discontinuity designs: Foundations. Cambridge University Press. DOI logoGoogle Scholar
De Poli, S., Vergolini, L., & Zanini, N. (2018). The impact of a study abroad program on learning abilities and personality traits: Evidence from a randomization. Applied Economics Letters, 25 (8), 562–566. DOI logoGoogle Scholar
Di Pietro, G. D. (2015). Do study abroad programs enhance the employability of graduates? Education Finance and Policy, 10 (2), 223–243. DOI logoGoogle Scholar
Dörnyei, Z. (2014). Researching complex dynamic systems: Retrodictive qualitative modeling in the language classroom. Language Teaching, 47 (1), 80–91. DOI logoGoogle Scholar
Freed, B. F. (1995). Second language acquisition in a study abroad context. John Benjamins. DOI logoGoogle Scholar
Geeslin, K. L., & Schmidt, L. B. (2018). Study abroad and L2 learner attitudes. In C. Sanz & A. Morales-Font (Eds.), The Routledge handbook of study abroad research and practice (pp. 387–405). Routledge. DOI logoGoogle Scholar
Gelman, A., & Imbens, G. (2019). Why high-order polynomials should not be used in regression discontinuity designs. Journal of Business & Economic Statistics, 37 (3), 447–456. DOI logoGoogle Scholar
Gertler, P. J., Martínez, S., Premand, P., Rawlings, L. B., & Vermeersch, C. M. J. (2016). Impact evaluation in practice (2nd ed.). World Bank Group.Google Scholar
Grey, S. (2018). Quantitative approaches for study abroad research. In C. Sanz & A. Morales-Font (Eds.), The Routledge handbook of study abroad research and practice (pp. 48–57). Routledge. DOI logoGoogle Scholar
Higuchi, Y., Nakamuro, M., Roever, C., Sasaki, M., & Yashima, T. (2023). Impact of studying abroad on language skill development: Regression discontinuity evidence from Japanese university students. Journal of the Japanese and International Economies, 70 1, 101284. DOI logoGoogle Scholar
Imbens, G. W., & Kalyanaraman, K. (2012). Optimal bandwidth choice for the regression discontinuity estimator. Review of Economic Studies, 79 (3), 933–959. DOI logoGoogle Scholar
Imbens, G. W. & Lemieux, T. (2008). Regression discontinuity designs: A guide to practice. Journal of Econometrics, 142 (2), 615–635. DOI logoGoogle Scholar
Kato, M., & Suzuki, K. (2019). Effective or self-selective: Random assignment demonstrates short-term study abroad effectively encourages further study abroad. Journal of Studies in International Education, 23 (4), 411–428. DOI logoGoogle Scholar
Kawaguchi, D. (2016). Fewer school days, more inequality. Journal of the Japanese and International Economies, 39 1, 35–52. DOI logoGoogle Scholar
Lee, D. S., & Lemieux, T. (2010). Regression discontinuity designs in economics. Journal of Economic Literature, 48 (2), 281–355. DOI logoGoogle Scholar
MacIntyre, P. D., & Charos, C. (1996). Personality, attitudes, and affect as predictors of second language communication. Journal of Language and Social Psychology, 15 (1), 3–26. DOI logoGoogle Scholar
McCrary, J. (2008). Manipulation of the running variable in the regression discontinuity design: A density test. Journal of Econometrics, 142 (2), 698–714. DOI logoGoogle Scholar
Ministry of Education, Culture, Sports, Science, and Technology, Japan (MEXT). (2022a). About Tobitate! [Leap for Tomorrow] Study Abroad Initiative. Retrieved February 24, 2022, from [URL]
. (2022b). Tobitate! [Leap for Tomorrow] Ryugaku [Study Abroad] Japan. Retrieved February 24, 2022, from [URL]
Oosterbeek, H., & Webbink, D. (2011). Does studying abroad induce a brain drain? Economica, 78 (310), 347–366. DOI logoGoogle Scholar
Parey, M., & Waldinger, F. (2010). Studying abroad and the effect on international labor market mobility: Evidence from the introduction of ERASMUS. Economic Journal, 121 (551), 194–222. DOI logoGoogle Scholar
Robinson, J. P. (2011). Evaluating criteria for English learner reclassification: A causal-effects approach using a binding-score regression discontinuity design with instrumental variables. Educational Evaluation and Policy Analysis, 33 (3), 267–292. DOI logoGoogle Scholar
Sanz, C. & Morales-Front, A. (2018). Issues in study abroad research and practice. In C. Sanz & A. Morales-Font (Eds.), The Routledge handbook of study abroad research and practice (pp. 1–16). Routledge. DOI logoGoogle Scholar
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.Google Scholar
Taguchi, N. (2008). The role of the learning environment in the development of pragmatic comprehension: A comparison of gains between EFL and ESL learners. Studies in Second Language Acquisition, 30 (4), 423–452. DOI logoGoogle Scholar
Thistlethwaite, D. L., & Campbell, D. T. (1960). Regression-discontinuity analysis: An alternative to the ex post facto experiment. Journal of Educational Psychology, 51 (6), 309–317. DOI logoGoogle Scholar
Tullock, B., & Ortega, L. (2017). Fluency and multilingualism in study abroad: Lessons from a scoping review. System, 71 1, 7–21. DOI logoGoogle Scholar
Urquiola, M., & Verhoogen, E. (2009). Class-size caps, sorting, and the regression discontinuity design. American Economic Review, 99 (1), 179–215. DOI logo
Waibel, S., Rüger, H., Ette, A., & Sauer, L. (2017). Career consequences of transnational educational mobility: A systematic literature review. Educational Research Review, 20 1, 81–98. DOI logoGoogle Scholar
Xiao, F. (2015). Adult second language learners’ pragmatic development in the study-abroad context: A review. Frontiers: The Interdisciplinary Journal of Study Abroad, 25 (1), 132–149. DOI logoGoogle Scholar
Xu, Y. (2019). Changes in interlanguage complexity during study abroad: A meta-analysis. System, 80 1, 199–211. DOI logoGoogle Scholar
Yashima, T. (2002). Willingness to communicate in a second language: The Japanese EFL context. The Modern Language Journal, 86(1), 54–66. DOI logoGoogle Scholar
(2009). International posture and the ideal L2 self in the Japanese EFL context. In Z. Dörnyei, & E. Ushioda (Eds.), Motivation, language identity, and the L2 self (pp. 144–163). Multilingual Matters. DOI logoGoogle Scholar
Yashima, T., & Zenuk-Nishide, L. (2008). The impact of learning contexts on proficiency, attitudes, and L2 communication: Creating an imagined international community. System, 36 (4), 566–585. DOI logoGoogle Scholar