Article published in:
Sign Language & Linguistics
Vol. 24:2 (2021) ► pp. 226–258

Challenges and solutions in test adaption

Comparing international experiences with the British Sign Language Production Test (Narrative Skills)

| University of Manitoba University of Manitoba
| Humboldt-Universität zu Berlin Humboldt-Universität zu Berlin
| Humboldt-Universität zu Berlin Humboldt-Universität zu Berlin

Sign language assessment tools are important for professionals working with DHH children to measure sign language development and competence. Adaptation of an existing test can be a solution when initiating assessment in a sign language community; the adaptation process must adhere to key principles and procedures. We introduce the principles of test adaptation and outline the challenges we faced in adapting the British Sign Language Production Test (Herman, Grove, Holmes, Morgan, Sutherland & Woll 2004) to German Sign Language and American Sign Language. Challenges included decisions regarding the normative sample, the use of terminology, and variations in the scoring protocols to fit with each language. The steps taken throughout the test adaptation process are described, together with a comparison of parallels and differences. We conclude that test adaptation is an effective method of developing practical tools for sign language assessment and contributes to a better understanding of sign language development.

Keywords: sign language assessment, test adaptation, test adaptation principles, test adaptation challenges, score sheets, sign language development
Article outline
  • 1.Introduction
  • 2.Principles of test adaption
  • 3.Description of the original test
  • 4.Structure of test adaptation process
  • 5.Challenges and solutions from two adaptation processes
    • 5.1Normative samples
    • 5.2Terminology
    • 5.3Revisions to scoring system
      • Combining narrative structure and content
      • Expanding role shift / constructed action
      • Item substitution
      • Recording systems
  • 6.Conclusion
  • Acknowledgements
  • References
Published online: 19 July 2021
https://doi.org/10.1075/sll.20010.enn
References
Aarons, Debra & Ruth Morgan
2003 Classifier predicates and the creation of multiple perspectives in South African Sign Language. Sign Language Studies 3(2). 125–156. DOI logoGoogle Scholar
Anderson, Diane
2006 Lexical development of deaf children acquiring signed languages. In Brenda Schick (ed.), Advances in the sign language development of deaf children, 135–160. New York, NY: Oxford University Press.Google Scholar
Baker, Anne, Beppie van den Bogaerde, Roland Pfau & Trude Schermer
2016 The linguistics of sign languages: An introduction. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Becker, Claudia
2009 Narrative competences of Deaf children in German Sign Language. Sign Language & Linguistics 12(2). 113–160. DOI logoGoogle Scholar
Becker, Claudia, Martje Hansen & Patricia Barbeito Rey-Geissler
2018 Narrative Kompetenzen hörgeschädigter Kinder – Die Interaktion von Gebärdenspracherwerb und Theory of Mind. Das Zeichen 1081. 90–105.Google Scholar
Boudreault, Patrick, Kyra Zimmer & Charlotte Enns
2015 Creating videos to assess children’s signed language narrative skills. Proceedings of the 22nd International Congress on the Education of the Deaf (ICED 2015) . Athens, Greece, 6–9 July.
Brown, Linda, Rita J. Sherbenou & Susan K. Johnsen
2010 Test of nonverbal intelligence, 4th edition. New York, NY: Pearson.Google Scholar
Chen Pichler, Deborah
2012 Acquisition. In Roland Pfau, Markus Steinbach & Bencie Woll (eds.), Sign language: An international handbook, 647–686. Berlin: De Gruyter Mouton. DOI logoGoogle Scholar
Cokely, Dennis & Charlotte Baker
1991 American Sign Language: A teacher’s resource text on grammar and culture. Washington, DC: Gallaudet University Press.Google Scholar
Cormier, Kearsy, David Quinto-Pozos, Zed Sevcikova & Adam Schembri
2012 Lexicalisation and delexicalisation processes in sign languages: Comparing depicting constructions and viewpoint gestures. Language & Communication 321. 329–348. DOI logoGoogle Scholar
Cormier, Kearsy, Sandra Smith & Zed Sevcikova
2013 Predicate structures, gesture, and simultaneity in the representation of action in British Sign Language: Evidence from deaf children and adults. Journal of Deaf Studies and Deaf Education 18(3). 370–390. DOI logoGoogle Scholar
Council of Europe
2001 Common European Framework of References for Languages: Learning, teaching, assessment. Retrieved from https://​rm​.coe​.int​/CoERM​Public​Common​Search​Ser​vices​/DisplayDCTMContent​?documentId​=0900001680459f97. Access date 2020-​03-​02.
Cravens, Elisabeth
2013 Evaluating the utility of the Test of Narrative Language for use with deaf children via American Sign Language. Austin, TX: University of Texas Master’s thesis.
Dudis, Paul
2004 Body partitioning and real-space blends. Cognitive Linguistics 15(2). 223–238. DOI logoGoogle Scholar
Dunn, Lloyd M. & Douglas M. Dunn
1997 Peabody picture vocabulary test (3rd ed.). Circle Pines, MN: American Guidance Services.Google Scholar
ELAN (Version 5.5) [Computer software]
(2019) Nijmegen: Max Planck Institute for Psycholinguistics, The Language Archive. Retrieved from https://​archive​.mpi​.nl​/tla​/elan
Emmorey, Karen & Judy Reilly
1998 The development of quotation and reported action: Conveying perspective in ASL. In Eve V. Clark (ed.), Proceedings of the Twenty-ninth Annual Stanford Child Language Research Forum, 81–90. Stanford, CA: CSLI Publications.Google Scholar
Enns, Charlotte, Kyra Zimmer, Patrick Boudreault, Sarah Rabu & Cheryle Broszeit
2013 American Sign Language Receptive Skills Test. Winnipeg, MB: Northern Signs Research. www​.northernsignsresearch​.com
Enns, Charlotte, Tobias Haug, Rosalind Herman, Robert Hoffmeister, Wolfgang Mann & Lynn McQuarrie
2016 Exploring signed language assessment tools around the world. In Marc Marschark, Venetta Lampropoulou & Emmanouil K. Skordilis (eds.), Diversity in deaf education, 171–218. New York, NY: Oxford University Press. DOI logoGoogle Scholar
Enns, Charlotte, Kyra Zimmer, Cheryle Broszeit & Sarah Rabu
2019 American Sign Language Expressive Skills Test. Winnipeg, MB: Northern Signs Research.Google Scholar
Erlenkamp, Sonja
2012 Syntax: Aus Gebärden Sätze bilden. In Hanna Eichmann, Martje Hansen & Jens Heßmann (eds.), Handbuch Deutsche Gebärdensprache: sprachwissenschaftliche und anwendungsbezogene Perspektiven, 165–198. Seedorf: Signum-Verlag.Google Scholar
Fischer, Renate & Simon Kollien
2006 Constructed action in DGS: Roses Aktions = Fragmente (Teil II). Das Zeichen 741. 448–463.Google Scholar
Gagarina, Natalia, Daleen Klop, Sari Kunnari, Koula Tantele, Taina Välimaa, Ingrida Balčiūnienė, Ute Bohnacker & Joel Walters
2012 MAIN – Multilingual Assessment Instrument for Narratives. ZAS Papers in Linguistics 561. 1–155.Google Scholar
Hall, Wyatte
2017 What you don’t know can hurt you: The risk of language deprivation by impairing sign language development in Deaf children. Maternal and Child Health Journal 21(5). 961–965. DOI logoGoogle Scholar
Hänel, Barbara
2005 Der Erwerb der Deutschen Gebärdensprache als Erstsprache: die frühkindliche Sprachentwicklung von Subjekt- und Objektverbkongruenz in DGS. Tübingen: Narr.Google Scholar
Happ, Daniela & Marc-Oliver Vorkörper
2014 Deutsche Gebärdensprache: Ein Lehr- und Arbeitsbuch. Frankfurt: Fachhochschulverlag.Google Scholar
Harris, Raychelle, Heidi Holmes & Donna Mertens
2009 Research ethics in sign language communities. Sign Language Studies 9(2). 104–131. DOI logoGoogle Scholar
Hastie, Trevor & Robert Tibshirani
1990 Generalized additive models. London: Chapman and Hall.Google Scholar
Haug, Tobias & Wolfgang Mann
2008 Adapting tests of sign language assessment for other sign languages – A review of linguistic, cultural, and psychometric problems. Journal of Deaf Studies and Deaf Education 13(1). 138–147. DOI logoGoogle Scholar
Haug, Tobias
2011 Approaching sign language test construction: Adaptation of the German Sign Language receptive skills test. Journal of Deaf Studies and Deaf Education 16(3). 343–361. DOI logoGoogle Scholar
Henner, Jon, Catherine Caldwell-Harris, Rama Novodgrodsky & Robert Hoffmeister
2016 American Sign Language syntax and analogical reasoning skills are influenced by early acquisition and age of entry to signing schools for the deaf. Frontiers in Psychology 261. DOI logoGoogle Scholar
Herman, Rosalind
1998 The need for an assessment of deaf children’s signing skills. Deafness and Education: Journal of the British Association of the Teachers of the Deaf 22(3). 3–8.Google Scholar
2015 Language assessment of Deaf learners. In Harry Knoors & Marc Marschark (eds.), Educating Deaf learners: Creating a global evidence base. New York, NY: Oxford University Press. DOI logoGoogle Scholar
Herman, Rosalind, Sallie Holmes & Bencie Woll
1999 Assessing BSL development: Receptive skills test. Coleford, UK: Forest Bookshop.Google Scholar
Herman, Rosalind, Nicola Grove, Sallie Holmes, Gary Morgan, Hillary Sutherland & Bencie Woll
2004 Assessing BSL development: Production test (narrative skills). London, UK: City University.Google Scholar
Hoffmeister, Robert, Catherine Caldwell-Harris, Jonathan Henner, Rachel Benedict, Sarah Fish, Patrick Rosenburg, Frances Conlin-Luippold & Rama Novogrodsky
2014 The American Sign Language Assessment Instrument (ASLAI): Progress report and preliminary findings. Working paper. Boston, MA: Center for the Study of Communication and the Deaf.Google Scholar
Iliescu, Dragos
2017 Adapting tests in linguistic and cultural situations. New York, NY: Cambridge University Press. DOI logoGoogle Scholar
International Test Commission
2017 The ITC guidelines for translating and adapting tests (Second edition). Retrieved from www​.InTestCom​.org​/page​/16. Access Date 2020-06-02.
Johnston, Trevor
2004 The assessment and achievement of proficiency in a native sign language within a sign bilingual program: The pilot Auslan Receptive Skills Test. Deafness and Education International 6(2). 57–81. DOI logoGoogle Scholar
Johnston, Trevor & Adam Schembri
2007 Australian Sign Language. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Jones, Neil
2012 Reliability and dependability. In Glenn Fulcher & Fred Davidson (eds.), The Routledge handbook of language testing, 350–362. London, UK: Routledge.Google Scholar
Klima, Edward & Ursula Bellugi
1979 The signs of language. Cambridge, MA: Harvard University Press.Google Scholar
Kolbe, Vera
2019 Beobachtungsverfahren für Deutsche Gebärdensprache. Hörgeschädigtenpädagogik 3/2019 169–174. DOI logoGoogle Scholar
Levshina, Natalia
2015 How to do linguistics with R: data exploration and statistical analysis. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Liddell, Scott
2003 Grammar, gesture, and meaning in American Sign Language. New York: Cambridge University Press. DOI logoGoogle Scholar
Labov, William & Joshua Waletsky
1967 Oral versions of personal experiences. In June Helm (ed.), Essays on the verbal and visual arts, 12–44. Seattle, WA: University of Washington Press.Google Scholar
Lillo-Martin, Diane & Richard Meier
2011 On the linguistic status of ‘agreement’ in sign languages. Theoretical Linguistics 371. 95–142. DOI logoGoogle Scholar
Lillo-Martin, Diane
2012 Utterance reports and constructed action in sign and spoken languages. In Roland Pfau, Markus Steinbach & Bencie Woll (eds.), Sign language: An international handbook, 365–387. Berlin: De Gruyter Mouton.Google Scholar
Meier, Richard P.
1991 Language acquisition by Deaf children. American Scientist 79(1). 60–70.Google Scholar
2002 Why different, why the same? Explaining effects and non-effects of modality upon linguistic structure in sign and speech. In Richard P. Meier, Kearsy Cormier & David Quinto-Pozos (eds.), Modality and structure in signed and spoken language, 1–25. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Meristo, Marek, Kerstin Falkman, Erland Hjelmquist, Mariantonia Tedoldi, Luca Surian & Michael Siegal
2007 Language and theory of mind reasoning: Evidence from deaf children in bilingual and oralist environments. Developmental Psychology 431. 1156–1169. DOI logoGoogle Scholar
Mitchell, Ross & Michael Karchmer
2004 Chasing the mythical ten percent: Parental hearing status of deaf and hard of hearing students in the United States. Sign Language Studies 41. 138–163. DOI logoGoogle Scholar
Neidle, Carol, Judy Kegl, Dawn MacLaughlin, Benjamin Bahan & Robert Lee
2000 The syntax of American Sign Language: Functional categories and hierarchical structure. Cambridge, MA: MIT Press.Google Scholar
Morgan, Gary & Bencie Woll
2007 Understanding sign language classifiers through a polycomponential approach. Lingua 1171. 1159–1168. DOI logoGoogle Scholar
Novogrodsky, Rama & Natalia Meir
2020 Age, frequency, and iconicity in early sign language acquisition: Evidence from the Israeli Sign Language MacArthur-Bates Communicative Developmental Inventory. Applied Psycholinguistics 41(4). 817–845. DOI logoGoogle Scholar
Oakland, Thomas & Hollie Lane
2004 Language, reading and readability formulas: Implications for developing and adapting tests. International Journal of Testing 4(3). 239–252. DOI logoGoogle Scholar
O’Sullivan, Barry & Cyril Weir
2011 Test development and validation. In O’Sullivan, Barry (ed.), Language testing: theories and practices (1st publ. ed.). Basingstoke: Palgrave Macmillan.Google Scholar
Papaspyrou, Chrissostomos, Alexander von Meyenn, Michaela Matthaei & Bettina Herrmann
2008 Grammatik der Deutschen Gebärdensprache aus der Sicht gehörloser Fachleute. Seedorf: Signum Verlag.Google Scholar
Quinto-Pozos, David
2007 Can constructed action be considered obligatory? Lingua 1171. 1285–1314. DOI logoGoogle Scholar
R Core Team
2019 R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. https://​www​.R​-project​.org/Google Scholar
Reilly, Judy
2005 How faces come to serve grammar: The development of nonmanual morphology in American Sign Language. In Brenda Schick, Marc Marschark & Patricia Elisabeth Spencer (eds.), Advances in the sign language development of deaf children, 262–290. New York, NY: Oxford University Press. DOI logoGoogle Scholar
Reckwitz, Andreas
2011 Die Kontingenzperspektive der ›Kultur‹. Kulturbegriffe, Kulturtheorien und das kulturwissenschaftliche Forschungsprogramm. In Friedrich Jaeger & Burkhard Liebsch (eds.), Handbuch der Kulturwissenschaften: Grundlagen und Schlüsselbegriffe, 1–20. Stuttgart: J.B. Metzler. DOI logoGoogle Scholar
Schembri Adam, Kearsy Cormier & Jordan Fenlon
2018 Indicating verbs as typologically unique constructions: Reconsidering verb ‘agreement’ in sign languages. Glossa: a Journal of General Linguistics 3(89). 1–40. DOI logoGoogle Scholar
Schembri, Adam, Gillian Wigglesworth, Trevor Johnston, Greg Leigh, Robert Adam & Roz Barker
2002 Issues in development of the Test Battery for Australian Sign Language Morphology and Syntax. Journal of Deaf Studies and Deaf Education 7(1). 18–40. DOI logoGoogle Scholar
Schick, Brenda
2002 The expression of grammatical relations in deaf toddlers learning ASL. In Gary Morgan & Bencie Woll (eds.), Directions in sign language acquisition, 143–158. Amsterdam: John Benjamins. DOI logoGoogle Scholar
2010 The development of American Sign Language and Manually Coded English systems. In Marc Marschark & Patricia Elisabeth Spencer (eds.), The Oxford handbook of deaf studies, language, and education (Volume 1, second edition), 229–240. New York, NY: Oxford University Press.Google Scholar
Schwager, Waldemar
2012 Morphologie: Bildung und Modifikation von Gebärden. In Hanna Eichmann, Martje Hansen & Jens Heßmann (eds.) Handbuch Deutsche Gebärdensprache: sprachwissenschaftliche und anwendungsbezogene Perspektiven, 61–110. Seedorf: Signum-VerlagGoogle Scholar
Sign Language Linguistic Society
2016 Ethics statement for sign language research. Retrieved from https://​slls​.eu​/slls​-ethics​-statement/. Access date 2019-09-18.
Simms, Laurene, Sharon Baker & Diane M. Clark
2013 The Standardized Visual Communication and Sign Language Checklist for signing children. Sign Language Studies 14(1). 101–124. DOI logoGoogle Scholar
Singleton, Jenny L. & Gary Morgan
2006 Natural signed language acquisition within the social context of the classroom. In Brenda Schick, Marc Marschark & Patricia Elisabeth Spencer (eds.), Advances in the sign language development of deaf children, 344–376. New York, NY: Oxford University Press.Google Scholar
Singleton, Jenny L. & Elissa L. Newport
2004 When learners surpass their models: The acquisition of American Sign Language from inconsistent input. Cognitive Psychology 491. 370–407. DOI logoGoogle Scholar
Singleton, Jenny L. & Samuel Supalla
2011 Assessing children’s proficiency in natural signed languages. In Marc Marschark & Patricia E. Spencer (eds.), Oxford handbook of deaf studies, language, and education (Vol. 1, 2nd edition), 289–302. Oxford: Oxford University Press. DOI logoGoogle Scholar
Valli, Clayton, Ceil Lucas & Kristin Mulroney
2005 Linguistics of American Sign Language: An introduction (Fourth edition). Washington, DC: Gallaudet University Press.Google Scholar
Valmaseda, Marian, Mar Pérez, Rosalind Herman, Nuria Ramírez & Ignacio Montero
. September 2013 Evaluación de la competencia gramatical en LSE: Proceso de adaptación del BSL Receptive Skill Test (test de habilidades receptivas). Paper presented at the Congreso CNLSE sobre la Investigación de la Lengua de Signos. Madrid, Spain.

video