Special SLS Colloquium
Wednesday, January 15th
12:30pm – 1:20pm
BIOMED T211
Development and Validation of an L2 Korean Pronunciation Diagnostic
Dr. Daniel R. Isbell
PhD, Second Language Studies, Michigan State University
Assistant Professor, California Polytechnic State University
Abstract:
Pronunciation teaching is arguably in the midst of a renaissance (Derwing & Munro, 2015; Lee, Jang, & Plonsky, 2014; Levis, 2019). Yet it is also the case that some research shows whole-class pronunciation instruction may fall short in some respects (Inceoglu, 2019; Isbell, Lee, & Park, 2019; Levis, et al., 2016), perhaps owing to mismatches between curricular targets and individual needs. Tailoring pronunciation instruction to individual learners is thus critical (Levis, 2018), but in practice difficult to realize, in no small part due to the challenge of assessing individuals’ specific strengths and weaknesses (Lado, 1961; Lappin-Fortin & Rye, 2014; Trofimovich et al., 2016).
One approach to assessment which addresses that challenge is Diagnostic Language Assessment (DLA, Alderson, 2005; Alderson, Brunfaut, & Harding, 2015). DLA, in contrast to proficiency tests, achieves relevance to individual learner needs by providing detailed information on specific strengths and weaknesses in a targeted language area. Furthermore, diagnostic instruments are designed in reference to a theory of language development. In turn, resulting diagnostic information can be used to support instructional decisions, such as selecting common learner difficulties to focus on in a classroom setting, planning tailored learning activities for individuals, or simply raising awareness and directing the attention of learners to troublesome forms. In this talk, I report on the development and validation of a diagnostic tool for L2 learners called the Korean Pronunciation Diagnostic (KPD). I begin by presenting the design of the diagnostic, which features multiple tasks targeting each Korean phoneme in both perception and production, connecting assessment targets to a well-supported principle of L2 speech learning (Flege, 1995; Sakai & Moorman, 2018). Then, I introduce the test’s validity argument (Chapelle, Enright, & Jamieson, 2010; Kane, 2013), used to justify score interpretation and use, and highlight supporting evidence at each step. Validity evidence came from field testing with 198 learners and interviews with 21 learners. Field testing participants completed the KPD, a pronunciation self-assessment, a spontaneous speaking task, and an oral proficiency test. Interviewees provided their initial reactions to diagnostic feedback and discussed potential applications; 14 interviewees returned for a second interview 3-4 months later to discuss their learning activity and retake the KPD. Specifically, I show how the test’s measurement qualities (reliability, item statistics), relationships between KPD scores and external measures (self-assessments, oral proficiency, and pronunciation errors in spontaneous speech), and learners’ interpretation and application of diagnostic information support the use of the KPD. Finally, I provide an evaluation of the KPD’s validity argument and discuss some broader considerations for diagnosing L2 pronunciation.