以多面向Rasch测量模式分析TOCFL口语测验评分者-华语文能力测验.PDFVIP

以多面向Rasch測量模式分析 TOCFL口語測驗 評分者訓練效果 藍珮君 國家華語測驗推動工作委員會 /研究員 摘要 本研究目的為探討在接受持續且長期的評分訓練後，評分者評分一致性的變 化情形，包含與其他評分者之間的一致性，以及評分者自身的一致性。研究方法 為收集 2009 年2月至 12月五次華語文口語能力測驗簡稱( TOCFL口語測驗 )基礎 級預試評分結果，採用多面向Rasch測量模式 (many-facet Rasch measurement)分析 軟體 FACETS ，針對評分者嚴格度(rater severity) 、評分結果一致性 (consistency) 以 及評分者面向之適配度(fit)進行分析。 研究結果顯示： 1. 經過多次且密集的評分訓練後，評分者之間嚴格度 差異逐 漸縮小，但整體評分嚴格度達到顯著差異水準，顯示評分者之間的嚴格度仍有不 同 ；2.評分者本身的給分相當一致，適配度數值均在可接受之範圍內；3.部分評 分者間隔二個月後的評分嚴格度有所變化 。 現今以華語為第二語言的口語測驗研究相當缺乏，尤其是採用多面向 Rasch 模式進行分析之研究。本研究結果提供華語文口語測驗的實徵資料，對於華語文 口語能力測驗的評分訓練效果有初 步的瞭解 ，也能做為未來進行華語文口語能力 測驗評分訓練及測驗實施的參考。 關鍵字：多面向 Rasch測量模式、口語測驗、評分者訓練 1 Using many-facet Rasch measurement to examining rater training effects of TOCFL Speaking Pei-Jiun Lan Steering Committee for the Test Of Proficiency-Huayu / Researcher Abstract This research aims to investigate how rater consistency varies after a continuing, long-term rater training by observing the inter-rater consistency and the intra-rater consistency. We collect the rating results from the five pilot tests of the TOCFL Speaking (Test of Chinese as a Foreign Language- Speaking, holding from February, 2009 to December) at beginner level. This research applies the many-facet Rash measurement by adopting the FACETS software to analyze the rater severity, rater consistency, and the fit statistics of the raters. Three major findings discussed in this research are that: 1) after frequent rater training sections, the variation of inter-rater severity has reduced, but the overall rating severity has reached the significant differences, which indicates that the discrepancy of severity still exists among