2008年4月19日の月例研究会 (JACET 関東支部)

■■ 2008-2009 JACET Kanto Monthly Meeting ■■


Title：A Many-Faceted Rasch Analysis of a Graduation Thesis Assessment 発表題目：卒業論文評価におけるラッシュ・アナリシス

Performance testing has become common in the area of foreign language assessment, whereby language ability is tested through some kind of judged performance. This might involve oral interview tests in the case of speaking, or rated essays in the case of writing. The advantage of performance testing is that it elicits more authentic language behavior on the part of the testee. One disadvantage is that there is an element of subjectivity in rater assessments, as compared with more traditional testing methods such as multiple choice.
Many-faceted Rasch measurement (MFRM) is a statistical technique for taking subjective raw scores based on rater judgments and transforming them into an objective integral scale. It enables the measurement and comparison of test-taker ability, item difficulty, and rater severity. It takes into consideration variations in rater severity levels, and accordingly adjusts the measurement of test scores.
This presentation reports on MFRM of 25 English theses written by Japanese university students as a graduation requirement and rated by nine faculty members. The study attempted to determine the effect of individual rater differences on the thesis scores and the presence of rater bias in the assessment of the theses. Results show that while significant rater bias was a factor for only one faculty member, faculty members did in fact interpret the rating scale in different ways.