语言测试与理论第二章.pptVIP

We should choose at random a sample of all the students taking the test. These students would then be subjected to the full 45 minute oral component necessary for coverage of all the functions, using perhaps four scorers to ensure reliable scoring. This would be the criterion test against which the shorter test would be judged. The students’ scores on the full test would be compared with the ones they obtained on the ten-minute session, which would have been conducted and scored in the usual way, without knowledge of their performance on the longer version. If the comparison between the two sets of scores reveals a high level of agreement, then the shorter version of oral component may be considered valid, inasmuch as it gives results similar to those obtained with the longer version. If, on the other hand, the two sets of scores show little agreement, the shorter version cannot be considered valid; it cannot be used as a dependable measure of achievement with respect to the functions specified in the objectives. Of course, if ten minutes really is all that can be spared for each student, then the oral component may be included for the contribution that it makes to the assessment of students’ overall achievement and for its backwash effect. But it cannot be regarded as an accurate measure in itself. ‘a high level of agreement’ ‘little agreement’ How is the level of agreement measured? Standard procedures for comparing sets of scores : ‘validity coefficient’ a mathematical measure of similarity Perfect agreement between two sets of scores will result in a validity coefficient of 1. Total lack of agreement will give a coefficient of zero. It is best to square that coefficient. a coefficient of 0.7 between the two oral tests Squared 0.49 converted to a percentage, 49 per cent On the basis of this, we can say that the scores on the short test predict 49 per cent of the variation in scores on the longer test. In broad terms, there is almost 50 per cent agreement between