Measurement of Interrater Agreement间的一致性.pdf

Statistics, Data Analysis, and Modeling Measurement of Interrater Agreement  A SAS/IML Macro Kappa Procedure for Handling Incomplete Data Honghu Liu, GIM/HSR, UCLA Department of Medicine, LA, CA Ron D. Hays, GIM/HSR, UCLA Department of Medicine, LA, CA ABSTRACT of the most popular indicator of interrater agreement for nominal and ordinal data. The Reliability refers to the extent to which the current kappa procedure in SAS PROC same score is obtained on multiple FREQ works only with complete data (i.e., administrations. Interrater reliability is each rater uses every possible choice on the evaluated by comparing scores assigned to response scale at least once). For incomplete the same targets by two or more raters. data, one will either be unable to get kappa Kappa is perhaps the most commonly used (if it is a non-square table) or one will get a index of interrater agreement. Kappa is wrong kappa (if it is a irregular square scaled to be 0 when the amount of table). Unfortunately, incomplete agreement is what would be expected to be contingency tables are common especially observed by chance and 1 when there is when the number of possible choices on the perfect agreement. Kappa can be obtained response scale is large and the sample size is from SAS PROC FREQ, but this procedure relative small. Therefore, a proce