computer-aided diagnosis of renal obstruction utility of log-linear modeling versus standard roc and kappa analysis对数线性模型的计算机辅助诊断肾梗阻效用与标准roc和kappa分析.pdfVIP

Manatunga et al. EJNMMI Research 2011, 1:5 /content/1/1/5 ORIGINAL RESEARCH Open Access Computer-aided diagnosis of renal obstruction: utility of log-linear modeling versus standard ROC and kappa analysis 1 1 2* Amita K Manatunga , José Nilo G Binongo and Andrew T Taylor Abstract Background: The accuracy of computer-aided diagnosis (CAD) software is best evaluated by comparison to a gold standard which represents the true status of disease. In many settings, however, knowledge of the true status of disease is not possible and accuracy is evaluated against the interpretations of an expert panel. Common statistical approaches to evaluate accuracy include receiver operating characteristic (ROC) and kappa analysis but both of these methods have significant limitations and cannot answer the question of equivalence: Is the CAD performance equivalent to that of an expert? The goal of this study is to show the strength of log-linear analysis over standard ROC and kappa statistics in evaluating the accuracy of computer-aided diagnosis of renal obstruction compared to the diagnosis provided by expert readers. Methods: Log-linear modeling was utilized to analyze a previously published database that used ROC and kappa statistics to compare diuresis renography scan interpretations (non-obstructed, equivocal, or obstructed) generated by a renal expert system (RENEX) in 185 kidneys (95 patients) with the independent and consensus scan interpretations of three experts who were blinded to clinical information and prospectively and independently graded each kidney as obstructed, equivocal, or non-obstructed. Results: Log-linear modeling showed that RENEX and the expert consensus had beyond-chance agreement in both non-obstructed and obstru