ResponsetoNIJ’sAMethodologyforEvaluatingGeographic.docVIP

An Evaluation of NIJ’s Evaluation Methodology for Geographic Profiling Software D. Kim RossmoResearch Professor and DirectorCenter for Geospatial Intelligence and InvestigationDepartment of Criminal JusticeTexas State University March 9, 2005 Executive Summary This is a response to the National Institute of Justice’s A Methodology for Evaluating Geographic Profiling Software: Final Report (Rich Shively, 2004). The report contains certain errors, the most critical of which involves suggested performance measurements. Output accuracy is the single most important criterion for evaluating geographic profiling software. The report discusses various performance measures; unfortunately only one of these (hit score percentage/search cost) accurately captures how police investigations actually use geographic profiling. This response addresses the various problems associated with the other measures. Geographic profiling evaluation methodologies must respect the limitations and assumptions underlying geographic profiling, and accurately measure the actual function of a geographic profile. Geographic profiling assumes: (1) the case involves a series of at least five crimes; (2) the offender has a single stable anchor point; (3) the offender is using an appropriate hunting method; and (4) the target backcloth is reasonably uniform. Additionally, for various theoretical and methodological reasons, not all crime locations in a given series can be used in the analysis. The most appropriate measure of geographic profiling performance is “hit score percentage/search cost.” It is the ratio of the area searched (following the geographic profiling prioritization) before the offender’s base is found, to the total hunting area; the smaller this ratio, the better the geoprofile’s focus. There are no intrinsic disadvantages to this measure. The other evaluation measures discussed in the NIJ report are all linked to the problematic “error distance.”