JAFROC
JAFROC analysis is intended for analysis of human observer free-response data or CAD data in two or more modalities.
The JAFROC figure-of-merit (A_J) is the probability that lesions, obvioulsy on abnormal images, are rated higher than false positive marks on normal images. It is similar to the area under the ROC curve (AUC) used in ROC studies, which is the probability that an abnormal image is rated higher than a normal image (ROC is not concerned with location; the ROC rating applies to the whole image). Lesions rated higher than false positives is good! So is abnormal images being rated higher than normal images! Therefore one expects both figures of merit to correlate.Users of JAFROC are sometimes surprised that a modality (for example, mod-A) on which the observer marks some of the lesions without marking any normal image does not yield a perfect figure of merit (A_J = 1) and conversely a modality (for example, mod-B) on which the observer marks some of the normal images and does not mark any of the lesions does not yield a zero figure of merit (A_J = 0). In fact it is observed that 0 < (A_J)mod-B < (A_J)mod-A < 1. The observer who marks only lesions is obviously better than the observer who marks only normal images. If the observer marked every lesion and did not mark any normal image, the figure of merit would be unity and if the observer marked every normal image and did not mark any lesion, the figure of merit would be zero.
References
D. P. Chakraborty, "Maximum Likelihood analysis of free-response receiver operating characteristic (FROC) data," Med. Phys. 16 (4), 561-568 (1989).
D. P. Chakraborty and L. H. L. Winter, "Free-Response Methodology: Alternate Analysis and a New Observer-Performance Experiment," Radiology 174, 873-881 (1990).
D. D. Dorfman, K. S. Berbaum and C. E. Metz, "ROC characteristic rating analysis: Generalization to the Population of Readers and Patients with the Jackknife method," Invest. Radiol. 27 (9), 723-731 (1992).
D. P. Chakraborty and K. S. Berbaum, "Observer studies involving detection and localization: Modeling, analysis and validation," Medical Physics 31 (8), 2313-2330 (2004).
D. P. Chakraborty, "Analysis of location specific observer performance data: validated extensions of the jackknife free-response (JAFROC) method," Acad Radiol. 13 (10), 1187-1193 (2006).
D. P. Chakraborty and H. J. Yoon, "Investigation of methods for analyzing location specific observer performance data," Proc. SPIE Medical Imaging 2008: Image Perception, Observer Performance, and Technology Assessment 6917 (2008).