FAQ's
Most recent update (2/23/2010): reverted to JAFROC_V3f as JAFROC_V3g had a bug. The changes involve the curve fitting algorithm and do not affect JAFROC analysis.
A document describing the sample size estimation procedure has been added.
The new software also fits the inferred ROC data obtained by using the rating of the highest rated mark on an image as the inferred ROC rating for that image. As before, it provides fitted AFROC and FROC plots. Use the plot_tempate.xls file to import the appropriate xx_yy_PLOT.txt file to see the plots (xx is based on user data file name and yy is the type of plot, eg., ROC, AFROC or FROC).
1. Recent changes in recommendations:
Unless there are no normal cases, one should be using JAFROC. If there are no normal cases then JAFROC1 should be used.
I encountered a data set where JAFROC1 gave an insignificant finding while JAFROC gave a highly significant finding, which was unexpected. In this data set the readers made much fewer FPs on abnormal image than on normal images (about a factor of 5 less). Until I understood the problem better, I decided to recommend using JAFROC, unless one has to use JAFROC1 because there are no normal images. The problem may be related to very different false positive rates on normal and abnormal cases. When there are no normal images, the unbalance issue is irrelevant. JAFROC Version 3 software checks for this unbalance and issues a warning.
2. A preliminary readme file is now available.
3: JAFROC Version 3 is undergoing beta testing and the software is being continually updated based on bugs reported by users and added features. You may wish to check the web site for the most recent version. The simplest way is to save the original e-mail with the subject line "JAFROC Software" that was sent to you automatically when you first requested the software. The link in the e-mail will work for future downloads. So you don't need to repeatedly sign in with your e-mail address. The added features have to do mostly with sample size estimation and curve plotting. The previously reported statistics (F, ddf, p-value) should not change appreciably (but see below) and your study conclusion, i.e., the modality difference is significant or not significant, should not change.
4: To run the software you must download IDL Virtual machine software first; the .sav file is not an SAS file, so if you have SAS installed you must release its "hold" on that extension.
5. I am getting an unusual number of questions with data files which have very few cases (e.g., 12). JAFROC is not magic. One needs about 50 (25 + 25) cases and about 4 readers for a pilot study, and more for the planned study (this is where the power estimation procedure built into JAFROC Version 3 comes in.
6. Some questions have to do with poor understanding of observer performance studies. One of the posted presentations gives an overview of ROC and FROC methods. Please read it and other application papers using FROC analysis before you start collecting data. Useful GOOGLE search terms are my name, FROC and JAFROC. Contact me if you need some of these papers. Read my 2004 Med Phys paper to understand what JAFROC does.
7. A user reported that the p-value for a data set has changed, very slightly, from previous versions. This has to do with a change in the ANOVA code. I limit the F-statistic and the denominator degrees of freedom (ddf) to 1000. With such large values of F and ddf, the p-value will be extremely small, therefore your study conclusion (difference is significant or not significant) should not change. However, if the limit was reached you may see small changes in reported F, ddf and p-values from previous versions.