The AUC metric measure how well you order a group of results. Ideally, you would order them from worst to best, giving the negative cases small scores and the positive cases high scores. If you did this perfectly, your AUC would be 1.0. On the other hand, if you gave all of the cases the same score, your AUC would be 0.5, and if you returned a random ordering, your expected score would be 0.5.

To calculate your exact score, you must consider the ROC curve of your results. This curve is formed by considering every possible threshold value, there you predict the cases scoring above the threshold to be positive instances, and all of the cases below the threshold to be negative instances. For each such threshold, you achieve some false positive rate and some true positive rate. The false positive rate (FPR) is the fraction of all negative instances that a threshold predicts to be positive. Similarly, the true positive rate (TPR) is the fraction of all positive instances correctly predicted to be positive.

Each such threshold produces a point (FPR,TPR), and these points can be plotted, producing a curve. The area under this curve can then be calculated using the trapezoid method to interpolate. This area gives the AUC metric.

Code and Java class files are provided which will compute the AUC for you. To use these, you should create output with each line formatted as "<correct answer> <your score>". <correct answer> is an integer 0 or 1, and <your score> is a double. For instance, if you had created a file with all of your predictions in this way, you could run "java AUC < myOutput" and get the AUC for your predictions. myOutput might look like