<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><br><div>Begin forwarded message:</div><br class="Apple-interchange-newline"><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="4" color="#000000" style="font: 14.0px Helvetica; color: #000000"><b>From: </b></font><font face="Helvetica" size="4" style="font: 14.0px Helvetica">Steven Salzberg <<a href="mailto:salzberg@umd.edu">salzberg@umd.edu</a>></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="4" color="#000000" style="font: 14.0px Helvetica; color: #000000"><b>Date: </b></font><font face="Helvetica" size="4" style="font: 14.0px Helvetica">July 16, 2008 3:54:04 PM EDT</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="4" color="#000000" style="font: 14.0px Helvetica; color: #000000"><b>To: </b></font><font face="Helvetica" size="4" style="font: 14.0px Helvetica">"Dr. Tristan J. Fiedler" <<a href="mailto:fiedler@fit.edu">fiedler@fit.edu</a>>, <a href="mailto:alc@sanger.ac.uk">alc@sanger.ac.uk</a>, <a href="mailto:lstein@cshl.edu">lstein@cshl.edu</a>, Steven Salzberg <<a href="mailto:salzberg@umiacs.umd.edu">salzberg@umiacs.umd.edu</a>></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="4" color="#000000" style="font: 14.0px Helvetica; color: #000000"><b>Subject: </b></font><font face="Helvetica" size="4" style="font: 14.0px Helvetica"><b>Re: nGASP manuscript</b></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div> </div><div>hi Tristan, Avril, and Lincoln,<br>Thanks for sending the manuscript. I've not had time to read it closely yet, but I noticed one problem that<br>I want to point out. This is an error that many of us (including me) in the gene-finding community have<br>made before, but now that I know about it I want to avoid it in the future.<br> The problem is our use of the term "specificity." The way you (we) have used it in the manuscript follows<br>the usage in the EGASP competition, which also got it wrong. Our definition in EGASP was the percentage<br>of a gene finders' predictions that were correct; i.e.:<br> (# correct predictions)/(total # predictions)<br>However, a good friend and colleague of mine (a biostatistician) pointed out that this measure should instead<br>be called "precision." You can find standard definitions of sensitivity and specificity in any text, and also in Wikipedia:<br> <a href="http://en.wikipedia.org/wiki/Sensitivity_and_specificity">http://en.wikipedia.org/wiki/Sensitivity_and_specificity</a><br>The proper definition of "specificity" is the ratio of true negatives to all "negative" predictions. This isn't<br>really meaningful in our context, because we don't attempt to predict non-gene regions. (Another way to<br>look at this is that we aren't taking a putative gene and saying yes/no.) In fact, we don't even have a good<br>way to say with certainty that a region isn't a gene, so we just look at positive predictions.<br><br>The other term for what we're measuring is "positive predictive value" (PPV):<br> <a href="http://en.wikipedia.org/wiki/Positive_predictive_value">http://en.wikipedia.org/wiki/Positive_predictive_value</a><br>although I like "precision" better. I think you'll agree that this is what the EGASP competition was<br>calling "specificity" - and it's been used this way in previous papers too. But this definition is<br>quite confusing to statisticians, and I think we should revert to the standard usage.<br><br>A simple global replace of "specificity" with "precision" will probably fix the manuscript,<br>though it would be best to check carefully. I hope you'll agree.<br><br>Steven<br><br>Steven L. Salzberg, Ph.D.<br>Horvitz Professor of Computer Science<br>Director, Center for Bioinformatics and Computational Biology<br>3125 Biomolecular Sciences Building<br>University of Maryland, College Park, MD 20742<br>Phone: 301-405-9611<br>Email: <a href="mailto:salzberg@umd.edu">salzberg@umd.edu</a><br>Blog: <a href="http://genefinding.blogspot.com">http://genefinding.blogspot.com</a><br><br><br></div></div><br></body></html>