[Ngasp-help] Re: nGASP manuscript

[Tristan Fiedler]

Thank you for the comments.  We are reviewing them now.

Cheers,

Tristan

> Hi Tristan,
>
> the worm genomics meeting in Cambridge kept me busy most of the day. I
> finally came to read the manuscript.
> I think the paper is already pretty good. I have a few comments (some
> of which came from the mGene team), which you may consider before
> submitting the manuscript:
>
> * I missed a more detailed discussion of the annotation dataset that
> has been used for evaluation. There are a few issues:
> 	* For unconfirmed genes, how where they predicted? Did any of the
> compared methods produce these predictions? This would lead to biases
> in the evaluation, which should not be hidden.
> 	* Also, for the EST-confirmed genes: these gene models where
> generated using EST alignments, some manual curation and probably also
> gene finder predictions. Except the manual curation, the input is the
> same as for cat-3 gene finding. Who knows what the correct gene models
> are...
>     (or something in this direction, you probably have your own
> thoughts on this).
>
> * The evaluation for the combiners was done on only the 3' end of the
> regions. We noticed there is a significant difference for in the
> performance between the 5' and 3' ends (several percent in transcript
> level). Hence, the performance on the 3' end and the performance on
> the whole region are not directly comparable. I don't necessarily want
> you to redo the evaluation, but at least this should be noted in the
> main text. Also, it is important to state in the main text that
> combiners had about 50% more data for training.
>
> * Since the result of the paper is that combiners are the method of
> choice, it would be very interesting to understand how important the
> accuracy and the choice of the base-gene finders are. It would be
> really great if you could show some results, indicating which
> (sub-)set of (the three) gene finders lead to which performance.
>
> * I really would love to have the evaluation separately for each
> category. This would make the comparison between the methods
> considerably easier. It would be very nice to have, at least in the
> supplement.
>
> * One feature that came to our mind that you did not seem to have
> checked whether it leads to wrong predictions, is whether there is a
> gene on the opposite strand. Many gene finders only predict genes on
> one strand. mGene, for instance, predicts the genes independently on
> both strands.
>
> * It would be a great service to the gene finding community and also
> would facilitate reproducibility of this research, if you'd provide
> the evaluation scripts which lead to exactly the numbers given in the
> table in the supplement of this paper. (In any case I'd like to get
> them to evaluate new predictions that we have in the same way.)
>
> Finally, I'd like to ask you to reconsider the choice of the Journal.
> I think the paper would have a good chance in PLoS Comp Bio or Genome
> Biology, which I find considerably better suited for this work than
> BMC Bioinformatics.
>
> (Please let me know if anything is unclear or you need any additional
> information).
>
> Thanks a lot for your efforts in writing this manuscript and in
> organising the nGASP competition!
>
> All the best,
>
> Gunnar
>
> On 16.07.2008, at 19:26, Dr. Tristan J. Fiedler wrote:
>
>> Dear nGASP Participants,
>>
>> We thank you again for your participation in nGASP.
>>
>> The nGASP analysis team has now written up the results of nGASP
>> as a paper, which we plan to submit to BMC Bioinformatics.
>>
>> As agreed, we are sending you a copy of the draft manuscript
>> for your perusal before submission.
>>
>> We would be very grateful if you can let us know if you have any
>> major comments on the draft manuscript by Thursday 24th July.
>>
>> Comments may be sent to ngasp-help at wormbase.org
>>
>> Yours sincerely,
>>
>> The nGASP analysis team.
>>
>> <ngasp_16jul08b_alc.doc>
>
> +-------------------------------------------------------------------+
> Gunnar Rätsch                         http://www.fml.mpg.de/raetsch
> Friedrich Miescher Laboratory       Gunnar.Raetsch at tuebingen.mpg.de
> Max Planck Society                          Tel: (+49) 7071 601 820
> Spemannstraße 39, 72076 Tübingen, Germany   Fax: (+49) 7071 601 801
>
>
>
>
>