[Gmod-help] fast loader for Postgresql

[Margie Manker]

Hi Scott/GMOD Help Desk

 

We are using 'bp_seqfeature_load.pl' to load a Postgresql
DB::SeqFeature::Store database. Upon load of a SNP.gff file, we found the
task took about 1.5 weeks to complete. Thus we have researched whether there
exists a fast loader for Postgresql; we are aware that there are fast and
bulk loaders for MySQL.

 

Our search (GMOD site, mailing lists, general Google search, etc.) yielded
no information on an existing fast/bulk loader for Postgresql, which led us
to send this email.

 

It seems we have two options available to us to increase the speed of load:
1) move to a MySQL platform for our GBrowse database; 2) split large files
to be loaded into smaller bits and load as such.

 

Option 1 is not one we prefer. We have other Postgresql databases that
interact with the GBrowse database, which we do not want to move to the
MySQL platform. Also, we would prefer not to have two different database
platforms (i.e. MySQL and Postgresql) serving the web site that features
GBrowse because we have a query feature on the web site that interacts with
GBrowse, both of which we would like to be served by a Postgresql database.

 

Our question: has anyone written a fast or bulk loader for Posgresql? If so,
could you point us in that direction? If not, do you have any other
suggestions we may not have presented above?

 

Please advise. Thank you for your time.

 

Margie Manker

 

Project Analyst

The Centre for Applied Genomics

The Hospital for Sick Children

manker at populargenetics.ca 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://brie4.cshl.edu/pipermail/gmod-help/attachments/20110126/e1793d02/attachment.html>