[Gmod-help] gmod bulk upload

[Stephen Ficklin]

Hi Dave,

Somehow I missed the performance tips document you link to below.  I
have tweaked some of the postgres parameters but I did not initialize
the database with --locale=C

I'm going to experiment with that and keep my fingers crossed.

Thanks,
Stephen

-----Original Message-----
From: Dave Clements, GMOD Help Desk [mailto:gmodhelp at googlemail.com] 
Sent: Thursday, May 08, 2008 1:18 PM
To: Stephen Ficklin; GMOD Schema List
Cc: Meg Staton
Subject: Re: [Gmod-help] gmod bulk upload

Dear Stephen,

I don't think this (lack of) performance is typical.  It suggests to
me something in the database is going awry.  It could be any number of
things (see http://gmod.org/PostgreSQL_Performance_Tips for some of
them).

It could also be the default encoding for the database.  If Postgres
is using a multibyte character encoding then that can slow things down
by a couple orders of magnitude.  See

http://sourceforge.net/mailarchive/forum.php?thread_name=200711082012.lA
8KCtV15976%40cricket.bio.indiana.edu&forum_name=gmod-schema

for a discussion of that.

Has anything improved or have you discovered anything new since you
sent the e-mail?

I am also cross-posting this to the GMOD Schema list as people there
may have suggestions.

Thanks,

Dave C
GMOD Help Desk

On Wed, May 7, 2008 at 7:10 PM, Stephen Ficklin
<FICKLIN at exchange.clemson.edu> wrote:
> Hello,
>
>
>
> We have an installation of chado that has about 7million records in
the
> feature table.  We're uploading our data as GFF files using the
> gmod_bulk_load_gff3.pl  and we find that it is taking a very long
time.  It
> has taken about 28 hours to upload 190,220 entries in two GFF files.
Is
> this normal?   It seems the more entries we add to the database the
slower
> these uploads become.  We still have over a million more records to
add to
> the database.  Is there any way we can speed up this upload?
>
>
>
> Thanks,
>
> Stephen Ficklin
>