[MonetDB-users] Slow Data Load

Jos van Dongen jvdongen at tholis.com
Sun Dec 2 15:00:01 CET 2007


Dear Paulo,

The fastest way to get large amounts of data into MonetDB is through a flat
file and the use of the 'copy' command. This is even much faster than e.g.
Oracle Bulk Loader. Note however that your 32bit environment will cause
problems with large files (that's what I experienced). It's better to switch
to 64 bit. 

Met vriendelijke groet/kind regards,

Jos van Dongen
Tholis Consulting
T: 06-51169606
F: 0842-231136
M: jvdongen at tholis.com
www.tholis.com 

-----Original Message-----
From: monetdb-users-bounces at lists.sourceforge.net
[mailto:monetdb-users-bounces at lists.sourceforge.net] On Behalf Of Paulo
Jorge Dias
Sent: zaterdag 1 december 2007 20:26
To: monetdb-users at lists.sourceforge.net
Subject: [MonetDB-users] Slow Data Load


Hi,

I am working on a Windows XP SP2 32bit with MonetDB v5.2.2 kernel v1.20.0

I have built a java program to upload data from an Oracle Database Table
with 6milion rows.

After about 4milion of rows loaded the performance decreases dramatically.

I have tried to use other way, which is generating all insert statements to
a file and then using mclient to execute all the insert statements.

Slower that the java option because the autocommit=on and parsing of each
statement, but the result is the same.

After about 4milion of rows loaded the performance decreases dramatically.

Can someone give an explanation for this behavior?

Thanks,

Paulo Jorge Dias

-- 
View this message in context:
http://www.nabble.com/Slow-Data-Load-tf4929292.html#a14108678
Sent from the monetdb-users mailing list archive at Nabble.com.


-------------------------------------------------------------------------
SF.Net email is sponsored by: The Future of Linux Business White Paper
from Novell.  From the desktop to the data center, Linux is going
mainstream.  Let it simplify your IT future.
http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4
_______________________________________________
MonetDB-users mailing list
MonetDB-users at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/monetdb-users

No virus found in this incoming message.
Checked by AVG Free Edition. 
Version: 7.5.503 / Virus Database: 269.16.12/1162 - Release Date: 30-11-2007
21:26
 

No virus found in this outgoing message.
Checked by AVG Free Edition. 
Version: 7.5.503 / Virus Database: 269.16.12/1163 - Release Date: 1-12-2007
12:05
 





More information about the users-list mailing list