Hi, You should plot the times of each of your runs to see the trend. This single run may be an outlier, which could come from anything in your system environment. Even a seemingly harmless concurrent program using significant memory could compete with MonetDB.
And indeed, at some point you will see disk IO.
regards, Martin
On 7/26/13 10:40 AM, integrity wrote:
Hi dear all,
I did a test of insert 2,000,000,000 rows into MonetDB with "COPY BINARY INTO FROM binary_file", in this test
- i generate 21 files, each file represents a table column and has
200000 rows. I created them with rand() number and use fwrite() binary write method.
the table creation sql command is : *create table tmatch(id bigint,a float,b float,c float, d float, e float,* *f float,g float,h float, i float,j float,* *k float,l float,m float, n float,o float,* *p float,q float,r float, s float,t float);* the table has 21 columns,each column has 8 bytes, so each column file is c1=200000*21*8 Byte= 268800000 Byte=3.2MB 2. I use "COPY BINARY INTO FROM above_binary" to load each binary file into tmatch. The test was run 10000 times repeatedly.
the average time of 10000 times is only 1.0635589558955727, but when at 9043th time, it cost 227m36.136 ,some times later the time value increase to a large number, is it because of flush data from cache into database after the cache is full? The problem is since we have to control the total process time within 15 seconds , I am wondering if you can help me reduce the maximum time to a lower point?
Thanks very much! Meng
users-list mailing list users-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list