-----Original Message----- From: Martin Kersten Martin.Kersten@cwi.nl Sent: Friday, February 22, 2008 17:07 To: Communication channel for MonetDB users monetdb-users@lists.sourceforge.net Subject: Re: [MonetDB-users] COPY TO corrupting data
mobigital1 wrote:
FYI, i am using 500K rows at a time.
I've seen the mserver5.exe process memory utilization go up to 13GB (free memory was around 1.5 GB). but then it winds down pretty quickly.
Thanks. This means that the expected database chunk size is topped at 500K * (7 * 8+ 2*20+6 *10+15+144*8 ) bytes i.e. 500K* 1323= 0.670Gb
chucksize wise there does not seem a problem. However, virtual memory footprint quickly increases and I am wondering if we hit a limit in OS regaring memory mapped files.
I would check a single batch for consistency. Then built a merge table over the smaller batches. In a later stage you can then glue together the pieces until we hit another limit
thanks for the info and help in improving MonetDB
------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users