On Fri, Feb 11, 2005 at 03:45:06PM -0500, Edmund Dengler wrote:
Hi Niels!
So, if I was doing multiple loads (which would be how I would be operating, see previous posts re structuring for multiple loads), then I would definitely need to load to dummy/tmp BATs, and then append my data to the real BATs (as I will be exceeding the 1 Billion row limit using a 30-bit OID)?
The << lng >> is being used since there is no 64-bit VOID.
Assuming you are using a 32 bit box your right. But on a 32bit box you will not run out of rows because of the 32 bit number, but because you run out of adresing space. Remember MonetDB is a main memory database, this means that a bat needs to fit into memory. So a single bat should be smaller 4GB (to be precise < 2GB because of signedness issues).
Your right that ascii io should append to your existing bats, but I don't see that as a problem, bat (bulk) inserts are fast.
Niels
Regards! Ed
On Fri, 11 Feb 2005, Niels Nes wrote:
On Fri, Feb 11, 2005 at 03:17:16PM -0500, Edmund Dengler wrote:
Greetings!
Looking at the ascii_io module, I can't seem to find a way for loading up multiple BATs where one column of the input is specified as the head value. Is there such an option, or is it always a VOID loadup?
In particular, I have a << lng >> column as the primary key. So, using my current understanding, I would need to load up a set of dummy/temp BATs, and then using these load the data to the real BATs.
Ascii_io was designed for loading wide-tables. Each column of the table is loaded into a bat[void,val]. The void heads link the columns together. So if one of the columns of this table is your primary key (lng) that is not a problem. Probably you could do without the primary (unless referenced in one of the other columns).
Niels
Regards! Ed
SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
--
Niels Nes, Centre for Mathematics and Computer Science (CWI) Kruislaan 413, 1098 SJ Amsterdam, The Netherlands room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312 url: http://www.cwi.nl/~niels e-mail: Niels.Nes@cwi.nl
SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users