[Monetdb-developers] Re: [MonetDB-users] speed comparison off inserts and deletes, and db size question
On Aug 3, 2005, at 10:40 AM, Fabian wrote:
You should keep in mind that if you use a (V)OID on an 32-bits platform, that you have a limitation of the number of rows that you can contain in a BAT. Also, and OID cannot be bigger than the int it is, so if you do need more rows, or sequence numbers that are higher, you will have to jump over to lng and do the sequence numbers yourself or something.
Is it 2^32-1 for the maximum number of rows in a BAT?
You can never join BATs that are bigger than max(OID), so you have to play tricks like doing the selects on all it's subparts and then unioning the results, given they are small enough to be stored in one BAT.
Are you referring to BAT * BATkunion (BAT *b, BAT *c) in the kernel guide?
I assume that I need to use BATkunion as opposed to BATsunion since I'm looking only at the head column. And that the fragments satisfy the set property since they are fragments of a larger BAT with a non- repeating oid range. Am I thinking right?
Do I need to set any properties on the BAT for the union to work?
Thanks, Joel
I don't have enough in-depth knowledge to answer these questions. I ope someone else has the time to give you the right answers.
Joel Reymont wrote:
On Aug 3, 2005, at 10:40 AM, Fabian wrote:
You should keep in mind that if you use a (V)OID on an 32-bits platform, that you have a limitation of the number of rows that you can contain in a BAT. Also, and OID cannot be bigger than the int it is, so if you do need more rows, or sequence numbers that are higher, you will have to jump over to lng and do the sequence numbers yourself or something.
Is it 2^32-1 for the maximum number of rows in a BAT?
You can never join BATs that are bigger than max(OID), so you have to play tricks like doing the selects on all it's subparts and then unioning the results, given they are small enough to be stored in one BAT.
Are you referring to BAT * BATkunion (BAT *b, BAT *c) in the kernel guide?
I assume that I need to use BATkunion as opposed to BATsunion since I'm looking only at the head column. And that the fragments satisfy the set property since they are fragments of a larger BAT with a non- repeating oid range. Am I thinking right?
Do I need to set any properties on the BAT for the union to work?
Thanks, Joel
On Wed, Aug 03, 2005 at 12:42:33PM +0200, Joel Reymont wrote:
On Aug 3, 2005, at 10:40 AM, Fabian wrote:
You should keep in mind that if you use a (V)OID on an 32-bits platform, that you have a limitation of the number of rows that you can contain in a BAT. Also, and OID cannot be bigger than the int it is, so if you do need more rows, or sequence numbers that are higher, you will have to jump over to lng and do the sequence numbers yourself or something.
Is it 2^32-1 for the maximum number of rows in a BAT?
its even 2^31-1 as internally monet only knows signed values. As MonetDB is a main memory db it requires all (needed) bats to be in memory (we basicaly assume there is enough virtual memory). But on a 32 bit system this will give problems, ie you'll run out of address space. A 32 bits system can only address 4GB. This means your database workset (those bats currently in use) cannot exceed this maximum.
You can never join BATs that are bigger than max(OID), so you have to play tricks like doing the selects on all it's subparts and then unioning the results, given they are small enough to be stored in one BAT.
Are you referring to BAT * BATkunion (BAT *b, BAT *c) in the kernel guide?
indeed BAT(k/s)union are the gdk functions which implement the union operator on bats.
I assume that I need to use BATkunion as opposed to BATsunion since I'm looking only at the head column. And that the fragments satisfy the set property since they are fragments of a larger BAT with a non- repeating oid range. Am I thinking right?
Kunion indeed looks only at the head for its double elemination (ie for the set properties). Having two void headed bats this could be done relatively cheap as the non disjoint ranges could be easily found.
Do I need to set any properties on the BAT for the union to work?
No.
Niels
Thanks, Joel-- http://wagerlabs.com/uptick
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click _______________________________________________ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
participants (3)
-
Fabian -
Joel Reymont -
Niels Nes