[MonetDB-users] Counting open BATs
Stefan.Manegold at cwi.nl
Thu May 5 13:16:40 CEST 2005
sorry for the delayed reply ...
> The pattern of usage:
> Read a row of data. Determine the "partition" to stick the data in
> (generates a number). In one BAT, track all such partitions. If this
> partition does not exist, then create the BAT, rename it, set the base,
> etc. Store the name in the tracking BAT. Insert the data into the data
> partition BAT. Every 10,000 rows, do a commit.
> From your description, after each row, I do not have a variable that is
> keeping the BAT open (I always use "bat("name").XXX", or keep a ref
> variable in a block. So, as far as I can tell, I should not have 7500 BATs
> open at once.
> Am I misunderstanding?
No, as far as I can see from your discription, everything looks fine.
Unfortunately, I cannot yet see, where your memory problems come from.
Would it be possbile to provide us with your MIL code --- or at least the
Maybe, you could even send us a kind of trace or log of actions that lead to
> Or is there some underlying component/module keeping the BATs open?
No, none that I'm aware of.
> Or is this just the amount of memory MonetDB will use?
Well, could be, but to be sure, I'd need more details of you usage of
MonetDB (see above).
> I do not have a BAT of BATs (as I was sure that this would cause
> problems, and from your description, will indeed).
Good. BATs of BATs are indeed considered "problematic" (if not "evil");
better use BATs of bat-names (I guess, that's what you do with your one BAT
that tracks all the partitions?).
| Dr. Stefan Manegold | mailto:Stefan.Manegold at cwi.nl |
| CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ |
| 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 |
| The Netherlands | Fax : +31 (20) 592-4312 |
More information about the users-list