[Monetdb-developers] Could not create hash table for key test

Stefan de Konink stefan at konink.de
Sun Nov 23 11:46:20 CET 2008


On Sun, 23 Nov 2008, Martin Kersten wrote:

> The system is trying to malloc space for (in this case) a hash table,
> but that instruction fails. Then it attempts to free up memory by
> swapping out tables.
> Given the fact that after this sweep it still fails, indicates that
> your  swap memory has become fragmented, such that there is no
> consecutive area of 14 GB left.

Maybe a very strange idea, could be your better solution. If by default
these operations are mmap'ed, they should be taken care of by a disk based
memory extension. Hence being independent of RAM/SWAP but having all pro's
of both.

> For large table loads (relative to the swap size) it is advisable to
> load in multiple steps.
> First load the data into the tables without (foreign) key checks.
> After this step ALTER the tables to respect the (foreign) keys.

You are again right for many reasons to do it in this way. Sadly my 'data
provider' is not so strict on data integrety I found out again, and this
is probably the only efficient way to load the tables, and find all non
existing relations.



Stefan





More information about the developers-list mailing list