MonetDB using all availaible disk space
Hello,
I'm performing some bulk inserts into some tables (COPY INTO). Meanwhile some JOINs between those tables are executed and also some DELETEs.
During this JOIN's, the disk begins to fill up and only stops after there is no more space available (60GB in my case). Same for RAM. After a (long) while the space gets freed up.
The COPY INTO's are done without any significant delay.
The files that use that space are some .tail and .head in the database bat folder.
If the queries are not concurrent there aren't any problems. The JOIN it's not using any PK or FK.
Have some one noticed this before? Should I open a bug report?
Ruben Silva * *
Hi
I assume you are running this load concurrently. Then all queries will fight for memory resources. Given the materialize intermediates all approach in MonetDB, this will lead to swapping them out to disk. This might overflow.
Indeed, all columns are .head and .tail columns.
On 10/30/13 8:01 PM, Ruben Silva wrote:
Hello,
I'm performing some bulk inserts into some tables (COPY INTO). Meanwhile some JOINs between those tables are executed and also some DELETEs.
During this JOIN's, the disk begins to fill up and only stops after there is no more space available (60GB in my case). Same for RAM. After a (long) while the space gets freed up.
The COPY INTO's are done without any significant delay.
The files that use that space are some .tail and .head in the database bat folder.
If the queries are not concurrent there aren't any problems. The JOIN it's not using any PK or FK.
Have some one noticed this before? Should I open a bug report?
Ruben Silva
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Hello
Be aware that running COPY INTO and queries leads to a sizeable resource claim, because the system creates a log of all entries added as well. See LOCKED in http://www.monetdb.org/Documentation/Manuals/SQLreference/CopyInto
See also http://www.monetdb.org/Documentation/Userguide/diskspace
regards, Martin
On 10/30/13 8:01 PM, Ruben Silva wrote:
Hello,
I'm performing some bulk inserts into some tables (COPY INTO). Meanwhile some JOINs between those tables are executed and also some DELETEs.
During this JOIN's, the disk begins to fill up and only stops after there is no more space available (60GB in my case). Same for RAM. After a (long) while the space gets freed up.
The COPY INTO's are done without any significant delay.
The files that use that space are some .tail and .head in the database bat folder.
If the queries are not concurrent there aren't any problems. The JOIN it's not using any PK or FK.
Have some one noticed this before? Should I open a bug report?
Ruben Silva
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
participants (2)
-
Martin Kersten -
Ruben Silva