GDK reported error. MT_mremap: mremap failed. !OS: Cannot allocate memory
Hello all,
I did a tpc-ds testing on MonetDB, and when it ran some queries, the error message was appeared:
GDK reported error.
MT_mremap: mremap(0x7ffdc0000000,2054553600,2465464320) failed
!OS: Cannot allocate memory
HEAPextend: failed to extend to 2465464320 for 44/4415.tail: GDKmremap() failed
It seems that the memory has been ran out. But I have no idea what is the cause of this error.
The version of Monetdb is 11.27.13, and OS is Ubunbtu-16.04 with memory is 8G, and with 20G testing data.(It is worth mentioning, a table named store_sales is 7.5G in the query.)
The query statement is:
with frequent_ss_items as ( select substr(i_item_desc,1,30) itemdesc ,i_item_sk item_sk ,d_date solddate ,count(*) cnt from store_sales ,date_dim ,item where ss_sold_date_sk = d_date_sk and ss_item_sk = i_item_sk and d_year in (1999,1999+1,1999+2,1999+3) group by itemdesc ,i_item_sk ,d_date having count(*) >4 ), max_store_sales as ( select max(csales) tpcds_cmax from ( select c_customer_sk ,sum(ss_quantity*ss_sales_price) csales from store_sales ,customer ,date_dim where ss_customer_sk = c_customer_sk and ss_sold_date_sk = d_date_sk and d_year in (1999,1999+1,1999+2,1999+3) group by c_customer_sk ) x ), best_ss_customer as ( select c_customer_sk ,sum(ss_quantity*ss_sales_price) ssales from store_sales ,customer where ss_customer_sk = c_customer_sk group by c_customer_sk having sum(ss_quantity*ss_sales_price) > (95/100.0) * ( select * from max_store_sales ) ) select sum(sales) from ( select cs_quantity*cs_list_price sales from catalog_sales ,date_dim where d_year = 1999
and d_moy = 1 and ws_sold_date_sk = d_date_sk and ws_item_sk in (select item_sk from frequent_ss_items) and ws_bill_customer_sk in (select c_customer_sk from best_ss_customer)) y limit 100;
Please give me some advice if you have any idea, either of the direct question or my expression. Thanks a lot!
Regards,
Rancho
Hi Rancho,
how much free disk space is there on the partition where your DBFARM (and DBEXTRA) is/are located, in particular at the very moment (or just before) the mremap fails?
Does the query also fail if you run the query right after a restart of the MonetDB server (mserver5)?
Best, Stefan
----- On Mar 15, 2018, at 7:29 AM, RanchoYuan yuanshijia@ww-it.cn wrote:
Hello all,
I did a tpc-ds testing on MonetDB, and when it ran some queries, the error message was appeared:
GDK reported error.
MT_mremap: mremap(0x7ffdc0000000,2054553600,2465464320) failed
!OS: Cannot allocate memory
HEAPextend: failed to extend to 2465464320 for 44/4415.tail: GDKmremap() failed
It seems that the memory has been ran out. But I have no idea what is the cause of this error.
The version of Monetdb is 11.27.13, and OS is Ubunbtu-16.04 with memory is 8G, and with 20G testing data.(It is worth mentioning, a table named store_sales is 7.5G in the query.)
The query statement is:
with frequent_ss_items as ( select substr(i_item_desc,1,30) itemdesc ,i_item_sk item_sk ,d_date solddate ,count(*) cnt from store_sales ,date_dim ,item where ss_sold_date_sk = d_date_sk and ss_item_sk = i_item_sk and d_year in (1999,1999+1,1999+2,1999+3) group by itemdesc ,i_item_sk ,d_date having count(*) >4 ), max_store_sales as ( select max(csales) tpcds_cmax from ( select c_customer_sk ,sum(ss_quantity*ss_sales_price) csales from store_sales ,customer ,date_dim where ss_customer_sk = c_customer_sk and ss_sold_date_sk = d_date_sk and d_year in (1999,1999+1,1999+2,1999+3) group by c_customer_sk ) x ), best_ss_customer as ( select c_customer_sk ,sum(ss_quantity*ss_sales_price) ssales from store_sales ,customer where ss_customer_sk = c_customer_sk group by c_customer_sk having sum(ss_quantity*ss_sales_price) > (95/100.0) * ( select * from max_store_sales ) ) select sum(sales) from ( select cs_quantity*cs_list_price sales from catalog_sales ,date_dim where d_year = 1999
and d_moy = 1 and ws_sold_date_sk = d_date_sk and ws_item_sk in (select item_sk from frequent_ss_items) and ws_bill_customer_sk in (select c_customer_sk from best_ss_customer)) y limit 100;
Please give me some advice if you have any idea, either of the direct question or my expression. Thanks a lot!
Regards,
Rancho
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
also: how big is your mserver5, both virtual and resident size, at the time (or just before) the remap fails?
----- On Mar 15, 2018, at 9:07 AM, Stefan Manegold Stefan.Manegold@cwi.nl wrote:
Hi Rancho,
how much free disk space is there on the partition where your DBFARM (and DBEXTRA) is/are located, in particular at the very moment (or just before) the mremap fails?
Does the query also fail if you run the query right after a restart of the MonetDB server (mserver5)?
Best, Stefan
----- On Mar 15, 2018, at 7:29 AM, RanchoYuan yuanshijia@ww-it.cn wrote:
Hello all,
I did a tpc-ds testing on MonetDB, and when it ran some queries, the error message was appeared:
GDK reported error.
MT_mremap: mremap(0x7ffdc0000000,2054553600,2465464320) failed
!OS: Cannot allocate memory
HEAPextend: failed to extend to 2465464320 for 44/4415.tail: GDKmremap() failed
It seems that the memory has been ran out. But I have no idea what is the cause of this error.
The version of Monetdb is 11.27.13, and OS is Ubunbtu-16.04 with memory is 8G, and with 20G testing data.(It is worth mentioning, a table named store_sales is 7.5G in the query.)
The query statement is:
with frequent_ss_items as ( select substr(i_item_desc,1,30) itemdesc ,i_item_sk item_sk ,d_date solddate ,count(*) cnt from store_sales ,date_dim ,item where ss_sold_date_sk = d_date_sk and ss_item_sk = i_item_sk and d_year in (1999,1999+1,1999+2,1999+3) group by itemdesc ,i_item_sk ,d_date having count(*) >4 ), max_store_sales as ( select max(csales) tpcds_cmax from ( select c_customer_sk ,sum(ss_quantity*ss_sales_price) csales from store_sales ,customer ,date_dim where ss_customer_sk = c_customer_sk and ss_sold_date_sk = d_date_sk and d_year in (1999,1999+1,1999+2,1999+3) group by c_customer_sk ) x ), best_ss_customer as ( select c_customer_sk ,sum(ss_quantity*ss_sales_price) ssales from store_sales ,customer where ss_customer_sk = c_customer_sk group by c_customer_sk having sum(ss_quantity*ss_sales_price) > (95/100.0) * ( select * from max_store_sales ) ) select sum(sales) from ( select cs_quantity*cs_list_price sales from catalog_sales ,date_dim where d_year = 1999
and d_moy = 1 and ws_sold_date_sk = d_date_sk and ws_item_sk in (select item_sk from frequent_ss_items) and ws_bill_customer_sk in (select c_customer_sk from best_ss_customer)) y limit 100;
Please give me some advice if you have any idea, either of the direct question or my expression. Thanks a lot!
Regards,
Rancho
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
-- | Stefan.Manegold@CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |
participants (2)
-
RanchoYuan -
Stefan Manegold