[Monetdb-developers] Another big group by query

Stefan Manegold Stefan.Manegold at cwi.nl
Mon Apr 13 08:58:20 CEST 2009


On Mon, Apr 13, 2009 at 02:30:44AM +0200, Stefan de Konink wrote:
> Martin Kersten wrote:
> > You can trace the execution even for a server started already using
> > the stethoscope.

What does this/you you want to tell/show us?
I.e., does your query still hang, or did it finish now?
("end s27_1;" seem to suggest so, right?)
Please try to be a bit more verbose in your statements/emails...

I see that aggr.count() (not unexpectedly) is the most expensive one of the
operations shown below --- but "only" ~100 sec --- not too extreme for
counting the size of 362105182 groups over 365652137 BUNs ...

The sum of all shown times is "only" 116865583 usec (i.e., just under 2 min).

Did you compile from CVS or from tarball?
Did you configure with either --disable-assert (CVS) or --enable-assert
(tarball)?
If you compiled from CVS without --disable-assert or from tarball with
--enable-assert, you also might want to start your mserver5 with
--performance (or at least --debug=33554432) to disable "cleaning" of
to-be-freed memory just before free() is called --- this might make the ":=
nil;" assignments (a bit?) faster ...

Stefan
 
> p
> [ 605303,      mt,     # name
> [ 51,    "_30 := bat.mirror(ext35=<tmp_4076>[362105182]);" ]
> [ 1282955,       "grp20 := nil;"]
> [ 13,   955,     "grp20 := nil;" ]
> [ 13,    "ext35 := nil;" ]
> [ 103806553,     "_31 := aggr.count(_17=<tmp_4101>[365652137],grp33=<tmp_3722>[365652137],_30=<tmp_4021>[362105182]);" ]
> [ 14,    "_17 := nil;" ]
> [ 1088736,       "grp33 := nil;" ]
> [ 2605761,       "_33 := algebra.thetauselect(_31=<tmp_3737>:bat[:oid,:wrd][362105182],_32=2:wrd,">");" ]
> [ 45,    "_36 := algebra.markT(_33=<tmp_4024>[6054],0 at 0);" ]
> [ 7,     "_37 := bat.reverse(_36=<tmp_3722>[6054]);" ]
> [ 8,     "_33 := nil;" ]
> [ 9,     "_36 := nil;" ]
> [ 17917,         "_38 := algebra.join(_37=<tmpr_3722>[6054],_31=<tmp_3737>:bat[:oid,:wrd][362105182]);" ]
> [ 1283903,       "_31 := nil;" ]
> [ 1314255,       "_39 := algebra.joinPath(_37=<tmpr_3722>[6054],_30=<tmp_4021>[362105182],_16=<tmp_4101>[365652137]);" ]
> [ 1314255,       "_40 := algebra.joinPath(_37=<tmpr_3722>[6054],_30=<tmp_4021>[362105182],_27=<tmp_3742>[365652137]);" ]
> [ 67,    "_37 := nil;" ]
> [ 967626,        "_27 := nil;" ]
> [ 1205150,       "_16 := nil;" ]
> [ 1360389,       "_30 := nil;" ]
> [ 28,    "_41 := sql.resultSet(3,1,_38=<tmp_3736>[6054]);" ]
> [ 10, "sql.rsColumn(_41=32,"sys.way_nds","L6","wrd",32,0,_38=<tmp_3736>[6054]);" ]
> [ 7,     "_38 := nil;" ]
> [ 6, "sql.rsColumn(_41=32,"sys.way_nds","way","int",32,0,_39=<tmp_4103>[6054]);" ]
> [ 2,     "_39 := nil;" ]
> [ 5, "sql.rsColumn(_41=32,"sys.way_nds","to_node","int",32,0,_40=<tmp_4100>[6054]);" ]
> [ 2,     "_40 := nil;" ]
> [ 2,     "_51 := io.stdout();" ]
> [ 12489,         "sql.exportResult(_51=27443088,_41=32);" ]
> [ 2,     "end s27_1;" ]
> 
> 
> 
> Stefan
> 
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by:
> High Quality Requirements in a Collaborative Environment.
> Download a free trial of Rational Requirements Composer Now!
> http://p.sf.net/sfu/www-ibm-com
> _______________________________________________
> Monetdb-developers mailing list
> Monetdb-developers at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/monetdb-developers
> 

-- 
| Dr. Stefan Manegold | mailto:Stefan.Manegold at cwi.nl |
| CWI,  P.O.Box 94079 | http://www.cwi.nl/~manegold/  |
| 1090 GB Amsterdam   | Tel.: +31 (20) 592-4212       |
| The Netherlands     | Fax : +31 (20) 592-4312       |




More information about the developers-list mailing list