developers-list February 2016

developers-list@monetdb.org

2 participants
3 discussions

by Luciano Sasso

Hi, I have two tables , deleted records of the two tables , then tried to delete again in Table 1 , an error occurred ( could not get print) , and in Table 2 records erased, again show in Table 2. Everything was running on a SQL console ( NetBeans ) and auto commit on. -- Luciano Sasso Vieira Data Scientist & Solutions Architect luciano(a)gsgroup.com.br <http://www.gsgroup.com.br> | tel: 17 3353-0833 | cel: 17 99706-9335 www.gsgroup.com.br <http://www.gsgroup.com.br> --- Este email foi escaneado pelo Avast antivírus. https://www.avast.com/antivirus

6 years, 5 months

Slow query

by Luciano Sasso

Hi, I have MonetDB JUL2015-SP2, with table of 100M rows and 20 coluns, 4 of number type, others string type. A simple "select count(*) from table", run on 90 minutes. Hardware and SO is: Debian 8, 4 cores, 11G RAM and 20G free on disc. While monet run the query, command top, show 42G of virtual memory. Its normal ?, any idea ? thanks. -- Luciano Sasso Vieira Data Scientist & Solutions Architect luciano(a)gsgroup.com.br <http://www.gsgroup.com.br> | tel: 17 3353-0833 | cel: 17 99706-9335 www.gsgroup.com.br <http://www.gsgroup.com.br> --- Este email foi escaneado pelo Avast antivírus. https://www.avast.com/antivirus

6 years, 5 months

Re: MonetDB: default - use same optimizer pipe for inlined queries.

by Stefan Manegold

cwi@achtbaan:~/manegold/source$ hg clone http://dev.monetdb.org/hg/MonetDB/ Het programma ‘hg’ is momenteel niet geïnstalleerd. Vraag uw beheerder om het pakket 'mercurial' te installeren om 'hg' uit te kunnen voeren nou, dan maar wget de tarball - werkt maar: cwi@achtbaan:~/manegold/build$ ../source/MonetDB-11.21.13/configure --enable-optimize --prefix=/home/cwi/manegold/prefix 2>&1 | tee config.out checking build system type... powerpc64le-unknown-linux-gnu checking host system type... powerpc64le-unknown-linux-gnu checking target system type... powerpc64le-unknown-linux-gnu checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /bin/mkdir -p checking for gawk... no checking for mawk... mawk checking whether make sets $(MAKE)... yes checking whether make supports nested variables... yes checking whether UID '1001' is supported by ustar format... yes checking whether GID '1001' is supported by ustar format... yes checking how to create a ustar tar archive... gnutar checking for pkg-config... no configure: error: pkg-config is required to configure MonetDB ... jammer ... Stefan ----- On Feb 3, 2016, at 10:45 AM, Niels Nes commits+niels=cwi.nl(a)monetdb.org wrote: > Changeset: 039a2b238d13 for MonetDB > URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=039a2b238d13 > Modified Files: > sql/backends/monet5/sql_optimizer.c > sql/backends/monet5/sql_optimizer.h > sql/backends/monet5/sql_scenario.c > Branch: default > Log Message: > > use same optimizer pipe for inlined queries. > > > diffs (107 lines): > > diff --git a/sql/backends/monet5/sql_optimizer.c > b/sql/backends/monet5/sql_optimizer.c > --- a/sql/backends/monet5/sql_optimizer.c > +++ b/sql/backends/monet5/sql_optimizer.c > @@ -215,8 +215,8 @@ addOptimizers(Client c, MalBlkPtr mb, ch > addtoMalBlkHistory(mb, "getStatistics"); > } > > -void > -addQueryToCache(Client c) > +str > +optimizeQuery(Client c) > { > MalBlkPtr mb; > backend *be; > @@ -226,13 +226,12 @@ addQueryToCache(Client c) > assert(be && be->mvc); /* SQL clients should always have their state set */ > pipe = getSQLoptimizer(be->mvc); > > - insertSymbol(c->nspace, c->curprg); > trimMalBlk(c->curprg->def); > c->blkmode = 0; > mb = c->curprg->def; > chkProgram(c->fdout, c->nspace, mb); > #ifdef _SQL_OPTIMIZER_DEBUG > - mnstr_printf(GDKout, "ADD QUERY TO CACHE\n"); > + mnstr_printf(GDKout, "Optimize query\n"); > printFunction(GDKout, mb, 0, LIST_MAL_ALL); > #endif > /* > @@ -250,23 +249,34 @@ addQueryToCache(Client c) > if (msg != MAL_SUCCEED) > GDKfree(msg); /* ignore error */ > } > - return; > + return NULL; > } > addOptimizers(c, mb, pipe); > msg = optimizeMALBlock(c, mb); > - if (msg != MAL_SUCCEED) { > - showScriptException(c->fdout, mb, 0, MAL, "%s", msg); > - GDKfree(msg); > - return; > - } > + if (msg) > + return msg; > > /* time to execute the optimizers */ > if (c->debug) > optimizerCheck(c, mb, "sql.baseline", -1, 0); > #ifdef _SQL_OPTIMIZER_DEBUG > - mnstr_printf(GDKout, "ADD optimized QUERY TO CACHE\n"); > + mnstr_printf(GDKout, "End Optimize Query\n"); > printFunction(GDKout, mb, 0, LIST_MAL_ALL); > #endif > + return NULL; > +} > + > +void > +addQueryToCache(Client c) > +{ > + str msg = NULL; > + > + insertSymbol(c->nspace, c->curprg); > + msg = optimizeQuery(c); > + if (msg != MAL_SUCCEED) { > + showScriptException(c->fdout, c->curprg->def, 0, MAL, "%s", msg); > + GDKfree(msg); > + } > } > > /* > diff --git a/sql/backends/monet5/sql_optimizer.h > b/sql/backends/monet5/sql_optimizer.h > --- a/sql/backends/monet5/sql_optimizer.h > +++ b/sql/backends/monet5/sql_optimizer.h > @@ -12,6 +12,7 @@ > > //#define _SQL_OPTIMIZER_DEBUG > > +sql5_export str optimizeQuery(Client c); > sql5_export void addQueryToCache(Client c); > sql5_export str SQLoptimizer(Client c); > sql5_export void SQLsetAccessMode(Client c); > diff --git a/sql/backends/monet5/sql_scenario.c > b/sql/backends/monet5/sql_scenario.c > --- a/sql/backends/monet5/sql_scenario.c > +++ b/sql/backends/monet5/sql_scenario.c > @@ -1263,17 +1263,20 @@ recompilequery: > > chkTypes(c->fdout, c->nspace, c->curprg->def, TRUE); /* resolve types */ > if (opt) { > + str msg = optimizeQuery(c); > + /* > MalBlkPtr mb = c->curprg->def; > > trimMalBlk(mb); > chkProgram(c->fdout, c->nspace, mb); > addOptimizers(c, mb, "default_pipe"); // TODO change to active pipe! > msg = optimizeMALBlock(c, mb); > + */ > if (msg != MAL_SUCCEED) { > sqlcleanup(m, err); > goto finalize; > } > - c->curprg->def = mb; > + //c->curprg->def = mb; > } > //printFunction(c->fdout, c->curprg->def, 0, LIST_MAL_ALL); > /* we know more in this case than chkProgram(c->fdout, c->nspace, > c->curprg->def); */ > _______________________________________________ > checkin-list mailing list > checkin-list(a)monetdb.org > https://www.monetdb.org/mailman/listinfo/checkin-list -- | Stefan.Manegold(a)CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |

6 years, 5 months

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

developers-list February 2016