[Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer opt_derivepath.mx, Feb2009, NONE, 1.1.2.1 Makefile.ag, Feb2009, 1.66, 1.66.2.1 opt_prelude.mx, Feb2009, 1.55.2.3, 1.55.2.4

Sjoerd Mullender sjoerd at acm.org
Thu Mar 5 09:57:16 CET 2009


How is this not a new feature?

Martin Kersten wrote:
> Update of /cvsroot/monetdb/MonetDB5/src/optimizer
> In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv2998/optimizer
> 
> Modified Files:
>       Tag: Feb2009
> 	Makefile.ag opt_prelude.mx 
> Added Files:
>       Tag: Feb2009
> 	opt_derivepath.mx 
> Log Message:
> Added the framework for the commissioned derivepath optimizer.
> 
> 
> U opt_prelude.mx
> Index: opt_prelude.mx
> ===================================================================
> RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/opt_prelude.mx,v
> retrieving revision 1.55.2.3
> retrieving revision 1.55.2.4
> diff -u -d -r1.55.2.3 -r1.55.2.4
> --- opt_prelude.mx	1 Mar 2009 19:15:30 -0000	1.55.2.3
> +++ opt_prelude.mx	5 Mar 2009 00:17:12 -0000	1.55.2.4
> @@ -57,6 +57,7 @@
>  opt_export  str deleteRef;
>  opt_export  str depositRef;
>  opt_export  str deriveRef;
> +opt_export  str derivePathRef;
>  opt_export  str differenceRef;
>  opt_export  str divRef;
>  opt_export  str disconnectRef;
> @@ -216,6 +217,7 @@
>  str deleteRef;
>  str depositRef;
>  str deriveRef;
> +str derivePathRef;
>  str differenceRef;
>  str divRef;
>  str disconnectRef;
> @@ -371,6 +373,7 @@
>  		deleteRef = putName("delete",6);
>  		depositRef = putName("deposit",7);
>  		deriveRef = putName("derive",6);
> +		derivePathRef = putName("derivePath",10);
>  		differenceRef= putName("difference",10);
>  		divRef = putName("/",1);
>  		disconnectRef= putName("disconnect",10);
> 
> U Makefile.ag
> Index: Makefile.ag
> ===================================================================
> RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/Makefile.ag,v
> retrieving revision 1.66
> retrieving revision 1.66.2.1
> diff -u -d -r1.66 -r1.66.2.1
> --- Makefile.ag	18 Jan 2009 11:44:14 -0000	1.66
> +++ Makefile.ag	5 Mar 2009 00:17:12 -0000	1.66.2.1
> @@ -32,7 +32,7 @@
>  		opt_singleton.mx opt_costModel.mx opt_reduce.mx opt_macro.mx \
>  		opt_accumulators.mx opt_qep.mx opt_mergetable.mx \
>  		opt_remoteQueries.mx opt_joinselect.mx opt_partitions.mx \
> -		opt_evaluate.mx opt_inline.mx opt_pushranges.mx \
> +		opt_evaluate.mx opt_inline.mx opt_pushranges.mx opt_derivepath.mx \
>  		opt_accessmode.mx opt_joinpath.mx opt_heuristics.mx opt_remap.mx \
>  		opt_statistics.mx opt_trace.mx  opt_recycler.mx opt_dataflow.mx \
>  		opt_mitosis.mx opt_octopus.mx opt_replicator.mx opt_history.mx
> @@ -53,7 +53,7 @@
>  		opt_singleton.mx opt_costModel.mx opt_reduce.mx opt_macro.mx \
>  		opt_accumulators.mx opt_qep.mx opt_mergetable.mx \
>  		opt_remoteQueries.mx opt_joinselect.mx opt_partitions.mx \
> -		opt_evaluate.mx opt_inline.mx opt_pushranges.mx \
> +		opt_evaluate.mx opt_inline.mx opt_pushranges.mx opt_derivepath.mx \
>  		opt_accessmode.mx opt_joinpath.mx opt_heuristics.mx opt_remap.mx \
>  		opt_statistics.mx opt_trace.mx  opt_recycler.mx opt_dataflow.mx \
>  		opt_mitosis.mx opt_octopus.mx opt_replicator.mx opt_history.mx
> @@ -72,7 +72,7 @@
>  		opt_singleton.mx opt_costModel.mx opt_reduce.mx opt_macro.mx \
>  		opt_accumulators.mx opt_qep.mx opt_mergetable.mx \
>  		opt_remoteQueries.mx opt_joinselect.mx opt_partitions.mx \
> -		opt_evaluate.mx opt_inline.mx opt_pushranges.mx \
> +		opt_evaluate.mx opt_inline.mx opt_pushranges.mx opt_derivepath.mx \
>  		opt_accessmode.mx opt_joinpath.mx opt_heuristics.mx opt_remap.mx \
>  		opt_statistics.mx opt_trace.mx  opt_recycler.mx opt_dataflow.mx \
>  		opt_mitosis.mx opt_octopus.mx opt_replicator.mx opt_history.mx
> 
> --- NEW FILE: opt_derivepath.mx ---
> @' The contents of this file are subject to the MonetDB Public License
> @' Version 1.1 (the "License"); you may not use this file except in
> @' compliance with the License. You may obtain a copy of the License at
> @' http://monetdb.cwi.nl/Legal/MonetDBLicense-1.1.html
> @'
> @' Software distributed under the License is distributed on an "AS IS"
> @' basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the
> @' License for the specific language governing rights and limitations
> @' under the License.
> @'
> @' The Original Code is the MonetDB Database System.
> @'
> @' The Initial Developer of the Original Code is CWI.
> @' Portions created by CWI are Copyright (C) 1997-July 2008 CWI.
> @' Copyright August 2008-2009 MonetDB B.V.
> @' All Rights Reserved.
> 
> @f opt_derivepath
> @- Group derive paths
> The routine @sc{optimizer.derivepaths()}
> walks through the program looking for grouping operations
> and cascades them into a single multiple group paths.
> To illustrate, consider
> @verbatim
> 	(ext1,grp1) := group.new(b);
> 	(ext2,grp2) := group.derive(ext1,grp1, c);
> 	(ext3,grp3) := group.derive(ext2,grp2, d);
> @end verbatim
> The result becomes.
> @verbatim
> 	(ext3,grp3) := group.derivepath(b,c,d);
> @end verbatim
> @{
> @mal
> pattern optimizer.derivePath():str
> address OPTderivePath;
> pattern optimizer.derivePath(mod:str, fcn:str):str
> address OPTderivePath
> comment "Join path constructor";
> pattern group.derivePath(l:bat[:any,:any]...):bat[:any,:any]
> address ALGderivePath
> comment "internal routine to handle join paths.
> 	The type analysis is rather tricky.";
> @h
> #ifndef _OPT_DERIVEPATH_
> #define _OPT_DERIVEPATH_
> #include "opt_prelude.h"
> #include "opt_support.h"
> #include "mal_interpreter.h"
> 
> /* #define DEBUG_OPT_DERIVEPATH  */
> @c
> #include "mal_config.h"
> #include "opt_derivepath.h"
> 
> static int
> OPTderivePathImplementation(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr p)
> {
> 	int i, actions=0;
> 	int *pc;
> 	str derivePathRef = putName("derivePath",10);
> 	InstrPtr q;
> 	InstrPtr *old;
> 	int limit,slimit;
> 
> 	(void) cntxt;
> 	(void) stk;
> 	if (varGetProp(mb, getArg(mb->stmt[0], 0), inlineProp) != NULL)
> 		return 0;
> 
> 	old= mb->stmt;
> 	limit= mb->stop;
> 	slimit= mb->ssize;
> 	newMalBlkStmt(mb,mb->ssize);
> 
> 	/* beware, new variables and instructions are introduced */
> 	pc= (int*) GDKmalloc(sizeof(int)* mb->vtop * 2); /* to find last assignment */
> 	memset((char*) pc, 0, sizeof(int)* mb->vtop *2);
> 
> 	for (i = 0; i<limit; i++){
> 		p= old[i];
> 		if( getModuleId(p)== groupRef && getFunctionId(p)== newRef ){
> 			pc[getArg(p,0)] = i;
> 			pc[getArg(p,1)] = i;
> 		}
> 		if( getModuleId(p)== groupRef && getFunctionId(p)== deriveRef ){
> @-
> Try to expand its argument list with what we have found so far.
> This creates a series of join paths, many of which will be removed during deadcode elimination.
> @c
> 			if (pc[getArg(p,2)]== pc[getArg(p,3)]){
> 				q= copyInstruction(getInstrPtr(mb,pc[getArg(p,2)]));
> 				q= pushArgument(mb,q, getArg(p,5));
> 				pc[getArg(p,0)] = i;
> 				pc[getArg(p,1)] = i;
> 				setFunctionId(q,derivePathRef);
> 				p= q;
> 				actions++;
> #ifdef DEBUG_OPT_DERIVEPATH 
> 				stream_printf(cntxt->fdout,"new derivePath instruction\n");
> 				printInstruction(cntxt->fdout,mb, 0, q, LIST_MAL_ALL);
> #endif
> 			}
> 		} 
> 		pushInstruction(mb,p);
> 	}
> 	for(; i<slimit; i++)
> 	if(old[i])
> 		freeInstruction(old[i]);
> 	GDKfree(old);
> 	GDKfree(pc);
> 	OPTDEBUG
> 	if ( actions)
> 		stream_printf(cntxt->fdout,"#opt_derivepath: %d statements glued\n",actions);
> 	return actions;
> }
> @include optimizerWrapper.mx
> @h
> @:exportOptimizer(derivePath)@
> #endif
> opt_export str ALGderivePath(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr pci);
> @c
> #include "opt_statistics.h"
> @:wrapOptimizer(derivePath,OPT_CHECK_ALL)@
> @-
> The derive path optimizer takes a derivation sequence and
> attempts to minimize the intermediate result.
> The choice depends on a good estimate of intermediate
> results using properties.
> For the time being, we use a simplistic model, based
> on the assumption that most joins are foreign key joins anyway.
> @c
> str
> ALGderivePath(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr pci)
> {
> 	(void) cntxt;
> 	(void) mb;
> 	(void) stk;
> 	(void) pci;
> 	throw(MAL,"group.derivePath","nyi");
> }
> @}
> 
> 
> ------------------------------------------------------------------------------
> Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
> -Strategies to boost innovation and cut costs with open source participation
> -Receive a $600 discount off the registration fee with the source code: SFAD
> http://p.sf.net/sfu/XcvMzF8H
> _______________________________________________
> Monetdb-checkins mailing list
> Monetdb-checkins at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/monetdb-checkins


-- 
Sjoerd Mullender

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 369 bytes
Desc: OpenPGP digital signature
URL: <http://www.monetdb.org/pipermail/developers-list/attachments/20090305/782929ca/attachment.sig>


More information about the developers-list mailing list