[Monetdb-developers] [Monetdb-checkins] MonetDB5/src/optimizer opt_joinpath.mx, May2009, 1.47.2.1, 1.47.2.2

Stefan Manegold Stefan.Manegold at cwi.nl
Mon Jun 8 20:11:50 CEST 2009


On Mon, Jun 08, 2009 at 05:53:42PM +0000, Martin Kersten wrote:
> Update of /cvsroot/monetdb/MonetDB5/src/optimizer
> In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv7310
> 
> Modified Files:
>       Tag: May2009
> 	opt_joinpath.mx 
> Log Message:
> Experimental stable code to deal with too large intermediate joins.
> It marks them as 'postpone' until at least one other join in the path
> has been executed. Multiple variations can be envisioned.
> 
> 
> U opt_joinpath.mx
> Index: opt_joinpath.mx
> ===================================================================
> RCS file: /cvsroot/monetdb/MonetDB5/src/optimizer/opt_joinpath.mx,v
> retrieving revision 1.47.2.1
> retrieving revision 1.47.2.2
> diff -u -d -r1.47.2.1 -r1.47.2.2
> --- opt_joinpath.mx	8 Jun 2009 10:56:25 -0000	1.47.2.1
> +++ opt_joinpath.mx	8 Jun 2009 17:53:40 -0000	1.47.2.2
> @@ -408,7 +408,9 @@
>  {
>  	BAT *b;
>  	BUN estimate, e;
> -	int i, j;
> +	int i, j, k;
> +	int *postpone= (int*) GDKzalloc(sizeof(int) *top);
> +	int postponed=0;
>  
>  	/* solve the join by pairing the smallest first */
>  	while (top > 2) {
> @@ -426,7 +428,7 @@
>  		stream_printf(cntxt->fdout,"#joinPath estimate join(%d,%d) %d\n", joins[i]->batCacheid, 
>  			joins[i+1]->batCacheid,(int)e);
>  #endif
> -			if (e < estimate) {
> +			if (e < estimate &&  ( !(postpone[i] && postpone[i+1]) || postponed==top-1)) {
>  				estimate = e;
>  				j = i;
>  			}
> @@ -434,10 +436,28 @@
>  @-
>  BEWARE. you may not use a size estimation, because it
>  may fire a BATproperty check in a few cases.
> +In case a join fails, we may try another order first before 
> +abandoning the task.
>  @c
>  		b = BATjoin(joins[j], joins[j + 1], BUN_NONE);
> -		if (b==NULL)
> -			return NULL;	/* something seriously went wrong */
> +		if (b==NULL){
> +			if ( postpone[j]){
                             ^^^^^^^^^^^
shouldn't this be (e.g.)
                             postpone[j] && postpone[j+1]
i.e., bailout (only) if the exact join has failed before,
not if an other join with the same (i.e., join(j-1,j))
has failed before?

... just a spontaneous idea ...

Stefan

> +				GDKfree(postpone);
> +				return NULL;
> +			}
> +			/* clear the GDKerrors */
> +			if( cntxt->errbuf )
> +				cntxt->errbuf[0]=0;
> +			postpone[j] = TRUE;
> +			postpone[j+1] = TRUE;
> +			postponed++;
> +			continue;
> +		} else {
> +			/* reset the postponed joins */
> +			for( k=0; k<top; k++)
> +				postpone[k]=FALSE;
> +			postponed = 0;
> +		}
>  #ifdef DEBUG_OPT_JOINPATH
>  		stream_printf(cntxt->fdout, "#joinPath %d:= join(%d,%d)\n", 
>  			b->batCacheid, joins[j]->batCacheid, joins[j + 1]->batCacheid);
> @@ -450,6 +470,7 @@
>  		for (i = j + 1; i < top; i++)
>  			joins[i] = joins[i + 1];
>  	}
> +	GDKfree(postpone);
>  #ifdef DEBUG_OPT_JOINPATH
>  	stream_printf(cntxt->fdout, "#joinPath final join %d and %d\n", joins[0]->batCacheid, joins[1]->batCacheid);
>  #endif
> 
> 
> ------------------------------------------------------------------------------
> Crystal Reports - New Free Runtime and 30 Day Trial
> Check out the new simplified licensing option that enables unlimited
> royalty-free distribution of the report engine for externally facing 
> server and web deployment.
> http://p.sf.net/sfu/businessobjects
> _______________________________________________
> Monetdb-checkins mailing list
> Monetdb-checkins at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
> 
> 

-- 
| Dr. Stefan Manegold | mailto:Stefan.Manegold at cwi.nl |
| CWI,  P.O.Box 94079 | http://www.cwi.nl/~manegold/  |
| 1090 GB Amsterdam   | Tel.: +31 (20) 592-4212       |
| The Netherlands     | Fax : +31 (20) 592-4312       |




More information about the developers-list mailing list