[Monetdb-developers] [Monetdb-checkins] MonetDB4/src/modules/contrib malalgebra.mx, MonetDB_4-24, 1.8.2.1, 1.8.2.2

Stefan Manegold Stefan.Manegold at cwi.nl
Wed Oct 29 10:27:35 CET 2008


Gents,

given (1) the inherent incompatibility between the PF_ROX branch (forked of
the XQuery_0-24 branch of pathfinder) and the pathfinder development trunk
and the facts that (2) PF_ROX (at least for now) is purely for internal
research purposes and (3) cahnges ("features") in PF_ROX do (e.g.) not
support updates, i.e., are far from becoming "main stream",

I guess the (only?) proper solution would (have been?) to also fork PF_ROX
branches of the MonetDB_1-24 branch (of MonetDB) and the MonetDB_4-24 branch
(of MonetDB4) to accomodate Peter's yesterday's changes.

Note though, that this would / will exclude PF_ROX-specific changes in
MonetDB & MonetDB$ from being tested --- this is already the case for the
pathfinder PF_ROX branch and will be the case for the MonetDB_2-4 &
MonetDB_4-24 branches, too, once we start prparing for the next release no
later than next week.

See the results of "Stable" testing (once they are ready later today) to
judged whether testing is "desired"/"required" or not ...

Stefan

On Tue, Oct 28, 2008 at 11:36:17PM +0100, Sjoerd Mullender wrote:
> On 2008-10-28 22:37, Peter Boncz wrote:
> > Hi Sjoerd,
> > 
> > The one problem that could occur is people trying to check out MonetDB_4-24
> > now, compiling it and trying to use with a precompiled MonetDB_1-24, without
> > updating/recompiling that one. So, I agree with you that a problem could
> > occur, but I would be surprised if any of our users would really hit this.
> 
> Because we are going to abandon this branch I was willing to let the 
> change pass.  But in general I am opposed.
> 
> I agree that in this particular case, it is unlikely that people will 
> encounter any problems.  But again, that is because we are going to 
> abandon this branch.
> 
> > With excuses to pick nits, but contrary to you I believe that API appends
> > are not "just as bad" as API modifications. An API modification in MonetDB,
> > would force a release in MonetDB5, for instance. An API append does not do
> > that (so it is "less bad").
> 
> It *is* bad.  See the disaster involving the addition of True and False 
> in Python in a minor release.  They have sworn to never make that 
> mistake again.  People expect API stability in minor releases and adding 
> something to the API is *not* stability.
> 
> > PF_ROX is quite different from the HEAD in the pathfinder code due to bot
> > branches making different changes to the indexing scheme; afaik this has
> > prevented us from keeping it in sync with the HEAD. Various API changes (not
> > appends) and even buildtool changes now force it to depend on the stable
> > MonetDB/MonetDB4. For PF_ROX itself this is not a problem as it has no
> > outside users. However, it might occur that this scenario will repeat
> > (PF_ROX triggering a change/addition in MonetDB/MonetDB4 that would trickle
> > towards the head via MonetDB_[14]-24).
> 
> I am not talking about the PF_ROX branch.  It is not a "stable" branch 
> and you can do with it what you want.  But please do consider whether it 
> is possible to use the development branches of MonetDB and MonetDB4 with 
> PF_ROX.  If you can do that, you can make needed changes to the API there.
> 
> > Peter
> > 
> > 
> > 
> > -----Original Message-----
> > From: Sjoerd Mullender [mailto:sjoerd at acm.org] 
> > Sent: Tuesday, October 28, 2008 9:50 PM
> > To: monetdb-developers at lists.sourceforge.net
> > Cc: monetdb-checkins at lists.sourceforge.net
> > Subject: Re: [Monetdb-checkins] MonetDB4/src/modules/contrib malalgebra.mx,
> > MonetDB_4-24, 1.8.2.1, 1.8.2.2
> > 
> > On 2008-10-28 20:20, Peter Boncz wrote:
> >> Hi Sjoerd,
> >>
> >> Thanks for keeping an eye on developer behavior and effort to educate. My
> >> change, however, only introduces new functions in GDK and commands in MIL;
> > it
> >> is only an API append.
> > 
> > API appends are just as bad as any other API change.  If new code
> > depends on the extra calls, it cannot run with the older version of the
> > release.  That is bad.  It means that the branch is *not* "stable" anymore.
> > 
> >> It is checked into the stable branch because afaik PF_ROX only works with
> >> that. But I may as often be mistaken.
> > 
> > I noticed it was for PF_ROX, and that does pose a problem.  It seems
> > that the PF_ROX branch should depend on the current version of MonetDB
> > and not on the stable version.
> > 
> >> Peter
> >>
> >>> This type of change is not allowed on a stable branch.  Changes of API
> >>> are *not* allowed.
> >>>
> >>> Since we're going to abandon this stable branch soon, I'll let it pass.
> >>>
> >>> On 2008-10-28 18:49, Peter Boncz wrote:
> >>>> Update of /cvsroot/monetdb/MonetDB4/src/modules/contrib
> >>>> In directory
> >>>> 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv901/src/modules/contrib
> >>>>
> >>>> Modified Files:
> >>>>       Tag: MonetDB_4-24
> >>>>  malalgebra.mx
> >>>> Log Message:
> >>>> changes to minimize sampling overhead (mostly in XML text index)
> >>>>
> >>>> - introduce bandmergejoin(), and leftmergejoin()/bandmergejoin() with a
> >>>> cutoff
> >>>>
> >>>>
> >>>>
> >>>> U malalgebra.mx
> >>>> Index: malalgebra.mx
> >>>> ===================================================================
> >>>> RCS file: /cvsroot/monetdb/MonetDB4/src/modules/contrib/malalgebra.mx,v
> >>>> retrieving revision 1.8.2.1
> >>>> retrieving revision 1.8.2.2
> >>>> diff -u -d -r1.8.2.1 -r1.8.2.2
> >>>> --- malalgebra.mx 10 Oct 2008 08:42:22 -0000 1.8.2.1
> >>>> +++ malalgebra.mx 28 Oct 2008 17:49:34 -0000 1.8.2.2
> >>>> @@ -43,7 +43,6 @@
> >>>>  @:leftjoin(fetch,\nCAUTION: positional matches are assumed not to be
> >>>> out-of-bounds!!)@
> >>>>  @:leftjoin(merge)@
> >>>>  @:leftjoin(hash)@
> >>>> -
> >>>>    .COMMAND leftthetajoin ( BAT[any::1,any::2] left, BAT[any::2,any::3]
> >>>> right, int mode) :
> >>>>                      BAT[any::1,any::3] = CMDleftthetajoin;
> >>>>    "Hook directly into the 'leftthetajoin' implementation of the join.
> >>>> @@ -58,6 +57,22 @@
> >>>>     Also, for each left tuple, all matching right tuples will appear in
> >>>> their order
> >>>>     of appearrance in the right BAT. This property is handy for XQuery
> >>>> processing."
> >>>>
> >>>> +  .COMMAND bandmergejoin ( BAT[any::1,any::2] outer, BAT[any::2,any::3]
> >>>> inner,
> >>>> +                               any::2 minus, any::2 plus, bit l_in, bit
> >>>> h_in) :
> >>>> +                                BAT[any::1,any::3] = CMDbandmergejoin;
> >>>> +  "The bandjoin algorithm, but forced to use (left) mergejoin rather
> > than
> >>>> nested loop"
> >>>> +
> >>>> +  .COMMAND bandmergejoin ( BAT[any::1,any::2] outer, BAT[any::2,any::3]
> >>>> inner,
> >>>> +                               any::2 minus, any::2 plus, bit l_in, bit
> >>>> h_in, lng limit) :
> >>>> +                                BAT[lng,BAT] = CMDbandmergejoin_limit;
> >>>> +  "A bandjoin that uses the merge algorithm and limits the result size.
> >>>> +   Result generation is cut off after this point and the
> >>>> +   result [lng,BAT] holds [estimated_full_resultsize,limited_result]"
> >>>> +
> >>>> +  .COMMAND leftmergejoin ( BAT[any::1,any::2] left, BAT[any::2,any::3]
> >>>> right, lng limit) :
> >>>> +                    BAT[lng,BAT] = CMDleftmergejoin_limit;
> >>>> +  "leftmergejoin, but limit the result (by cutting off result
> > generation)
> >>>> +   return a BAT[lng,BAT] containing
> > [full_result_estimate,limited_result]"
> >>>>  @= select
> >>>>  .COMMAND ord_ at 1select ( BAT[any::1,any::2] b, any::2 low, any::2 high)
> > :
> >>>> @@ -119,6 +134,7 @@
> >>>>  #ifndef __MALALGEBRA_H__
> >>>>  #define __MALALGEBRA_H__
> >>>>  #include "gdk.h"
> >>>> +#include "gdk_rangejoin.h"
> >>>>
> >>>>  /* nothing much */
> >>>>
> >>>> @@ -240,6 +256,40 @@
> >>>>   return (*result = BATnlthetajoin(l, r, *mode, (size_t) * estimate)) ?
> >>>> GDK_SUCCEED : GDK_FAIL;
> >>>>  }
> >>>>
> >>>> +int
> >>>> +CMDbandmergejoin(BAT **result, BAT *left, BAT *right, ptr minus, ptr
> > plus,
> >>>> bit *li, bit *hi)
> >>>> +{
> >>>> +        return (*result = BATbandmergejoin(left, right, minus, plus,
> > *li,
> >>>> *hi)) ? GDK_SUCCEED : GDK_FAIL;
> >>>> +}
> >>>> +
> >>>> +int
> >>>> +CMDbandmergejoin_limit(BAT **result, BAT *left, BAT *right, ptr minus,
> > ptr
> >>>> plus, bit *li, bit *hi, lng* limit)
> >>>> +{
> >>>> + size_t cutoff = *limit;
> >>>> +        *result = BATbandmergejoin_limit(left, right, minus, plus, *li,
> >>>> *hi, &cutoff);
> >>>> +        if (*result) {
> >>>> +                bat bid = (*result)->batCacheid;
> >>>> +                *result = BATnew(TYPE_lng, TYPE_bat, 1);
> >>>> +                if (*result) BUNins(*result, &cutoff, &bid, FALSE);
> >>>> +                BBPunfix(bid);
> >>>> +        }
> >>>> +        return (*result)?GDK_SUCCEED:GDK_FAIL;
> >>>> +}
> >>>> +
> >>>> +int
> >>>> +CMDleftmergejoin_limit(BAT **result, BAT *left, BAT *right, lng* limit)
> >>>> +{
> >>>> + size_t cutoff = *limit;
> >>>> +        *result = BATleftmergejoin_limit(left, right, cutoff, &cutoff);
> >>>> +        if (*result) {
> >>>> +                bat bid = (*result)->batCacheid;
> >>>> +                *result = BATnew(TYPE_lng, TYPE_bat, 1);
> >>>> +                if (*result) BUNins(*result, &cutoff, &bid, FALSE);
> >>>> +                BBPunfix(bid);
> >>>> +        }
> >>>> +        return (*result)?GDK_SUCCEED:GDK_FAIL;
> >>>> +}
> >>>> +
> >>>>  @= selectcmd
> >>>>  int CMDord_ at 1select1(BAT **result, BAT* b, ptr value) {
> >>>>   return (*result = BAT_select_(b, value, 0, TRUE, TRUE, @2, FALSE,
> >>>> TRUE))?GDK_SUCCEED:GDK_FAIL;
> >>>>
> >>>>
> >>>>
> > -------------------------------------------------------------------------
> >>>> This SF.Net email is sponsored by the Moblin Your Move Developer's
> > challenge
> >>>> Build the coolest Linux based applications with Moblin SDK & win great
> >>>> prizes
> >>>> Grand prize is a trip for two to an Open Source event anywhere in the
> > world
> >>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> >>>> _______________________________________________
> >>>> Monetdb-checkins mailing list
> >>>> Monetdb-checkins at lists.sourceforge.net
> >>>> https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
> >>> --
> >>> Sjoerd Mullender
> >>>
> >>> -------------------------------------------------------------------------
> >>> This SF.Net email is sponsored by the Moblin Your Move Developer's
> > challenge
> >>> Build the coolest Linux based applications with Moblin SDK & win great
> > prizes
> >>> Grand prize is a trip for two to an Open Source event anywhere in the
> > world
> >>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> >>> _______________________________________________
> >>> Monetdb-checkins mailing list
> >>> Monetdb-checkins at lists.sourceforge.net
> >>> https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
> >>>
> >>
> >> -------------------------------------------------------------------------
> >> This SF.Net email is sponsored by the Moblin Your Move Developer's
> > challenge
> >> Build the coolest Linux based applications with Moblin SDK & win great
> > prizes
> >> Grand prize is a trip for two to an Open Source event anywhere in the
> > world
> >> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> >> _______________________________________________
> >> Monetdb-checkins mailing list
> >> Monetdb-checkins at lists.sourceforge.net
> >> https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
> > 
> > 
> > --
> > Sjoerd Mullender
> > 
> > -------------------------------------------------------------------------
> > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> > Build the coolest Linux based applications with Moblin SDK & win great
> > prizes
> > Grand prize is a trip for two to an Open Source event anywhere in the world
> > http://moblin-contest.org/redirect.php?banner_id=100&url=/
> > _______________________________________________
> > Monetdb-checkins mailing list
> > Monetdb-checkins at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
> > 
> > 
> > -------------------------------------------------------------------------
> > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> > Build the coolest Linux based applications with Moblin SDK & win great prizes
> > Grand prize is a trip for two to an Open Source event anywhere in the world
> > http://moblin-contest.org/redirect.php?banner_id=100&url=/
> > _______________________________________________
> > Monetdb-checkins mailing list
> > Monetdb-checkins at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
> 
> 
> -- 
> Sjoerd Mullender
> 
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Monetdb-checkins mailing list
> Monetdb-checkins at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
> 

-- 
| Dr. Stefan Manegold | mailto:Stefan.Manegold at cwi.nl |
| CWI,  P.O.Box 94079 | http://www.cwi.nl/~manegold/  |
| 1090 GB Amsterdam   | Tel.: +31 (20) 592-4212       |
| The Netherlands     | Fax : +31 (20) 592-4312       |




More information about the developers-list mailing list