[Monetdb-developers] [Monetdb-checkins] MonetDB4/src/modules/contrib malalgebra.mx, MonetDB_4-24, 1.8.2.1, 1.8.2.2

Stefan Manegold Stefan.Manegold at cwi.nl
Wed Oct 29 10:35:02 CET 2008


On Wed, Oct 29, 2008 at 10:27:35AM +0100, Stefan Manegold wrote:
> Gents,
> 
> given (1) the inherent incompatibility between the PF_ROX branch (forked of
> the XQuery_0-24 branch of pathfinder) and the pathfinder development trunk

and (hence) the incompatibility between the PF_ROX branch and the
development trunks of MonetDB & MonetDB4

> and the facts that (2) PF_ROX (at least for now) is purely for internal
> research purposes and (3) cahnges ("features") in PF_ROX do (e.g.) not
> support updates, i.e., are far from becoming "main stream",
> 
> I guess the (only?) proper solution would (have been?) to also fork PF_ROX
> branches of the MonetDB_1-24 branch (of MonetDB) and the MonetDB_4-24 branch
> (of MonetDB4) to accomodate Peter's yesterday's changes.
> 
> Note though, that this would / will exclude PF_ROX-specific changes in
> MonetDB & MonetDB$ from being tested --- this is already the case for the
> pathfinder PF_ROX branch and will be the case for the MonetDB_1-24 &
> MonetDB_4-24 branches, too, once we start prparing for the next release no
> later than next week.

Also, changes from the PF_ROX branches of MonetDB & MonetDB4 would/will than
NOT be propagated to the respective development trunks; the same will be the
case for (potential0 changes to the MonetDB_1-24 & MonetDB_4-24 branches
as soon as we start prparing for the next release no later than next week.

In fact, I would even suggest to not propagate Peter's yesterday's changes
to the development trunk since they are "PF_ROX-specific" (at least for
now).

Stefan

> See the results of "Stable" testing (once they are ready later today) to
> judged whether testing is "desired"/"required" or not ...
> 
> Stefan
> 
> On Tue, Oct 28, 2008 at 11:36:17PM +0100, Sjoerd Mullender wrote:
> > On 2008-10-28 22:37, Peter Boncz wrote:
> > > Hi Sjoerd,
> > > 
> > > The one problem that could occur is people trying to check out MonetDB_4-24
> > > now, compiling it and trying to use with a precompiled MonetDB_1-24, without
> > > updating/recompiling that one. So, I agree with you that a problem could
> > > occur, but I would be surprised if any of our users would really hit this.
> > 
> > Because we are going to abandon this branch I was willing to let the 
> > change pass.  But in general I am opposed.
> > 
> > I agree that in this particular case, it is unlikely that people will 
> > encounter any problems.  But again, that is because we are going to 
> > abandon this branch.
> > 
> > > With excuses to pick nits, but contrary to you I believe that API appends
> > > are not "just as bad" as API modifications. An API modification in MonetDB,
> > > would force a release in MonetDB5, for instance. An API append does not do
> > > that (so it is "less bad").
> > 
> > It *is* bad.  See the disaster involving the addition of True and False 
> > in Python in a minor release.  They have sworn to never make that 
> > mistake again.  People expect API stability in minor releases and adding 
> > something to the API is *not* stability.
> > 
> > > PF_ROX is quite different from the HEAD in the pathfinder code due to bot
> > > branches making different changes to the indexing scheme; afaik this has
> > > prevented us from keeping it in sync with the HEAD. Various API changes (not
> > > appends) and even buildtool changes now force it to depend on the stable
> > > MonetDB/MonetDB4. For PF_ROX itself this is not a problem as it has no
> > > outside users. However, it might occur that this scenario will repeat
> > > (PF_ROX triggering a change/addition in MonetDB/MonetDB4 that would trickle
> > > towards the head via MonetDB_[14]-24).
> > 
> > I am not talking about the PF_ROX branch.  It is not a "stable" branch 
> > and you can do with it what you want.  But please do consider whether it 
> > is possible to use the development branches of MonetDB and MonetDB4 with 
> > PF_ROX.  If you can do that, you can make needed changes to the API there.
> > 
> > > Peter
> > > 
> > > 
> > > 
> > > -----Original Message-----
> > > From: Sjoerd Mullender [mailto:sjoerd at acm.org] 
> > > Sent: Tuesday, October 28, 2008 9:50 PM
> > > To: monetdb-developers at lists.sourceforge.net
> > > Cc: monetdb-checkins at lists.sourceforge.net
> > > Subject: Re: [Monetdb-checkins] MonetDB4/src/modules/contrib malalgebra.mx,
> > > MonetDB_4-24, 1.8.2.1, 1.8.2.2
> > > 
> > > On 2008-10-28 20:20, Peter Boncz wrote:
> > >> Hi Sjoerd,
> > >>
> > >> Thanks for keeping an eye on developer behavior and effort to educate. My
> > >> change, however, only introduces new functions in GDK and commands in MIL;
> > > it
> > >> is only an API append.
> > > 
> > > API appends are just as bad as any other API change.  If new code
> > > depends on the extra calls, it cannot run with the older version of the
> > > release.  That is bad.  It means that the branch is *not* "stable" anymore.
> > > 
> > >> It is checked into the stable branch because afaik PF_ROX only works with
> > >> that. But I may as often be mistaken.
> > > 
> > > I noticed it was for PF_ROX, and that does pose a problem.  It seems
> > > that the PF_ROX branch should depend on the current version of MonetDB
> > > and not on the stable version.
> > > 
> > >> Peter
> > >>
> > >>> This type of change is not allowed on a stable branch.  Changes of API
> > >>> are *not* allowed.
> > >>>
> > >>> Since we're going to abandon this stable branch soon, I'll let it pass.
> > >>>
> > >>> On 2008-10-28 18:49, Peter Boncz wrote:
> > >>>> Update of /cvsroot/monetdb/MonetDB4/src/modules/contrib
> > >>>> In directory
> > >>>> 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv901/src/modules/contrib
> > >>>>
> > >>>> Modified Files:
> > >>>>       Tag: MonetDB_4-24
> > >>>>  malalgebra.mx
> > >>>> Log Message:
> > >>>> changes to minimize sampling overhead (mostly in XML text index)
> > >>>>
> > >>>> - introduce bandmergejoin(), and leftmergejoin()/bandmergejoin() with a
> > >>>> cutoff
> > >>>>
> > >>>>
> > >>>>
> > >>>> U malalgebra.mx
> > >>>> Index: malalgebra.mx
> > >>>> ===================================================================
> > >>>> RCS file: /cvsroot/monetdb/MonetDB4/src/modules/contrib/malalgebra.mx,v
> > >>>> retrieving revision 1.8.2.1
> > >>>> retrieving revision 1.8.2.2
> > >>>> diff -u -d -r1.8.2.1 -r1.8.2.2
> > >>>> --- malalgebra.mx 10 Oct 2008 08:42:22 -0000 1.8.2.1
> > >>>> +++ malalgebra.mx 28 Oct 2008 17:49:34 -0000 1.8.2.2
> > >>>> @@ -43,7 +43,6 @@
> > >>>>  @:leftjoin(fetch,\nCAUTION: positional matches are assumed not to be
> > >>>> out-of-bounds!!)@
> > >>>>  @:leftjoin(merge)@
> > >>>>  @:leftjoin(hash)@
> > >>>> -
> > >>>>    .COMMAND leftthetajoin ( BAT[any::1,any::2] left, BAT[any::2,any::3]
> > >>>> right, int mode) :
> > >>>>                      BAT[any::1,any::3] = CMDleftthetajoin;
> > >>>>    "Hook directly into the 'leftthetajoin' implementation of the join.
> > >>>> @@ -58,6 +57,22 @@
> > >>>>     Also, for each left tuple, all matching right tuples will appear in
> > >>>> their order
> > >>>>     of appearrance in the right BAT. This property is handy for XQuery
> > >>>> processing."
> > >>>>
> > >>>> +  .COMMAND bandmergejoin ( BAT[any::1,any::2] outer, BAT[any::2,any::3]
> > >>>> inner,
> > >>>> +                               any::2 minus, any::2 plus, bit l_in, bit
> > >>>> h_in) :
> > >>>> +                                BAT[any::1,any::3] = CMDbandmergejoin;
> > >>>> +  "The bandjoin algorithm, but forced to use (left) mergejoin rather
> > > than
> > >>>> nested loop"
> > >>>> +
> > >>>> +  .COMMAND bandmergejoin ( BAT[any::1,any::2] outer, BAT[any::2,any::3]
> > >>>> inner,
> > >>>> +                               any::2 minus, any::2 plus, bit l_in, bit
> > >>>> h_in, lng limit) :
> > >>>> +                                BAT[lng,BAT] = CMDbandmergejoin_limit;
> > >>>> +  "A bandjoin that uses the merge algorithm and limits the result size.
> > >>>> +   Result generation is cut off after this point and the
> > >>>> +   result [lng,BAT] holds [estimated_full_resultsize,limited_result]"
> > >>>> +
> > >>>> +  .COMMAND leftmergejoin ( BAT[any::1,any::2] left, BAT[any::2,any::3]
> > >>>> right, lng limit) :
> > >>>> +                    BAT[lng,BAT] = CMDleftmergejoin_limit;
> > >>>> +  "leftmergejoin, but limit the result (by cutting off result
> > > generation)
> > >>>> +   return a BAT[lng,BAT] containing
> > > [full_result_estimate,limited_result]"
> > >>>>  @= select
> > >>>>  .COMMAND ord_ at 1select ( BAT[any::1,any::2] b, any::2 low, any::2 high)
> > > :
> > >>>> @@ -119,6 +134,7 @@
> > >>>>  #ifndef __MALALGEBRA_H__
> > >>>>  #define __MALALGEBRA_H__
> > >>>>  #include "gdk.h"
> > >>>> +#include "gdk_rangejoin.h"
> > >>>>
> > >>>>  /* nothing much */
> > >>>>
> > >>>> @@ -240,6 +256,40 @@
> > >>>>   return (*result = BATnlthetajoin(l, r, *mode, (size_t) * estimate)) ?
> > >>>> GDK_SUCCEED : GDK_FAIL;
> > >>>>  }
> > >>>>
> > >>>> +int
> > >>>> +CMDbandmergejoin(BAT **result, BAT *left, BAT *right, ptr minus, ptr
> > > plus,
> > >>>> bit *li, bit *hi)
> > >>>> +{
> > >>>> +        return (*result = BATbandmergejoin(left, right, minus, plus,
> > > *li,
> > >>>> *hi)) ? GDK_SUCCEED : GDK_FAIL;
> > >>>> +}
> > >>>> +
> > >>>> +int
> > >>>> +CMDbandmergejoin_limit(BAT **result, BAT *left, BAT *right, ptr minus,
> > > ptr
> > >>>> plus, bit *li, bit *hi, lng* limit)
> > >>>> +{
> > >>>> + size_t cutoff = *limit;
> > >>>> +        *result = BATbandmergejoin_limit(left, right, minus, plus, *li,
> > >>>> *hi, &cutoff);
> > >>>> +        if (*result) {
> > >>>> +                bat bid = (*result)->batCacheid;
> > >>>> +                *result = BATnew(TYPE_lng, TYPE_bat, 1);
> > >>>> +                if (*result) BUNins(*result, &cutoff, &bid, FALSE);
> > >>>> +                BBPunfix(bid);
> > >>>> +        }
> > >>>> +        return (*result)?GDK_SUCCEED:GDK_FAIL;
> > >>>> +}
> > >>>> +
> > >>>> +int
> > >>>> +CMDleftmergejoin_limit(BAT **result, BAT *left, BAT *right, lng* limit)
> > >>>> +{
> > >>>> + size_t cutoff = *limit;
> > >>>> +        *result = BATleftmergejoin_limit(left, right, cutoff, &cutoff);
> > >>>> +        if (*result) {
> > >>>> +                bat bid = (*result)->batCacheid;
> > >>>> +                *result = BATnew(TYPE_lng, TYPE_bat, 1);
> > >>>> +                if (*result) BUNins(*result, &cutoff, &bid, FALSE);
> > >>>> +                BBPunfix(bid);
> > >>>> +        }
> > >>>> +        return (*result)?GDK_SUCCEED:GDK_FAIL;
> > >>>> +}
> > >>>> +
> > >>>>  @= selectcmd
> > >>>>  int CMDord_ at 1select1(BAT **result, BAT* b, ptr value) {
> > >>>>   return (*result = BAT_select_(b, value, 0, TRUE, TRUE, @2, FALSE,
> > >>>> TRUE))?GDK_SUCCEED:GDK_FAIL;
> > >>>>
> > >>>>
> > >>>>
> > > -------------------------------------------------------------------------
> > >>>> This SF.Net email is sponsored by the Moblin Your Move Developer's
> > > challenge
> > >>>> Build the coolest Linux based applications with Moblin SDK & win great
> > >>>> prizes
> > >>>> Grand prize is a trip for two to an Open Source event anywhere in the
> > > world
> > >>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> > >>>> _______________________________________________
> > >>>> Monetdb-checkins mailing list
> > >>>> Monetdb-checkins at lists.sourceforge.net
> > >>>> https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
> > >>> --
> > >>> Sjoerd Mullender
> > >>>
> > >>> -------------------------------------------------------------------------
> > >>> This SF.Net email is sponsored by the Moblin Your Move Developer's
> > > challenge
> > >>> Build the coolest Linux based applications with Moblin SDK & win great
> > > prizes
> > >>> Grand prize is a trip for two to an Open Source event anywhere in the
> > > world
> > >>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> > >>> _______________________________________________
> > >>> Monetdb-checkins mailing list
> > >>> Monetdb-checkins at lists.sourceforge.net
> > >>> https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
> > >>>
> > >>
> > >> -------------------------------------------------------------------------
> > >> This SF.Net email is sponsored by the Moblin Your Move Developer's
> > > challenge
> > >> Build the coolest Linux based applications with Moblin SDK & win great
> > > prizes
> > >> Grand prize is a trip for two to an Open Source event anywhere in the
> > > world
> > >> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> > >> _______________________________________________
> > >> Monetdb-checkins mailing list
> > >> Monetdb-checkins at lists.sourceforge.net
> > >> https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
> > > 
> > > 
> > > --
> > > Sjoerd Mullender
> > > 
> > > -------------------------------------------------------------------------
> > > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> > > Build the coolest Linux based applications with Moblin SDK & win great
> > > prizes
> > > Grand prize is a trip for two to an Open Source event anywhere in the world
> > > http://moblin-contest.org/redirect.php?banner_id=100&url=/
> > > _______________________________________________
> > > Monetdb-checkins mailing list
> > > Monetdb-checkins at lists.sourceforge.net
> > > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
> > > 
> > > 
> > > -------------------------------------------------------------------------
> > > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> > > Build the coolest Linux based applications with Moblin SDK & win great prizes
> > > Grand prize is a trip for two to an Open Source event anywhere in the world
> > > http://moblin-contest.org/redirect.php?banner_id=100&url=/
> > > _______________________________________________
> > > Monetdb-checkins mailing list
> > > Monetdb-checkins at lists.sourceforge.net
> > > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
> > 
> > 
> > -- 
> > Sjoerd Mullender
> > 
> > -------------------------------------------------------------------------
> > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> > Build the coolest Linux based applications with Moblin SDK & win great prizes
> > Grand prize is a trip for two to an Open Source event anywhere in the world
> > http://moblin-contest.org/redirect.php?banner_id=100&url=/
> > _______________________________________________
> > Monetdb-checkins mailing list
> > Monetdb-checkins at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
> > 
> 
> -- 
> | Dr. Stefan Manegold | mailto:Stefan.Manegold at cwi.nl |
> | CWI,  P.O.Box 94079 | http://www.cwi.nl/~manegold/  |
> | 1090 GB Amsterdam   | Tel.: +31 (20) 592-4212       |
> | The Netherlands     | Fax : +31 (20) 592-4312       |

-- 
| Dr. Stefan Manegold | mailto:Stefan.Manegold at cwi.nl |
| CWI,  P.O.Box 94079 | http://www.cwi.nl/~manegold/  |
| 1090 GB Amsterdam   | Tel.: +31 (20) 592-4212       |
| The Netherlands     | Fax : +31 (20) 592-4312       |




More information about the developers-list mailing list