[Monetdb-developers] [Monetdb-checkins] MonetDB4/src/modules/contrib malalgebra.mx, MonetDB_4-24, 1.8.2.1, 1.8.2.2

Martin Kersten Martin.Kersten at cwi.nl
Wed Oct 29 10:56:24 CET 2008


I would only move items into the development trunk that are part of GDK.

Stefan Manegold wrote:
> On Wed, Oct 29, 2008 at 10:27:35AM +0100, Stefan Manegold wrote:
>   
>> Gents,
>>
>> given (1) the inherent incompatibility between the PF_ROX branch (forked of
>> the XQuery_0-24 branch of pathfinder) and the pathfinder development trunk
>>     
>
> and (hence) the incompatibility between the PF_ROX branch and the
> development trunks of MonetDB & MonetDB4
>
>   
>> and the facts that (2) PF_ROX (at least for now) is purely for internal
>> research purposes and (3) cahnges ("features") in PF_ROX do (e.g.) not
>> support updates, i.e., are far from becoming "main stream",
>>
>> I guess the (only?) proper solution would (have been?) to also fork PF_ROX
>> branches of the MonetDB_1-24 branch (of MonetDB) and the MonetDB_4-24 branch
>> (of MonetDB4) to accomodate Peter's yesterday's changes.
>>
>> Note though, that this would / will exclude PF_ROX-specific changes in
>> MonetDB & MonetDB$ from being tested --- this is already the case for the
>> pathfinder PF_ROX branch and will be the case for the MonetDB_1-24 &
>> MonetDB_4-24 branches, too, once we start prparing for the next release no
>> later than next week.
>>     
>
> Also, changes from the PF_ROX branches of MonetDB & MonetDB4 would/will than
> NOT be propagated to the respective development trunks; the same will be the
> case for (potential0 changes to the MonetDB_1-24 & MonetDB_4-24 branches
> as soon as we start prparing for the next release no later than next week.
>
> In fact, I would even suggest to not propagate Peter's yesterday's changes
> to the development trunk since they are "PF_ROX-specific" (at least for
> now).
>
> Stefan
>
>   
>> See the results of "Stable" testing (once they are ready later today) to
>> judged whether testing is "desired"/"required" or not ...
>>
>> Stefan
>>
>> On Tue, Oct 28, 2008 at 11:36:17PM +0100, Sjoerd Mullender wrote:
>>     
>>> On 2008-10-28 22:37, Peter Boncz wrote:
>>>       
>>>> Hi Sjoerd,
>>>>
>>>> The one problem that could occur is people trying to check out MonetDB_4-24
>>>> now, compiling it and trying to use with a precompiled MonetDB_1-24, without
>>>> updating/recompiling that one. So, I agree with you that a problem could
>>>> occur, but I would be surprised if any of our users would really hit this.
>>>>         
>>> Because we are going to abandon this branch I was willing to let the 
>>> change pass.  But in general I am opposed.
>>>
>>> I agree that in this particular case, it is unlikely that people will 
>>> encounter any problems.  But again, that is because we are going to 
>>> abandon this branch.
>>>
>>>       
>>>> With excuses to pick nits, but contrary to you I believe that API appends
>>>> are not "just as bad" as API modifications. An API modification in MonetDB,
>>>> would force a release in MonetDB5, for instance. An API append does not do
>>>> that (so it is "less bad").
>>>>         
>>> It *is* bad.  See the disaster involving the addition of True and False 
>>> in Python in a minor release.  They have sworn to never make that 
>>> mistake again.  People expect API stability in minor releases and adding 
>>> something to the API is *not* stability.
>>>
>>>       
>>>> PF_ROX is quite different from the HEAD in the pathfinder code due to bot
>>>> branches making different changes to the indexing scheme; afaik this has
>>>> prevented us from keeping it in sync with the HEAD. Various API changes (not
>>>> appends) and even buildtool changes now force it to depend on the stable
>>>> MonetDB/MonetDB4. For PF_ROX itself this is not a problem as it has no
>>>> outside users. However, it might occur that this scenario will repeat
>>>> (PF_ROX triggering a change/addition in MonetDB/MonetDB4 that would trickle
>>>> towards the head via MonetDB_[14]-24).
>>>>         
>>> I am not talking about the PF_ROX branch.  It is not a "stable" branch 
>>> and you can do with it what you want.  But please do consider whether it 
>>> is possible to use the development branches of MonetDB and MonetDB4 with 
>>> PF_ROX.  If you can do that, you can make needed changes to the API there.
>>>
>>>       
>>>> Peter
>>>>
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: Sjoerd Mullender [mailto:sjoerd at acm.org] 
>>>> Sent: Tuesday, October 28, 2008 9:50 PM
>>>> To: monetdb-developers at lists.sourceforge.net
>>>> Cc: monetdb-checkins at lists.sourceforge.net
>>>> Subject: Re: [Monetdb-checkins] MonetDB4/src/modules/contrib malalgebra.mx,
>>>> MonetDB_4-24, 1.8.2.1, 1.8.2.2
>>>>
>>>> On 2008-10-28 20:20, Peter Boncz wrote:
>>>>         
>>>>> Hi Sjoerd,
>>>>>
>>>>> Thanks for keeping an eye on developer behavior and effort to educate. My
>>>>> change, however, only introduces new functions in GDK and commands in MIL;
>>>>>           
>>>> it
>>>>         
>>>>> is only an API append.
>>>>>           
>>>> API appends are just as bad as any other API change.  If new code
>>>> depends on the extra calls, it cannot run with the older version of the
>>>> release.  That is bad.  It means that the branch is *not* "stable" anymore.
>>>>
>>>>         
>>>>> It is checked into the stable branch because afaik PF_ROX only works with
>>>>> that. But I may as often be mistaken.
>>>>>           
>>>> I noticed it was for PF_ROX, and that does pose a problem.  It seems
>>>> that the PF_ROX branch should depend on the current version of MonetDB
>>>> and not on the stable version.
>>>>
>>>>         
>>>>> Peter
>>>>>
>>>>>           
>>>>>> This type of change is not allowed on a stable branch.  Changes of API
>>>>>> are *not* allowed.
>>>>>>
>>>>>> Since we're going to abandon this stable branch soon, I'll let it pass.
>>>>>>
>>>>>> On 2008-10-28 18:49, Peter Boncz wrote:
>>>>>>             
>>>>>>> Update of /cvsroot/monetdb/MonetDB4/src/modules/contrib
>>>>>>> In directory
>>>>>>> 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv901/src/modules/contrib
>>>>>>>
>>>>>>> Modified Files:
>>>>>>>       Tag: MonetDB_4-24
>>>>>>>  malalgebra.mx
>>>>>>> Log Message:
>>>>>>> changes to minimize sampling overhead (mostly in XML text index)
>>>>>>>
>>>>>>> - introduce bandmergejoin(), and leftmergejoin()/bandmergejoin() with a
>>>>>>> cutoff
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> U malalgebra.mx
>>>>>>> Index: malalgebra.mx
>>>>>>> ===================================================================
>>>>>>> RCS file: /cvsroot/monetdb/MonetDB4/src/modules/contrib/malalgebra.mx,v
>>>>>>> retrieving revision 1.8.2.1
>>>>>>> retrieving revision 1.8.2.2
>>>>>>> diff -u -d -r1.8.2.1 -r1.8.2.2
>>>>>>> --- malalgebra.mx 10 Oct 2008 08:42:22 -0000 1.8.2.1
>>>>>>> +++ malalgebra.mx 28 Oct 2008 17:49:34 -0000 1.8.2.2
>>>>>>> @@ -43,7 +43,6 @@
>>>>>>>  @:leftjoin(fetch,\nCAUTION: positional matches are assumed not to be
>>>>>>> out-of-bounds!!)@
>>>>>>>  @:leftjoin(merge)@
>>>>>>>  @:leftjoin(hash)@
>>>>>>> -
>>>>>>>    .COMMAND leftthetajoin ( BAT[any::1,any::2] left, BAT[any::2,any::3]
>>>>>>> right, int mode) :
>>>>>>>                      BAT[any::1,any::3] = CMDleftthetajoin;
>>>>>>>    "Hook directly into the 'leftthetajoin' implementation of the join.
>>>>>>> @@ -58,6 +57,22 @@
>>>>>>>     Also, for each left tuple, all matching right tuples will appear in
>>>>>>> their order
>>>>>>>     of appearrance in the right BAT. This property is handy for XQuery
>>>>>>> processing."
>>>>>>>
>>>>>>> +  .COMMAND bandmergejoin ( BAT[any::1,any::2] outer, BAT[any::2,any::3]
>>>>>>> inner,
>>>>>>> +                               any::2 minus, any::2 plus, bit l_in, bit
>>>>>>> h_in) :
>>>>>>> +                                BAT[any::1,any::3] = CMDbandmergejoin;
>>>>>>> +  "The bandjoin algorithm, but forced to use (left) mergejoin rather
>>>>>>>               
>>>> than
>>>>         
>>>>>>> nested loop"
>>>>>>> +
>>>>>>> +  .COMMAND bandmergejoin ( BAT[any::1,any::2] outer, BAT[any::2,any::3]
>>>>>>> inner,
>>>>>>> +                               any::2 minus, any::2 plus, bit l_in, bit
>>>>>>> h_in, lng limit) :
>>>>>>> +                                BAT[lng,BAT] = CMDbandmergejoin_limit;
>>>>>>> +  "A bandjoin that uses the merge algorithm and limits the result size.
>>>>>>> +   Result generation is cut off after this point and the
>>>>>>> +   result [lng,BAT] holds [estimated_full_resultsize,limited_result]"
>>>>>>> +
>>>>>>> +  .COMMAND leftmergejoin ( BAT[any::1,any::2] left, BAT[any::2,any::3]
>>>>>>> right, lng limit) :
>>>>>>> +                    BAT[lng,BAT] = CMDleftmergejoin_limit;
>>>>>>> +  "leftmergejoin, but limit the result (by cutting off result
>>>>>>>               
>>>> generation)
>>>>         
>>>>>>> +   return a BAT[lng,BAT] containing
>>>>>>>               
>>>> [full_result_estimate,limited_result]"
>>>>         
>>>>>>>  @= select
>>>>>>>  .COMMAND ord_ at 1select ( BAT[any::1,any::2] b, any::2 low, any::2 high)
>>>>>>>               
>>>> :
>>>>         
>>>>>>> @@ -119,6 +134,7 @@
>>>>>>>  #ifndef __MALALGEBRA_H__
>>>>>>>  #define __MALALGEBRA_H__
>>>>>>>  #include "gdk.h"
>>>>>>> +#include "gdk_rangejoin.h"
>>>>>>>
>>>>>>>  /* nothing much */
>>>>>>>
>>>>>>> @@ -240,6 +256,40 @@
>>>>>>>   return (*result = BATnlthetajoin(l, r, *mode, (size_t) * estimate)) ?
>>>>>>> GDK_SUCCEED : GDK_FAIL;
>>>>>>>  }
>>>>>>>
>>>>>>> +int
>>>>>>> +CMDbandmergejoin(BAT **result, BAT *left, BAT *right, ptr minus, ptr
>>>>>>>               
>>>> plus,
>>>>         
>>>>>>> bit *li, bit *hi)
>>>>>>> +{
>>>>>>> +        return (*result = BATbandmergejoin(left, right, minus, plus,
>>>>>>>               
>>>> *li,
>>>>         
>>>>>>> *hi)) ? GDK_SUCCEED : GDK_FAIL;
>>>>>>> +}
>>>>>>> +
>>>>>>> +int
>>>>>>> +CMDbandmergejoin_limit(BAT **result, BAT *left, BAT *right, ptr minus,
>>>>>>>               
>>>> ptr
>>>>         
>>>>>>> plus, bit *li, bit *hi, lng* limit)
>>>>>>> +{
>>>>>>> + size_t cutoff = *limit;
>>>>>>> +        *result = BATbandmergejoin_limit(left, right, minus, plus, *li,
>>>>>>> *hi, &cutoff);
>>>>>>> +        if (*result) {
>>>>>>> +                bat bid = (*result)->batCacheid;
>>>>>>> +                *result = BATnew(TYPE_lng, TYPE_bat, 1);
>>>>>>> +                if (*result) BUNins(*result, &cutoff, &bid, FALSE);
>>>>>>> +                BBPunfix(bid);
>>>>>>> +        }
>>>>>>> +        return (*result)?GDK_SUCCEED:GDK_FAIL;
>>>>>>> +}
>>>>>>> +
>>>>>>> +int
>>>>>>> +CMDleftmergejoin_limit(BAT **result, BAT *left, BAT *right, lng* limit)
>>>>>>> +{
>>>>>>> + size_t cutoff = *limit;
>>>>>>> +        *result = BATleftmergejoin_limit(left, right, cutoff, &cutoff);
>>>>>>> +        if (*result) {
>>>>>>> +                bat bid = (*result)->batCacheid;
>>>>>>> +                *result = BATnew(TYPE_lng, TYPE_bat, 1);
>>>>>>> +                if (*result) BUNins(*result, &cutoff, &bid, FALSE);
>>>>>>> +                BBPunfix(bid);
>>>>>>> +        }
>>>>>>> +        return (*result)?GDK_SUCCEED:GDK_FAIL;
>>>>>>> +}
>>>>>>> +
>>>>>>>  @= selectcmd
>>>>>>>  int CMDord_ at 1select1(BAT **result, BAT* b, ptr value) {
>>>>>>>   return (*result = BAT_select_(b, value, 0, TRUE, TRUE, @2, FALSE,
>>>>>>> TRUE))?GDK_SUCCEED:GDK_FAIL;
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>> -------------------------------------------------------------------------
>>>>         
>>>>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's
>>>>>>>               
>>>> challenge
>>>>         
>>>>>>> Build the coolest Linux based applications with Moblin SDK & win great
>>>>>>> prizes
>>>>>>> Grand prize is a trip for two to an Open Source event anywhere in the
>>>>>>>               
>>>> world
>>>>         
>>>>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>>>>>> _______________________________________________
>>>>>>> Monetdb-checkins mailing list
>>>>>>> Monetdb-checkins at lists.sourceforge.net
>>>>>>> https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
>>>>>>>               
>>>>>> --
>>>>>> Sjoerd Mullender
>>>>>>
>>>>>> -------------------------------------------------------------------------
>>>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's
>>>>>>             
>>>> challenge
>>>>         
>>>>>> Build the coolest Linux based applications with Moblin SDK & win great
>>>>>>             
>>>> prizes
>>>>         
>>>>>> Grand prize is a trip for two to an Open Source event anywhere in the
>>>>>>             
>>>> world
>>>>         
>>>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>>>>> _______________________________________________
>>>>>> Monetdb-checkins mailing list
>>>>>> Monetdb-checkins at lists.sourceforge.net
>>>>>> https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
>>>>>>
>>>>>>             
>>>>> -------------------------------------------------------------------------
>>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's
>>>>>           
>>>> challenge
>>>>         
>>>>> Build the coolest Linux based applications with Moblin SDK & win great
>>>>>           
>>>> prizes
>>>>         
>>>>> Grand prize is a trip for two to an Open Source event anywhere in the
>>>>>           
>>>> world
>>>>         
>>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>>>> _______________________________________________
>>>>> Monetdb-checkins mailing list
>>>>> Monetdb-checkins at lists.sourceforge.net
>>>>> https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
>>>>>           
>>>> --
>>>> Sjoerd Mullender
>>>>
>>>> -------------------------------------------------------------------------
>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>>>> Build the coolest Linux based applications with Moblin SDK & win great
>>>> prizes
>>>> Grand prize is a trip for two to an Open Source event anywhere in the world
>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>>> _______________________________________________
>>>> Monetdb-checkins mailing list
>>>> Monetdb-checkins at lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
>>>>
>>>>
>>>> -------------------------------------------------------------------------
>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>>>> Build the coolest Linux based applications with Moblin SDK & win great prizes
>>>> Grand prize is a trip for two to an Open Source event anywhere in the world
>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>>> _______________________________________________
>>>> Monetdb-checkins mailing list
>>>> Monetdb-checkins at lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
>>>>         
>>> -- 
>>> Sjoerd Mullender
>>>
>>> -------------------------------------------------------------------------
>>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>>> Build the coolest Linux based applications with Moblin SDK & win great prizes
>>> Grand prize is a trip for two to an Open Source event anywhere in the world
>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>> _______________________________________________
>>> Monetdb-checkins mailing list
>>> Monetdb-checkins at lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
>>>
>>>       
>> -- 
>> | Dr. Stefan Manegold | mailto:Stefan.Manegold at cwi.nl |
>> | CWI,  P.O.Box 94079 | http://www.cwi.nl/~manegold/  |
>> | 1090 GB Amsterdam   | Tel.: +31 (20) 592-4212       |
>> | The Netherlands     | Fax : +31 (20) 592-4312       |
>>     
>
>   





More information about the developers-list mailing list