[Monetdb-developers] next release: mps or algebra?

Djoerd Hiemstra hiemstra at cs.utwente.nl
Wed Oct 17 00:30:23 CEST 2007

Dear all,

Very good: no more milprinting! In absence of Jan, please let me add the 
plans for PF/Tijah. To make things slightly more complicated: The PF/Tijah 
team would like to use the algebra release for porting our proprietary 
search extensions to XQuery Full-text, see: 
http://www.w3.org/TR/xquery-full-text/ Some initial design decisions that 
we like to follow have been made by Stefan Klinger at the University of 
Konstanz. The consequences of such a port would be:

 1. XQuery syntax will be extended for text search queries (now only via
    user-defined functions)
 2. All algebra operations must be score aware (i.e., if a score column
    exists, scores have to be propagated and aggregated)

We are aware that this affects everyone, although, only the extended 
queries will produce a score column, and algebra operations remain exactly 
the same if score columns are not there. The time goal of this release is 
XMAS 2008. In the mean time, we might not port the current PF/Tijah 
extensions to the algebra.

Best,  Djoerd.

On Tue, 16 Oct 2007, Martin Kersten wrote:

> LS,
> Good to see this discussion. Although the topic is primarily 
> MonetDB/XQuery it is good
> to keep an eye on the global picture. Releases include more stuff and 
> requires a lot of
> time to prepare. (the release itself easily consumes several months of 
> full time work,
> many delivered by free (weekend) time!!!!)
> Good planning and long-term internal use are a prerequisite.
> Something we still have to learn.
> Once the release cycle starts, the train can not stop and full power 
> is/should
> be in the hands of a single person. The release manager currently is Sjoerd,
> one of the project admins!
> As soon as a RC is available, those with a non-standard platform 
> (windows, sun,
> mac, vista) should immediately double check to assure it basically works.
> This holds for all addressed and developers.
> In this respect, the time frame mentioned below is overly optimistic. 
> Anyone close to
> the core group should work on the HEAD to help uncover problems from day 1.
> Code changes should *not* be collected by individual programmers until
> release time .
> This is a major source of frustration, because you simply may not expect
> your team members to spent day and night to fight the 'last' bugs. Let 
> alone to
> quick-test your latest feature or performance improvement.
> PLEASE, check-in daily your changes and make sure you have at least ran
> mtest on your platform. And most of all, know that your changes are
> scrutinized for potential quality problems by others. It is the best way
> to avoid many bugs.
> Furthermore, changes in the interfaces visible to users and the effect 
> on their databases
> should be kept to a minimum. Backward compatibility is a must as long as 
> it is manageable.
> Beware that a change in storage structure may help a few extreme users, 
> but may affects
> hundreds or thousands in their daily use. E.g. one of our users would 
> have to unload/reload
> a 0.5Tb business critical application.
> MonetDB/Query and MonetDB/SQL are intertwined. This creates responsibility
> both ways.
> Peter Boncz wrote:
> > Stefan suggested rightly to move this discussion to the mailing list.
> > I also changed the language into english for that purpose.
> >
> > Maurice' original questions were:
> >
> > (a) whether the new release would be algebra-only (short answer: hopefully).
> In my opinion this can only be answered if the algebra version, in its 
> full glory (without mps)
> is used for several months internally, e.g. using students for stress 
> testing.
> > (b) whether our talk of a new release also refers to submodules, such as
> > probxml and pftijah (short answer: yes)
> >   
> Submodules basically follow their own release cycle. Regardless the 
> source, e.g. datacells
> geom, ....  Keeping in line is the responsibility of their owners, and 
> the core developers
> should be aware that their changes may break the work of others. Or at 
> least drain a
> possibly substantial amount of time.
> > (c) whether all issues on the wiki, see:
> > http://www.pathfinder-xquery.org/wiki/index.php/Jupiter:_The_Algebra_Release
> >     needed for this action, have already been resolved (short answer: no)
> >   
> The pathfinder wiki is a good example of team documentation.
> I am looking forward to the upcoming release. The next one might be this 
> year,
> but not necessarily includes everything on the drawing board.
> regards, Martin
> > On Tue, Oct 16, 2007 at 05:23:43PM +0200, Peter Boncz wrote:
> >   
> >> Hi Maurice, et al,
> >>
> >> The release of (hopefully) this week will still be mps based. I also
> >> anticipate at least one bug-fix release (ie 0.20.2), or even more.  However,
> >> the next major release (0.22) should be algebra based, in my opinion. I am
> >> certainly reluctant to keep investing any more effort in mps. I would aim
> >> for an mps-free release, yes.
> >>
> >> The time-goal of that release is "XMAS 2007", as far as we are concerned.
> >> For the work of built-in functions I have good hope that with the help of
> >> Lefteris we can come a long way.
> >>
> >> I hope and think updates can also be ported by joerd in that time-frame. As
> >> for pftijah, if Jan Flokstra has time, it should be possible to port that as
> >> well in 2-3 months.
> >>
> >> There are some hairy issues, one of them being free-form recursion (needed
> >> by you), another one being precompiled modules (needed by XRPC). The
> >> resolution of these needs assistance from Munich, and maybe related. It needs
> >> a method to compile certain functions into scripted procedures (here MIL)
> >> and calling them. One algorithm is to do this for all functions in modules,
> >> as well as for all recursive functions. The issue to resolve is the
> >> representation of the parameters.
> >>
> >> Depending on how much time Jan Rittinger has available for this, we will
> >> have to find creative solutions for this. Indeed, a backup solution would be
> >> to keep handling certain queries for the time being with mps, but that
> >> surely is ugly.
> >>
> >> Peter
> >>
> >>   -----Original Message-----
> >>   From: Keulen, M. van (Maurice) [mailto:m.vankeulen at utwente.nl]
> >>   Sent: dinsdag 16 oktober 2007 16:38
> >>   To: Peter Boncz [CWI]; Stefan Manegold [CWI]
> >>   Cc: Djoerd Hiemstra; Jan Flokstra; Henning Rode; Abdel Kader, R. (Riham)
> >>   Subject: Nieuwe releases en mps-backend
> >>
> >>
> >>   Hoi,
> >>
> >>   Sorry dat ik niet bij de Skype-sessie van vandaag kon zijn, maar Riham
> >> vertelde me dat het onderwerp van nieuwe releases en de mps-backend ter
> >> sprake is geweest. Het riep wat specifieke vragen bij me op die ze niet
> >> allemaal kon beantwoorden:
> >>
> >>     a.. Is het de bedoeling dat een nieuwe release de mps-backend helemaal
> >> niet meer bevat of alleen maar de algebra-backend als default heeft?
> >>     b.. Als jullie praten over "nieuwe release", bedoel je dan alleen een
> >> package van alle modules van MonetDB/XQuery? Maw zegt het niets over
> >> eventuele `subreleases' van individuele modules?
> >>
> >>     c.. Op de wiki hebben we een pagina met open issues voor de
> >> algebra-release. Wat is de status van die open issues? Ik zie namelijk op de
> >> pagina nergens dat bepaalde issues al opgelost/geïmplementeerd zijn.
> >>   Ik Cc deze e-mail even aan de PF/Tijah-mensen, want  hoewel er levendige
> >> discussies zijn hoe PF/Tijah naar de algebra-backend to porten, ik heb niet
> >> het idee dat er oplossingen zijn die op korte termijn realiseerbaar zijn.
> >> Evenzo het recursie-vraagstuk. Het lijkt me bijvoorbeeld voor de
> >> zichtbaarheid naar buiten toe onwenselijk als deze features ineens zouden
> >> wegvallen voor non-insiders. Of andere onwenselijke consequenties van het
> >> niet meer (sub)releasen van modules die gebaseerd zijn op de mps-backend.
> >>
> >>   Groetjes,
> >>   Maurice.
> >>
> >> --
> >> ----------------------------------------------------------------------
> >> Dr.Ir. M. van Keulen - Assistant Professor, Data Management Technology
> >> Univ. of Twente, Dept of EEMCS, POBox 217, 7500 AE Enschede, Netherlands
> >> Email: m.vankeulen at utwente.nl, Phone: +31 534893688, Fax: +31 534892927
> >> Room: ZI 3039, WWW: http://www.cs.utwente.nl/~keulen
> >>     
> >
> >   
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> Monetdb-developers mailing list
> Monetdb-developers at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/monetdb-developers

More information about the developers-list mailing list