December 2005 - developers-list

[Monetdb-developers] CVS sql tree
by Fabian Groffen 23 Dec '05

23 Dec '05

Hi all, Unfortunately I seem to have broken SF's CVS system somehow. I tried to approve a test, but got in a 'hang' situation, which I killed after an hour of waiting. This didn't release some lockfile on the server, hence: | cvs diff: [17:27:03] waiting for mr-meltdown's lock in | /cvsroot/monetdb/sql/src/test/BugTracker/Tests Regardless how long I wait, it keeps spitting out this message. I made sure no connections are open with the SourceForge CVS server from my side (e.g. ps ax | grep sf.net), so I guess somehow I managed to melt down the system there causing a stale lockfile. If anyone knows how to resolve it, please speak up or fix it. I'm affraid one of the regular SF downtimes will not solve this stale lock thing.

2 3

[Monetdb-developers] what should happen when appending to a nil bat?
by Sjoerd Mullender 21 Dec '05

21 Dec '05

Consider this scenario: mil>var a := new(oid,void).insert(0@0,nil); mil>a.print(); #-----------------# # h t # name # oid void # type #-----------------# [ 0@0, nil ] mil>var b := new(oid,oid).insert(0@0,1@0); mil>b.print(); #-----------------# # h t # name # oid oid # type #-----------------# [ 0@0, 1@0 ] mil>a.append(b); mil>a.print(); The question is, what should the result be? As I see it, there are two possibilities: #-----------------# # h t # name # oid void # type #-----------------# [ 0@0, nil ] [ 1@0, nil ] or #-----------------# # h t # name # oid oid # type #-----------------# [ 0@0, nil ] [ 1@0, 1@0 ] The former keeps the nil tail (the tail of a was void without seqbase and stays that way), whereas the latter materializes the tail so that the tail value of b gets inserted. Opinions? -- Sjoerd Mullender

6 7

[Monetdb-developers] 2nd MonetDB-Bug-Day: Wed, Nov 9 2005 (next week)
by Stefan Manegold 20 Dec '05

20 Dec '05

(some of you might already have see this one coming; cf. http://www.cwi.nl/htbin/cwwwi/agenda?datum=2005_11+dag=09 ;-)) Dear all, given the success of the 1st MonetDB-Bug-Day on Oct 06 2005, 8 participants managed to check 347 of the then 721 bug reports, added 97 new test scripts, re-opened some bugs that still/again fail and filed 9 new bug reports; see https://sourceforge.net/mailarchive/forum.php?thread_id=8403629&forum_id=39… for details, there will by a 2nd MonetDB-Bug-Day to (hopefully) check the remaining 347 bug reports (plus those filed since Oct 06 2005), and produce proper test scripts for our automatic Nightly Multi-Platform Regression Test System (aka "TestWeb"; cf. http://monetdb.cwi.nl/Development/TestWeb/): When: Wednesday, Nov 9, 2005, "all day" (or as much time as you can spent) Where: Mainly at CWI, but you're also welcome to join "virtually" working from home or any other place in the world; for those who (again) want to turn the Bug-Day into a Bug-"Party", I reserved our meeting room (C0,01) from 8:00 - 23:00 --- you need to bring your own "equipment" though ... Who: Primarily everyone who is in touch with MonetDB, and/or one of its companions/front ends (SQL, XQuery, etc.), has rights to check-in to the MonetDB CVS repository on SourceForge, and is reasonably familiar with "Mtest.py" (cf. http://monetdb.cwi.nl/monet/src/testing/README). What: I will prepare a list of all bugs that have no test script, yet, and chop this in chunks, and assign each chunk to one participant so that [s]he can check them, and prepare the respective test scripts. More detailed instructions can be found at https://sourceforge.net/mailarchive/forum.php?thread_id=8403629&forum_id=39… and/or will follow before next Wednesday. Note: Though any help is appreciated, no one is "obliged" to help. Especially, there is no need to spent the whole day (though nine will complain if you do ;-)), anyone can join-in and leave whenever [s]he wants. For those of you participating at CWI, there will be free snacks and soft-drinks during the day (for the time you're "active"; as long as supplies lasts). Furthermore, there will be "real" drinks at the end of the Bug-Day (for participants, only; as long as supplies lasts; (maximum) amount per person will be proportional to the number of test scripts added). I hope to see especially (though not only) participants that I haven't seen on the 1st MonetDB-Bug-Day ;-) Once again thank you very much for your cooperation! CU next Wednesday, Stefan ps: Testing & the TestWeb are not (only?) my "toys"; it's "serious business", and helps all of you to (1) get a stable and reliable MonetDB, (2) monitor when your own bug reports get fixed, and (3) prevent the bugs you once thankfully detected, reported and/or fixed from occurring again! -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

8 53

[Monetdb-developers] MONET-PHP
by Goker Burak Cetin 19 Dec '05

19 Dec '05

Dear All, I am planning to use monetdb for my thesis. But I have a problem with PHP module. I'll be glad if somebody supply me a solution. I'm using gentoo... And compiling monetdb on that distribution produces debug version of PHP module. Because my PHP compiled with "no-debug" ... those two do not want to work together :) I even changed ebuild script and added "--no-debug" string. But could't be able to get no-debug version of PHP mapi client module. And I couldn't find anywhere that make.standalone script that told in README file.. Would anybody advise please ... Thanks in Advance. And Regards. -- -------------------------------------------------------- Goker Burak Cetin System Administration Istanbul Technical University Center for Satellite Communications & Remote Sensing 34469 Maslak Istanbul TURKEY TEL: +90 212 285 68 13 FAX: +90 212 285 71 67 http://www.cscrs.itu.edu.tr/goker goker(a)cscrs.itu.edu.tr --------------------------------------------------------

2 1

[Monetdb-developers] JDBC/MonetDB performance test
by Fabian Groffen 18 Dec '05

18 Dec '05

Inspired by a student's preliminary test program, I conducted a -- at first quite innocent -- experiment to see how 'fast' MonetDB is and it's upcoming major release; MonetDB/Five. The conducted experiment mainly consists of inserting 8000, 19200 and 152800 tuples into three tables respectively. There are no constraints in use, as such it is quite 'dumb', but aggressive inserting. This inserting of values was done in three different ways: 1) using plain INSERT INTO statements (DBTestI) 2) using INSERT INTO statements batched by the JDBC driver (DBTest) [1] 3) using PreparedStatements to insert the date (DBTestP) I timed the wall-clock times of PostgreSQL 8.0.3 (PG8), MySQL 4.0.25 with InnoDB (My4), MonetDB 4.8.2 (M4) and todays CVS version of MonetDB/Five (M5). The JDBC drivers used for PostgreSQL and MySQL were the latest stable versions available, for MonetDB the released version and a progressive/experimental version were tested. This is marked with an 's' for the stable released version and 'p' for the progressive version. The databases run on a Dual Athlon MP 1600+ with 1GB of main-memory running Gentoo Linux with a 2.6.12 kernel (pegatoo). Two series of tests were performed; a local and remote version. In the local variant both the client application (with JDBC driver) and database server run on the same machine (as mentioned before). In the remote variant the client is on a different machine that is hardware wise equal to the machine that runs the database (crux). Both machines are directly connected to each other. The results follow in the table below: PG8 My4 M4s M4p M5s M5p JDBC local - DBTestI 37s 27s 90s 86s 62s 60s - DBTest 19s 27s 60s 59s 46s 46s - DBTestP 14s 27s 63s 64s 59s 60s JDBC remote - DBTestI 49s 37s 84s 85s 62s 62s - DBTest 19s 37s 57s 58s 43s 43s - DBTestP 14s 37s 66s 62s 53s 53s JDBC remote [2] - DBTestI 197s 168s >1201s >1545s - DBTest 67s 168s 76s 77s 61s 63s - DBTestP 68s 168s 75s 77s 66s 70s The results are somewhat 'confusing'. [1] Note: the program was not called DBTestB because it was submitted by the students under this name. [2] To simulate a slow link, we used an ssh tunnel to the database server as connection link, instead of a direct connection

2 4

[Monetdb-developers] JDBC liberation: use ant instead of make
by Fabian Groffen 12 Dec '05

12 Dec '05

Hi list, Tonight's testweb reveals that not a single JDBC test has run, due to incorrect packaging of the driver. I have always disliked the was the driver was built and packaged, but I think this time again, it is proven that `make` is too ancient for Java, or at least too clumpsy to deal with it. This is not only my personal opinion. That is, long ago other people already found out that make doesn't work that well with Java environments[1]. Although I understand the need to keep it 'simple'[2], I personally feel that fixing the current setup again to make it work is just another hack. From the very start of JDBC development, I used `ant`[1] to build the driver. Ant is, unlike make, a build tool geared towards Java projects, and the ant build scripts[3] that are currently in CVS have a much better understanding of the various files than make does. Just to name a few: - ant has multiple build targets for JDBC, XML:DB, MCL, etc. - ant date stamps each build - ant generates documentation when required - ant understands Java - ant builds jar files much more flexible. I am strongly in favour of using ant in the future to build Java related files. That is, let configure detect if ant is available or not, when Java is available. If not, don't build, or complain. I think most of our test platforms (especially since Sun and IRIX are RIP) have ant available (even Darwin has it!), as such availability of the build tools shouldn't be an issue. On a side note, testing Java on many platforms isn't that interesting, since the JVM itself has been tested on all platforms Sun supports. For JDBC, currently only endianness, timezone and charset are important. Since Darwin is both different-endian as well as in another timezone and using another charset by default, it is the only machine that really is interesting from a testing perspective, next to a windows machine and a linux machine in our neigbourhood. [1] http://ant.apache.org/ [2] that is, using tools known to those who deal with it [3] ant's build scripts are XML files called build.xml

2 2

[Monetdb-developers] RFC: New Integer Atoms: byte & word
by Stefan Manegold 08 Dec '05

08 Dec '05

Dear MonetDB developers, I'm almost done with the first steps of adding two new integer atoms to MonetDB: a) a tiny (8-bit) integer. b) a machine-word sized integer that grows with the system, i.e., 32-bit on 32-bit systems and 64-bit on 64-bit systems. (a) is supposed to replace "chr" wherever "chr" is not used as a character type, but as a (tiny) 1-byte integer. (b) is supposed to be the MIL pendant of the C type "var_t" and can/should be used, e.g., for BUN counts of BATs which are limited to 32-bit on 32-bit systems, but can grow to 64-bit on 64-bit systems. For now, I chose the following names, sticking to the 3-letter "design", and picking combinations that do not yet exist in the code base: a) "bte" (read "byte") b) "wrd" (read "word") In a first checking (hopefully sometime next week), I will only add these new atoms (and add the proper functionality & new signatures where necessary), but not change any existing MIL proc/command signatures. In a second check-in (maybe also already next week?), I plan to change (at least) the signature(s) of "count(BAT[any,any]) : int" & "count(int) : lng" to a single "count(BAT[any,any]) : wrd". Maybe, I/we should later also consider to replace "chr" by "bte" in the enum module. Well, so far so good. I would be grateful, if you could comment on these plans. Especially, I'd like to hear, whether there are better(?) suggestions for the names "bte" & "wrd". Further, did I miss any places, where we should/must use "bte" iso. "chr" and/or "wrd" iso. "int"/"lng"? Thank you very much in advance! Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

7 8

[Monetdb-developers] DgF LS- L 0 LITA NOW is out !!! with 4 in1 q5Y
by Best Offer 08 Dec '05

08 Dec '05

Hello Monetdb-developers, At this time we can offer a small update at our system - LS-L0 LITA and LITTLE CUTIES! Studio and our little stars are proud to present their new project. You can now compare this one with our other sites, judge the level of professionalism and exposure of subject, with have never been so high! Starign from over 4,000 HQ pics, the project features sets made in the studio, as well as on the side of nature. http://carsick.offer4in1.com/6/?lining Our great and unique offer: for each subscribtion You get access to another three sites from our portal for 31 days... without any additional payments! Simply subscribe, select and use! MSGID: 0AzYTtO3nHUzOQMaccabeusRvnmvY6El7MHuH

1 0

[Monetdb-developers] Re: PFTIJAH CVS branch for Pathfinder
by Stefan Manegold 06 Dec '05

06 Dec '05

Jan, Henning, according to the TestWeb, there seem to be quite some compilation problems with the lastest version of Pathfinder; hence, CVS HEAD version of Friday Dec 02 2005 (now tagged as "XQuery_TIJAH_root") to create a branch called "XQuery_TIJAH". Please let me/us know, in case there are any problems and/or once you want/need to start propagating changes from the main trunk to the XQuery_TIJAH branch. Also let me/us know, one you want/need to start automatic nightly testing of the XQuery_TIJAH branch. Regards, Stefan ps: I cc´ed the MonetDB-developers list, since more people (i.e., those maintaining and/or developing for any module in the MonetDB CVS repository on SF) might be interested in what's going on with the Pathfinder CVS repository; and monetdb-developers(a)lists.sourceforge.net is the proper communication channel that should be used more actively... On Tue, Dec 06, 2005 at 11:59:57AM +0100, Jan Flokstra wrote: > Hi Stefan, > > Thanks for your response. Our answers to your questions are: > > (1) We could have a branch from the current HEAD. It looks this works properly > at this moment. And we would indeed like it A.S.A.P. beacause Jan is goiing > to Mauritania for 3 weeks december 17th and he would like to have the > software ready for Hennig to experiment. > > (2) We are not planning to propagate from the pftijah branch to the main > branch. I think we could use the existing scripts to propagate the changes > from the main branch to the pftijah branch. The only real overlap are a > couple of command declarations in "pathfinder.mx", all other stuff will be in > new .mx files. In the end we would like to change the serialization.mx stuff > because this will be used extensively. But we can change it in the main trunk > and propagate it to 'pftijah' > > (3) Jan F? We could create the branch and worry about the other stuff when he > returns 10 januari! > > Regards, Jan & Hennig. > > > On Tuesday 06 December 2005 11:15, Stefan Manegold wrote: > > Hi Jan, Hi Henning, > > > > first of all, sorry for the late reply --- I had been quite busy recently > > --- only now (on short holidays in Germany ;-)) I found some time to write > > a proper reply ... > > > > Well, in principle creating a branch for 'pftijah' is no problem at all. > > However, given the experiences I/we have with especially the update branch, > > we should make very clear, (1) when/of which version of the main trunk the > > branch is created, (2) how/when to propagate changes from the main trunk to > > the pftijah branch (preferably not from the pftijah branch to the main > > trunk, i.e. ALL BUG fixes that also apply to the main trunk MUST be checked > > in to the main branch (as well)), and (3) who is responsible for > > maintaining the pftijah branch, propagating changes/bug-fixes from the main > > trunk to the pftijah branch and testing(!) the pftijah branch. > > > > About (1): > > I guess, you prefer ASAP, hence waiting for the release that we plan for > > end of January is no option, right? > > This in turn means that all changes of the update branch, which will be > > merged into the main trunk before the release (I guess just after new > > year), then also have to be merged into the pftijah branch... > > > > About (2) & (3): > > Well, I have some experiences and script for the propagation which I'm more > > than willing to share with who ever becomes the maintainer of the pftijah > > branch --- surely not me ;-) > > > > Let me know, what you think about this ... > > > > Greetings from --- well, doesn't matter ;-) > > > > Stefan > > > > On Fri, Dec 02, 2005 at 09:03:27AM +0100, Jan Flokstra wrote: > > > Hi Stefan, > > > > > > I'm going to start working on the possible integration of Tijah and > > > Pathfinder. The idea at this moment is to create a Tijah index on > > > Pathfinder documents. Then we will implement a nexi() function in > > > Pathfinder which will do the query on this Tijah index and will return > > > Pathfinder nodes which can be processed by Pathfinder. > > > I tried to set up a separate module for this called 'pftijah' but this > > > failed beacause at this moment I need to be very close to Pathfinder. > > > There will be quite a lot of people on this problem so we need a CVS > > > repository. So we would like to have a special Pathfinder branch called > > > 'PFTIJAH' from where we could do our experiments and distribute them. > > > This branch could be a branch from the current HEAD or from an older > > > stable version, whatever you consider the best solution. > > > Is it possible that you or somebody else creates such a branch for us??? > > > > > > Regards, Jan Flokstra & Hennig Rode. > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

[Monetdb-developers] Re: PFTIJAH CVS branch for Pathfinder
by Stefan Manegold 06 Dec '05

06 Dec '05

Hi Jan, Hi Henning, first of all, sorry for the late reply --- I had been quite busy recently --- only now (on short holidays in Germany ;-)) I found some time to write a proper reply ... Well, in principle creating a branch for 'pftijah' is no problem at all. However, given the experiences I/we have with especially the update branch, we should make very clear, (1) when/of which version of the main trunk the branch is created, (2) how/when to propagate changes from the main trunk to the pftijah branch (preferably not from the pftijah branch to the main trunk, i.e. ALL BUG fixes that also apply to the main trunk MUST be checked in to the main branch (as well)), and (3) who is responsible for maintaining the pftijah branch, propagating changes/bug-fixes from the main trunk to the pftijah branch and testing(!) the pftijah branch. About (1): I guess, you prefer ASAP, hence waiting for the release that we plan for end of January is no option, right? This in turn means that all changes of the update branch, which will be merged into the main trunk before the release (I guess just after new year), then also have to be merged into the pftijah branch... About (2) & (3): Well, I have some experiences and script for the propagation which I'm more than willing to share with who ever becomes the maintainer of the pftijah branch --- surely not me ;-) Let me know, what you think about this ... Greetings from --- well, doesn't matter ;-) Stefan On Fri, Dec 02, 2005 at 09:03:27AM +0100, Jan Flokstra wrote: > Hi Stefan, > > I'm going to start working on the possible integration of Tijah and > Pathfinder. The idea at this moment is to create a Tijah index on Pathfinder > documents. Then we will implement a nexi() function in Pathfinder which will > do the query on this Tijah index and will return Pathfinder nodes which can > be processed by Pathfinder. > I tried to set up a separate module for this called 'pftijah' but this failed > beacause at this moment I need to be very close to Pathfinder. There will be > quite a lot of people on this problem so we need a CVS repository. So we > would like to have a special Pathfinder branch called 'PFTIJAH' from where we > could do our experiments and distribute them. This branch could be a branch > from the current HEAD or from an older stable version, whatever you consider > the best solution. > Is it possible that you or somebody else creates such a branch for us??? > > Regards, Jan Flokstra & Hennig Rode. -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0