September 2006 - users-list

Re: [MonetDB-users] Shredding DBLP XML document
by Stefan Manegold 22 Sep '06

22 Sep '06

Hi Erwin, shredding dblp.xml works fine for me with MonetDB/XQuery 0.13.1 (latest CVS version) on an Althon64 X2 with 2 GB RAM under Fedora Core 4: ======== # Monet Database Server V4.13.1 # Copyright (c) 1993-2006, CWI. All rights reserved. # Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs; dynamically linked. # Visit http://monetdb.cwi.nl/ for further information. MonetDB>shred_doc("/tmp/dblp.xml","dblp.xml"); # Elapsed time = 02m 05s 557ms 075us [005us/node] # Shredded 1 XML documents, total time after commit=133.708s ======== Hence, could you please be more verbose about your setup? Which version of MonetDB/XQuery are you using? What kind of "PC" are you using (operating system, hardware)? You say "the shredding process was not finished". Does this mean that MonetDB/XQuery was still busy (with shredding)? Stefan ps: I felt free to cc this to the MonetDB-users lists. On Thu, Sep 21, 2006 at 02:02:05PM +0800, Erwin Leonardi wrote: > Hi all, > > I tried to shred DBLP XML document (around 300MB), but I found the following > situations. I left my PC over the weekend, and the shredding process was not > finish and there is no error message. I tried to check my free disk space, > and I found out that the free disk space remains the same. That is, it seems > that MonetDB has not written any data to disk. Do you why this happened? Can > you try to shred DBLP XML document ? > > Thanks > Erwin -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

3 6

Re: [MonetDB-users] Comparing TPC-H benchmark results?
by moredata＠fastmail.net 21 Sep '06

21 Sep '06

Hello, I too would be interested in TPC-H comparisons of MonetDB versus other open source databases like MySQL and PostgreSQL, in addition to commercial databases like Oracle. If anyone has done such comparisons, please let me know. Martin also mentioned that the plans are not currently optimized but that it is targeted for V5. Are any of these optimizations already in the V5 CVS code (correlated subqueries?)? Thanks in advance. > -----Original Message----- > From: monetdb-users-bounces(a)lists.sourceforge.net > [mailto:monetdb-users-bounces@lists.sourceforge.net] On > Behalf Of Martin Kersten > Sent: Saturday, September 16, 2006 7:39 AM > To: monetdb-users(a)lists.sourceforge.net > Subject: Re: [MonetDB-users] Comparing TPC-H benchmark results? > > > If I a remember correctly the scale factor is 0.01 in the > testweb. Beware that some queries in TPC-H require a lot of > intermediate space and the plans produced for MIL are not > optimized. (It is the target for the MonetDB V5 release) > > Actually, we would be interested in experiences in the > upcoming TPC-DS, which is more modern and not well explored > by any system. > > Fabian Groffen wrote: > > Hi, > > > > No need to resubmit your question. It was received, but I'm sorry > > that we've been too busy to answer this question. > > > > On 15-09-2006 21:25:15 +0200, Wouter Scherphof wrote: > >> Hi, > >> > >> The TPC-H benchmark (http://www.tpc.org/tpch/default.asp) > is part of > >> the MonetDB SQL source and the benchmark is run during > weekends, as I > >> can see on the TestWeb > >> > (http://monetdb.cwi.nl/testing/projects/monetdb/Current/sql/.mTests03 > >> /times.html). > >> How > >> should I interpret the listed results? According to TPC, > results should be > >> stated as a "QphH@Size" value, where "QphH" is the number > of queries per > >> hour for the H benchmark, and "Size" is a "scaling > factor". On the TestWeb, > >> the results are stated as seconds of execution time. > > > > The testweb tpc-h is very very very small, to keep both > testweb small > > in diskspace, and to keep the test time reasonable on our slower > > testing machines. > > > >> Is it correct to compute the QphH value as follows: > >> Queries 01-22 took 1.985 seconds, so QphH is (3600 / 1.985) * 22 = > >> 39.899 ? > > > > Well, the testweb framework does way more than just executing the > > test, so the performance time of the test might not be accurate. > > Also, you probably want to test with a higher workload, to > get a more > > accurate result. > > > >> What scaling factor is used in the benchmark runs; is this > 1? Is it > >> possible to let the TestWeb run with scaling factor 100? Because > >> that's the smallest Size for which results are published > on the TPC > >> web site. > > > > It's very small. I don't recall how small, but very very small. > > > >> Did anyone yet try to determine the Price of the test platforms, > >> following the TPC Pricing Spec? TPC-H results normally include a > >> Price/Performance metric in the form of $/QphH@Size, and I guess > >> MonetDB could score particularly well on this metric. > > > > We never really did this. But, feel free to try and setup > such test, > > we are very interested in the outcomes, of course. > > > > > > I hope this answers your questions. > > > > Regards > > > > > ---------------------------------------------------------------------- > > --- > > Using Tomcat but need to do more? Need to support web > services, security? > > Get stuff done quickly with pre-integrated technology to > make your job easier > > Download IBM WebSphere Application Server v.1.0.1 based on > Apache Geronimo > > > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057& dat=121642 > _______________________________________________ > MonetDB-users mailing list > MonetDB-users(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-users -- moredata(a)fastmail.net -- http://www.fastmail.fm - IMAP accessible web-mail

1 0

[MonetDB-users] schema reconstruction
by Ronald Oussoren 19 Sep '06

19 Sep '06

Hi, I'm trying to build a driver for monetdb for sqlalchemy (a python ORM layer). One of the things I need to do for that is provide a way to reconstruct the (some of) database schema on the fly. It seems to be easily possible to reconstruct most of the table definition from the 'table' and 'columns' tables, but how do I retrieve key contraints? Specifically, given this table definition: create table test ( id integer primairy key, name varchar(10) ) , I get some data in the keycolums table, but don't know how to relate that to the tables and colums tables. I also need something simular for foreign keys, but haven't looked into that yet. Ronald P.S. I'm using the head of the CVS trunk.

4 3

[MonetDB-users] Comparing TPC-H benchmark results?
by Wouter Scherphof 16 Sep '06

16 Sep '06

Hi, The TPC-H benchmark (http://www.tpc.org/tpch/default.asp) is part of the MonetDB SQL source and the benchmark is run during weekends, as I can see on the TestWeb ( http://monetdb.cwi.nl/testing/projects/monetdb/Current/sql/.mTests03/times.…). How should I interpret the listed results? According to TPC, results should be stated as a "QphH@Size" value, where "QphH" is the number of queries per hour for the H benchmark, and "Size" is a "scaling factor". On the TestWeb, the results are stated as seconds of execution time. Is it correct to compute the QphH value as follows: Queries 01-22 took 1.985 seconds, so QphH is (3600 / 1.985) * 22 = 39.899 ? What scaling factor is used in the benchmark runs; is this 1? Is it possible to let the TestWeb run with scaling factor 100? Because that's the smallest Size for which results are published on the TPC web site. Did anyone yet try to determine the Price of the test platforms, following the TPC Pricing Spec? TPC-H results normally include a Price/Performance metric in the form of $/QphH@Size, and I guess MonetDB could score particularly well on this metric. Regards, Wouter Scherphof. Utrecht Professional School, The Netherlands.

3 3

[MonetDB-users] packages for YUM (fedora 5) and PORT (os x intel)?
by Shahbaz 15 Sep '06

15 Sep '06

Hi, I would like to use Monet DB, however the RPMs provided on your webpage don't seem to work for Fedora Core 5. Despite having these libraries installed, rpm -Uvh and rpm -ivh tell me that libMapi, libalgebra, lib_arith, etc. are needed by MonetDB-QL-server-2.12.0-1.x86_64.rpm. (I have AMD 64). Could someone please provide a yum repository so users can' install monetdb just by doing the following: sudo yum install monetdb I don't have an immediate need for a darwin-ports distribution, but it will be very helpful for those who would like to try monetdb on osx. Thanks

2 1

Re: [MonetDB-users] Fedora core 5 64 bit horribly slow compilation
by Fabian Groffen 11 Sep '06

11 Sep '06

On 11-09-2006 18:03:17 +0200, Tomas Brambora wrote: > Hello, Mr. Grofen, Hi Tomas, > I would like to ask whether there have been attempts to compile MonetDB on AMD Athlon 64 mobile achitecture OS Fedora Core 5 (64 bit version). > I ask because I was trying to compile it on my fresh installation of this OS and it just gets stuck on the file "gdk_scanselect.c". Or perhaps it doesn't get stuck, but I let it compile this file for 20 minutes and it didn't make any progress (it keeps on working with the harddisc though), then I lost patience. I know this file is BIG, but anyway, 20 minutes... On mobile chips we haven't tested, but on our own systems with athlon64's and Fedora Core 5 it takes a while to compile this file, especially with full optimisations on. 20 minutes sounds a bit long though. What's your CPU speed? And what is the active CPU speed when compiling MonetDB? I cc monetdb-users list, as maybe someone else has more experience with compiling on your CPU. Regards

2 1