Inspired by a student's preliminary test program, I conducted a -- at
first quite innocent -- experiment to see how 'fast' MonetDB is and it's
upcoming major release; MonetDB/Five.
The conducted experiment mainly consists of inserting 8000, 19200 and
152800 tuples into three tables respectively. There are no constraints
in use, as such it is quite 'dumb', but aggressive inserting. This
inserting of values was done in three different ways:
1) using plain INSERT INTO statements (DBTestI)
2) using INSERT INTO statements batched by the JDBC driver (DBTest) [1]
3) using PreparedStatements to insert the date (DBTestP)
I timed the wall-clock times of PostgreSQL 8.0.3 (PG8), MySQL 4.0.25
with InnoDB (My4), MonetDB 4.8.2 (M4) and todays CVS version of
MonetDB/Five (M5). The JDBC drivers used for PostgreSQL and MySQL were
the latest stable versions available, for MonetDB the released version
and a progressive/experimental version were tested. This is marked with
an 's' for the stable released version and 'p' for the progressive version.
The databases run on a Dual Athlon MP 1600+ with 1GB of main-memory
running Gentoo Linux with a 2.6.12 kernel (pegatoo). Two series of
tests were performed; a local and remote version. In the local variant
both the client application (with JDBC driver) and database server run
on the same machine (as mentioned before). In the remote variant the
client is on a different machine that is hardware wise equal to the
machine that runs the database (crux). Both machines are directly
connected to each other.
The results follow in the table below:
PG8 My4 M4s M4p M5s M5p
JDBC local
- DBTestI 37s 27s 90s 86s 62s 60s
- DBTest 19s 27s 60s 59s 46s 46s
- DBTestP 14s 27s 63s 64s 59s 60s
JDBC remote
- DBTestI 49s 37s 84s 85s 62s 62s
- DBTest 19s 37s 57s 58s 43s 43s
- DBTestP 14s 37s 66s 62s 53s 53s
JDBC remote [2]
- DBTestI 197s 168s >1201s >1545s
- DBTest 67s 168s 76s 77s 61s 63s
- DBTestP 68s 168s 75s 77s 66s 70s
The results are somewhat 'confusing'.
[1] Note: the program was not called DBTestB because it was submitted by
the students under this name.
[2] To simulate a slow link, we used an ssh tunnel to the database
server as connection link, instead of a direct connection
Hi Fabian,
I just got my new workstation and it is an AMD64 machine with windows64
installed.
I also got myself both the microsoft 64 bits compiler as well as the intel
64 bits compiler. With the particular goal of course to port MonetDB.
I am not promising any timeframe but if we don't have a native windows64 port
in the next couple of months I would be very surprised if not disappointed.
Peter
>>
>>
>>
>> Just a small note. I don't have any windows building experiences, but
>> we currently don't have access to such box running that operating
>> system. As far as I can tell, there is no real priority to do so
>> either. There are build instructions for windows, but it is unknown how
>> 64-bits are enabled and if this will compile and run. This seems an
>> unexplored area to me.
>>
>> Regards,
>> Fabian
>>
>>
>> Cardillo Raymond A Civ AFRL/IFEA wrote:
>> > Has anyone out there taken the time to build an MSI for the AMD64 versions
>> > of Windows? The x64 version of Windows was released a few months ago and
>> > was available in Beta for a while, so I'm hoping someone has either done
>> > this already, or can provide some advice on how to build/compile for this
>> > platform. Anyone know of future plans to make this one of the standard
>> > platform builds? Windows x64 is only going to grow in popularity from here,
>> > especially amongst researchers and others who can benefit from 64 bits.
>> >
>> > Thanks for any info,
>> > Ray Cardillo
>>
>>
>>
>>
>>
I'd like to add a more perlish layer around MapiLib (working title:
MonetDB::CLI). This may look like:
my $Cxn = MonetDB::CLI->connect( ... );
my $Req = $Cxn->query('select * from env');
while ( my $Cnt = $Req->fetch_row ) {
print $Req->fetch_field( $_ ) for 0 .. $Cnt-1;
}
Still much in the spirit of Mapi, but handles are now objects (with
destructors), functions are methods and the methods can throw exceptions.
Once we have such an interface, additional implementations are possible.
For instance, a XS module could skip the MapiLib layer and use libMapi
directly. Later on, an implementation for MCL will be added (I hope) -
either in pure perl or on top of libMCL (in case such a library will
emerge).
Of course, I'd like to use this new interface from DBD::monetdb to be
able to choose between various implementations and to ease the
transition to MCL.
On the downside: additional modules means additional installation efforts.
Especially the people resposible for the RPM's will be affected.
If this sounds too screwy, it's now the chance to stop me ;-)
Steffen
Hi all,
for our ongoing course "Database-Supported XML Processors" I wanted to
install MonetDB/XQuery in our Sun-based computer lab. I encountered a
number of pitfalls during the install that I will list below. System is
SunOS 5.9. There's a bunch of GNU tools installed (e.g., gcc), but a
number of tools is only available in their original Sun version. I've
used the 4.8.2/0.8.2 tarballs distributed on the SourceForge website and
set `--enable-optimize'.
1. Python
=========
MonetDB's ./configure complains about a wrong version of Python and
aborts. If I give it `--without-python', however, ./configure no longer
complains, and MonetDB compiles fine (apparently).
2. make
=======
Default `make' on that system is the one provided by SunOS. Apparently
this is *not* sufficient to compile MonetDB:
-- Sun's `make' does not like the wildcards (*.mk) in Makefile
includes. I'm explicitly mentioning that because that actually
annoys me for quite some time also on my system (independent of the
Sun make issue). The Makefiles include $(top_srcdir)/*.mk and
$(top_builddir)/*.mk. If I build MonetDB (or Pathfinder) directly
in the source directory, this leads to any *.mk file being included
twice---with loads of `make' warnings. Wouldn't it be more
consistent to explicitly list .mk files to be included here with
their full name? (Btw. with those wildcards, compilation will
inevitably fail if I drop some random .mk file in any of the two
directories---a thing that a normal user probably wouldn't
expect/understand.)
-- Lots of other stuff is not understood by Sun's `make'.
Shouldn't ./configure also check whether the `make' implementation on a
system suffices for the project?
gmake did the job for me. It was available on our Sun, fortunately.
3. libbz2
=========
There's no libbz2 on that system. ./configure still did not complain.
Compilation, however failed. Adding a `--without-bz2' fixed the issue.
4. Compilation issue
====================
gcc -DHAVE_CONFIG_H -I. -I../../../src/gdk -I../.. -I../../../src/gdk -I../common -DLIBGDK -O2 -std=c99 -D__EXTENSIONS__ -Wall -W -O2 -fomit-frame-pointer -finline-functions -DHWCOUNTERS -DHW_SunOS -DHW_sun4u -Werror-implicit-function-declaration -Werror -Wno-unused-function -Wno-unused-label -Wno-uninitialized -D_REENTRANT -c gdk_posix.c -fPIC -DPIC -o .libs/libbat_la-gdk_posix.o
cc1: warnings being treated as errors
../../../src/gdk/gdk_posix.mx: In function `MT_getrss':
../../../src/gdk/gdk_posix.mx:550: warning: int format, pid_t arg (arg 1)
gmake[4]: *** [libbat_la-gdk_posix.lo] Error 1
gmake[4]: Leaving directory `/u/halle/teubnerj/home_sun/MonetDB/src/MonetDB-4.8.2/build/src/gdk'
I've removed the -Werror to get past that error.
5. HwCounters
=============
./configure complained about libcpc.h: present but cannot be compiled.
Seems like the libcpc.h is there for some other reason. Compiling
failed with an error in ..counters.mx. I've thus set
`--without-hwcounters'.
6. Parallel Make
================
Tired of long make runs, I've tried `gmake -j4. Result:
[...lots of stuff...]
../../../src/mel/ops.mx: ./ops.h - created
../../../src/mel/symtable.mx: ./symtable.h - created
gmake[3]: *** No rule to make target `atom.mx', needed by `atom.h'. Stop.
gmake[3]: *** Waiting for unfinished jobs....
../../../src/mel/dependency.mx: ./dependency.h - created
gmake[3]: Leaving directory `/u/halle/teubnerj/home_sun/MonetDB/src/MonetDB-4.8.2/build/src/mel'
gmake[2]: *** [all-recursive] Error 1
Missing dependencies? Starting gmake again did it for me.
Oops! Another one:
/u/halle/teubnerj/home_sun/MonetDB/src/MonetDB-4.8.2/build/src/utils/Mx/Mx -x y ../../../src/monet/monet_parse.mx
../../../src/monet/monet_parse.mx: ./monet_parse.h - created
gmake[3]: *** No rule to make target `../gdk/../common/stream.mx', needed by `../gdk/../common/stream.h'. Stop.
gmake[3]: *** Waiting for unfinished jobs....
../../../src/monet/monet_parse.mx: ./monet_parse.c - created
Another one:
/u/halle/teubnerj/home_sun/MonetDB/src/MonetDB-4.8.2/build/src/utils/Mx/Mx -x sh ../../../../src/modules/contrib/ddbench.mx
gmake[4]: stat:../../../../src/modules/contrib/bat_mmath.mx: Interrupted system call
gmake[4]: *** No rule to make target `bat_mmath.mx', needed by `bat_mmath.m'. Stop.
gmake[4]: *** Waiting for unfinished jobs....
../../../../src/modules/contrib/ddbench.mx: ./ddbench - created
../../../../src/modules/contrib/ddbench.mx: ./ddbench.m - created
Another one:
/u/halle/teubnerj/home_sun/MonetDB/src/MonetDB-4.8.2/build/src/mel/mel -I../../../../src/modules/contrib -I../../common -I../../gdk -I../../monet -I../plain -glue oo7.m > oo7.glue.c
gmake[4]: *** No rule to make target `uchr.mx', needed by `uchr.m'. Stop.
gmake[4]: *** Waiting for unfinished jobs....
gmake[4]: Leaving directory `/u/halle/teubnerj/home_sun/MonetDB/src/MonetDB-4.8.2/build/src/modules/contrib'
(I know that parallel build have their problems. I've listed the errors
here anyway.)
7. Linker Error
===============
Finally I hit this linker error that I don't know how to solve:
gmake[4]: Entering directory `/u/halle/teubnerj/home_sun/MonetDB/src/MonetDB-4.8.2/build/src/tools'
/bin/bash ../../libtool --tag=CC --mode=link gcc -O2 -std=c99 -D__EXTENSIONS__ -Wall -W -O2 -fomit-frame-pointer -finline-functions -Werror-implicit-function-declaration -Werror -Wno-unused-function -Wno-unused-label -Wno-uninitialized -D_REENTRANT -o Mserver Mserver.o initmodules.o ../monet/libmonet.la ../modules/plain/lib_algebra.la ../modules/plain/lib_arith.la ../modules/plain/lib_bat.la ../modules/plain/lib_builtin.la ../modules/plain/lib_str.la ../modules/plain/lib_sys.la ../modules/plain/lib_trans.la ../gdk/libbat.la -lsocket -lnsl -ldl -lpthread -lposix4 -lmalloc ../common/libmutils.la ../common/libstream.la
gcc -O2 -std=c99 -D__EXTENSIONS__ -Wall -W -O2 -fomit-frame-pointer -finline-functions -Werror-implicit-function-declaration -Werror -Wno-unused-function -Wno-unused-label -Wno-uninitialized -D_REENTRANT -o .libs/Mserver Mserver.o initmodules.o ../monet/.libs/libmonet.so ../modules/plain/.libs/lib_algebra.so ../modules/plain/.libs/lib_arith.so ../modules/plain/.libs/lib_bat.so ../modules/plain/.libs/lib_builtin.so ../modules/plain/.libs/lib_str.so -liconv ../modules/plain/.libs/lib_sys.so ../modules/plain/.libs/lib_trans.so /u/halle/teubnerj/home_sun/MonetDB/src/MonetDB-4.8.2/build/src/monet/.libs/libmonet.so /u/halle/teubnerj/home_sun/MonetDB/src/MonetDB-4.8.2/build/src/gdk/.libs/libbat.so -lm ../gdk/.libs/libbat.so /u/halle/teubnerj/home_sun/MonetDB/src/MonetDB-4.8.2/build/src/common/.libs/libstream.so -ldl -lpthread -lposix4 -lmalloc ../common/.libs/libmutils.so ../common/.libs/libstream.so /u/halle/teubnerj/home_sun/MonetDB/src/MonetDB-4.8.2/build/src/common/.libs/libmutils.so -lsocket -lnsl -lz -Wl,--rpath -Wl,/u/halle/teubnerj/home_sun/MonetDB/lib -Wl,--rpath -Wl,/u/halle/teubnerj/home_sun/MonetDB/lib/MonetDB
../modules/plain/.libs/lib_arith.so: undefined reference to `__ashrdi3(a)GCC_3.0'
../modules/plain/.libs/lib_arith.so: undefined reference to `__ashldi3(a)GCC_3.0'
../modules/plain/.libs/lib_arith.so: undefined reference to `__floatdisf(a)GCC_3.0'
../modules/plain/.libs/lib_arith.so: undefined reference to `__floatdidf(a)GCC_3.0'
collect2: ld returned 1 exit status
gmake[4]: *** [Mserver] Error 1
Any ideas so I can continue and hit more compilation problems?
I'm not an expert on that system. Actually, I don't have any clue what
software, libraries, tools, etc. is available there. I have access to
the machine at any time, though, and I'm happy to give fixes a try or
figure out, e.g., more detailed error messages.
Jens
--
Jens Teubner
Technische Universitaet Muenchen, Department of Informatics
D-85748 Garching, Germany
Tel: +49 89 289-17259 Fax: +49 89 289-17263
Real programmers don't comment their code.
If it was hard to write, why should it be easy to read?
Dear monetdb-developers(a)lists.sourceforge.net:
Bullet-Proof server:
Fresh IPs
1024MB RAM
P4 CPU
72GB SCSI
Dedicated 100M fiber
Unlimited Data Transfer
Any software
Based China
US$599.00 per month
May use the server for:
Bulk web Hosting
Direct Mail
We also supply email list according to your
order and sending out your message for you.
Hope to service for you.
Cheers!
Jone
Server Team
kezunli666(a)sohu.com
For.NoSend: MoreNot(a)yahoo.com
Dear all,
in issue 20/2005, our favorite (German ;-)) computer magazine c't
(http://www.heise.de/ct/) calls for a "database contest". The original call
and further information (in German) is available online at
http://www.heise.de/ct/05/20/156/
an English translation can be found at
http://firebird.sourceforge.net/connect/ct-dbContest.html
Since we think, it would be a pity, if MonetDB would not participate in this
contest, we decided to give it a try.
Here at CWI, Stratos is currently looking at how to port the Dell's
"DVD-Store" reference implementation (cf. http://linux.dell.com/dvdstore/)
to MonetDB. Creating the database and loading the data needed only minor
changes of the provided scripts. The next step will be to setup the
middleware implementation for the webserver. We plan to use JSP/JDBC, here.
Next to the web-application, we also plan to implement a MonetDB-specific
driver for DVD-Store's "client-simulator" to allow direct client-server
communication.
Additionally, Fabian is busy improving the client-server communication (see
his recent mails on this list).
Of course, other MonetDB-developers/-users are more than welcome to also
participate in the c't database contest effort, maybe using other middleware
languages like PHP or Perl (Steffen? ;-)).
If so, please feel free to let us know, and to ask for support (if necessary
;-)).
We will keep you posted about our progress.
Regards from A'dam,
Stefan
--
| Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl |
| CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ |
| 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 |
| The Netherlands | Fax : +31 (20) 592-4312 |
A small update on the results from yesterday. I suspected the stream
level protocol we use to be a bit inefficient, so I implemented a IMHO
more efficient version. In short I omit doing a 4-byte read and write
call to indicate a block of data is done. I now communicate this by
misusing a bit somewhere. I hoped for the result to be a bit better,
but I won't complain on MonetDB 4: a rough 30% win on the DBTestI case.
Since this change first has to be ported into MonetDB/Five those results
will be made available later. The 'new' results are marked as M4n.
The results follow in the table below:
PG8 My4 M4p M4n M5p M5n
JDBC local
- DBTestI 37s 27s 86s 62s 60s ??s
- DBTest 19s 27s 59s 48s 46s ??s
- DBTestP 14s 27s 64s 61s 60s ??s
This email is to report a bug in MonetDB V4.8.2 for windows.
My name is Rómulo Gonçalves, number 0536601.
When I tried to insert one date value into one table with this code:
Data dat = new Date();
Insert Into Statistic_URL values(dat,2,1);
and I receive this error:
parse error, expecting `','' or `')'' in: "insert into statistic_url
values(tue oct"
So I tried wiht this way:
Insert Into Statistic_URL values('dat',2,1);
I receive this error:
ERROR: INSERT INTO: NOT NULL constraint violated for column
statistic_url.date_sta
I think this means that the data is not parseable, so it becomes null.
I hope you understand the bug. If you do not understand something just send
an email to me.
Greets, Rómulo........
Dear mr/mrs,
We are experimenting with MonetDB to create a google-like search engine with
a database of some 300.000 urls.
We used the COPY INTO TEST FROM /../urlFile.txt statement to load all the
data in a table. This worked fine and very fast.
After this we tried to run the same script again, but errors occured. The
connection was lost with the (local) monetDB and when we tried to regain it
through:
/home/mjmetter/MonetDB/bin/Mserver --dbinit "module(sql_server);
sql_server_start();"
we got the following output:
!FATAL: GDKlockHome: Database lock '.gdk_lock' denied.
We think that the loss of connection is caused by the amount of data that
was inserted into the table. We did not manage to solve this problem, so
maybe you can help us.
Thanks in advance,
Michael Metternich
Nicolaas Heyning
