Hello,
I am interested in evaluating MonetDB/XQuery. I have some XML collections
for a project I work on that are of 1-2 million individual XML documents
for which I use XQuery to access for OLAP style reporting operations. This
is a Java-based project, and while working on this project I developed a
thin XQuery-centric API so that I could evaluate/use many different XQuery
implementations, which my company released as an open source project
called Xaj (http://sourceforge.net/projects/xaj). Currently I am using a
Berkeley DB XML implementation, but am interested in exploring other
options.
I'd like to implement a MonetDB version of this API now, but I am having
some difficulty finding out how to effectively access MonetDB through
Java. So far I was able to use the MonetDB JDBC driver to construct XQuery
statements and get XML results, and also I was able to construct MonetDB
pf:add-doc() statements to add documents, but I am wondering if this is
the best approach (and if so, how to add documents to a remote MonetDB
server using the pf:add-doc() mechanism).
The API I am trying to implement you can view here, to give you an idea
what I'm trying to do:
http://xaj.svn.sourceforge.net/viewvc/xaj/xaj/src/net/sf/xaj/XmlDb.java?rev…
It was somewhat modeled after the defunct XML:DB API, but focused just on
add/store/XQuery operations. Any help/advise would be greatly appreciated!
-- m@
Is there any method, besides restarting the database,
of shrinking the sql log file?
When I am performing a large set of large inserts (via
COPY INTO) the log file gets very large. I wish there
was a way to cycle the log file.
Also during these large batches of inserts the
database server becomes less and less responsive until
I am forced to restart the server. Upon restart and
the normal recovery monet returns to a very responsive state.
Hi all,
I am new to monetDB and ran into the following problem (I'll try to be as
verbose as I can, perhaps it helps helping me):
I've installed MonetDB server 5.0.0, clients and commons 1.18.2 and sql
2.18.2 on Ubuntu Feisty. Then I've taken the JDBC example which is available
online and modified it to be able to insert data into a test database. More
specifically I'm using prepared statements and executeBatch() together with
the monetdb-1.6 JDBC driver.
Which is quite successful until about 5 million records are inserted -
afterwards the speed of the inserts drops dramatically.
I've experimented with different batch sizes (10000 records seems to work
ok), and I've tried to understand the chapter about memory tuning for
MonetDB (not sure yet about how to use mem_cursize() though).
In any case at first inserting 2 million records only takes about 5 minutes
(measured with Javas System.timemillis), later on this time increases and
the server gets rather unresponsive.
I found only two posts concerning the speed of JDBC inserts (from January
2007 I think) and I would be interested if a solution or explanation has
been found off-list.
Thank you
--
View this message in context: http://www.nabble.com/JDBC-insert-performance-tf4589411.html#a13100422
Sent from the monetdb-users mailing list archive at Nabble.com.
Hello,
I installed a copy of MonetDB on CentOS 5.1, but don't seem to be able to start it. I installed the RPMs, and then wanted to follow the "Getting Started" page on your website.
[root@localhost ~]# mserver5 --dbinit="include sql;"
# MonetDB server v5.2.2, based on kernel v1.20.0
# Serving database 'demo'
# Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs dynamically linked
# Copyright (c) 1993-2007 CWI, all rights reserved
# Visit http://monetdb.cwi.nl/ for further information
#warning: please don't forget to set your vault key!
#(see /etc/monetdb5.conf)
# Listening for connection requests on mapi:monetdb://127.0.0.1:50000/
!MALException:malInclude:could not open file: sql
[root@localhost ~]# uname -a
Linux localhost.localdomain 2.6.18-53.el5xen #1 SMP Mon Nov 12 02:46:57 EST 2007 x86_64 x86_64 x86_64 GNU/Linux
[root@localhost ~]# rpm -qa | grep MonetDB
MonetDB-client-1.20.2-1.fc7.oid64
MonetDB-1.20.0-3.fc7.oid64
MonetDB-SQL-2.20.2-1.fc7.oid64
MonetDB5-server-5.2.2-1.fc7.oid64
Any idea what I am doing wrong?
Regards,
Rahul
=
Criminal Justice Degrees
Online Criminal Justice studies - Free info pack now.
http://a8-asy.a8ww.net/a8-ads/adftrclick?redirectid=c47df44e10a9717326d9f94…
--
Powered by Outblaze
I want to use MonetDB for benchmarking on the TPC-H workload. According
to the paper "MonetDB/X100 - A DBMS In The CPU Cache", the X100 module
achieves significant acceleration against MonetDB/MIL on Query 1 of
TPC-H. Still, when I search your site, there is no reference about how
it is possible to activate X100 inside MonetDB. I have only found X100
in your repository for mserver5 in Sourceforge, but it is not currently
active. Therefore, I have the following questions:
a) Has X100 been integrated to the Mserver5 version of MonetDB, or
included as an extension module?
b) Does Mserver5 use vector processing as proposed in the paper
"MonetDB/X100: Hyper-Pipelining Query Execution" (CIDR 2005) ?
c) How is it possible to use MonetDB/X100 for benchmarking purposes? Are
there any instructions online?
Thank you.
Hey all-
I've been benchmarking my application against mysql and postgres.
For my test application which has some 1Mil rows of data and my sample
query that returns some 260K rows here's some stats.
Speed for query:
Postgres 15 sec (8.1)
Monetdb 33 sec (nightly stable)
Mysql 100 sec (5.0)
Bulk load speed:
Postgres 12 Min
Monetdb 1 Min
Mysql 2 Min
Memory during bulk load:
Postgres 20 Meg
Monetdb 700 Meg!!! (And it doesn't come back down)
Mysql 20 Meg
Memory during query:
Postgres 30Meg
Monetdb 180 Meg (And increases by 10-20 Meg /query)
Mysql 30 Meg
Niels has some trick up his sleeves that bring the query speed down
(using the nightly current). But I'm concerned about memory use.
It's fine to use memory if you are going to be fast. But if I can't
load my data (in all I have 67+ mil rows) or have to restart the
server after a few queries it is problematic.
I've submitted bugs with valgrind reports during load/query, but am
wondering if other steps can be taken to help with the memory issue?
I know that both mysql and postgres (and a host of other open source
projects) have benefitted from coverity's offering [0]
and klocworks [1] as well. Have these been looked into? It might be
worth it have some automated "eyes" along with the "many eyes" of the
open source community that make all bugs shallow.
Some food for thought
cheers,
-matt
0 - http://scan.coverity.com/
1 - http://www.klocwork.com/company/releases/06_26_06.asp
HI All,
I am wondering whether any effort is made to build a frontend which
accepts SPARQL Queries and convers them into MIL for the Monet Back-end ??
Or how difficult to use MIL Directly in a Programming language like Java??
(Which is basically developing the front-end)
If such front-end accepts either SQL or XQuery etc then it has to be a
Query Parser and Optimizer . How ever if it uses MIL Directly then i guess
we don't need a Query Parser and Optimizer..!!
Any comments or info reg SPARQL Related front-ends will be of great help
Thanks
Sateesh
--
View this message in context: http://www.nabble.com/Is-there-a-Front-End-for-SPARQL-to-MonetDB-Backend-tp…
Sent from the monetdb-users mailing list archive at Nabble.com.
