Hello,
I am interested in evaluating MonetDB/XQuery. I have some XML collections
for a project I work on that are of 1-2 million individual XML documents
for which I use XQuery to access for OLAP style reporting operations. This
is a Java-based project, and while working on this project I developed a
thin XQuery-centric API so that I could evaluate/use many different XQuery
implementations, which my company released as an open source project
called Xaj (http://sourceforge.net/projects/xaj). Currently I am using a
Berkeley DB XML implementation, but am interested in exploring other
options.
I'd like to implement a MonetDB version of this API now, but I am having
some difficulty finding out how to effectively access MonetDB through
Java. So far I was able to use the MonetDB JDBC driver to construct XQuery
statements and get XML results, and also I was able to construct MonetDB
pf:add-doc() statements to add documents, but I am wondering if this is
the best approach (and if so, how to add documents to a remote MonetDB
server using the pf:add-doc() mechanism).
The API I am trying to implement you can view here, to give you an idea
what I'm trying to do:
http://xaj.svn.sourceforge.net/viewvc/xaj/xaj/src/net/sf/xaj/XmlDb.java?rev…
It was somewhat modeled after the defunct XML:DB API, but focused just on
add/store/XQuery operations. Any help/advise would be greatly appreciated!
-- m@
Is there any method, besides restarting the database,
of shrinking the sql log file?
When I am performing a large set of large inserts (via
COPY INTO) the log file gets very large. I wish there
was a way to cycle the log file.
Also during these large batches of inserts the
database server becomes less and less responsive until
I am forced to restart the server. Upon restart and
the normal recovery monet returns to a very responsive state.
Hi all,
I am new to monetDB and ran into the following problem (I'll try to be as
verbose as I can, perhaps it helps helping me):
I've installed MonetDB server 5.0.0, clients and commons 1.18.2 and sql
2.18.2 on Ubuntu Feisty. Then I've taken the JDBC example which is available
online and modified it to be able to insert data into a test database. More
specifically I'm using prepared statements and executeBatch() together with
the monetdb-1.6 JDBC driver.
Which is quite successful until about 5 million records are inserted -
afterwards the speed of the inserts drops dramatically.
I've experimented with different batch sizes (10000 records seems to work
ok), and I've tried to understand the chapter about memory tuning for
MonetDB (not sure yet about how to use mem_cursize() though).
In any case at first inserting 2 million records only takes about 5 minutes
(measured with Javas System.timemillis), later on this time increases and
the server gets rather unresponsive.
I found only two posts concerning the speed of JDBC inserts (from January
2007 I think) and I would be interested if a solution or explanation has
been found off-list.
Thank you
--
View this message in context: http://www.nabble.com/JDBC-insert-performance-tf4589411.html#a13100422
Sent from the monetdb-users mailing list archive at Nabble.com.
If I create a table, by default it goes into the "tmp" schema.
But when I try to create a "users" table, it complains, since there is
one in the "sys" schema. Why doesn't it create one in the "tmp"
schema?
ie:
CREATE TABLE users (fcol FLOAT(25));
!CREATE TABLE: name 'users' already in use
Just wondering about the resolution order.
thanks
-matt
Hi,
I would like to know how encoding related issues are handles in
MonetDB. For example, when I run the following Python script
# -*- coding: utf-8 -*-
import MonetSQLdb
x = MonetSQLdb.connect(host = 'localhost', user = 'voc', password =
'voc', lang = 'sql')
c = x.cursor()
c.execute('DROP TABLE test')
c.execute('CREATE TABLE test(id int, name varchar(20))')
c.execute('INSERT INTO test (id, name) VALUES (%s, %s)', (0, 'ä ö ü'))
c.close()
x.close()
and then look at the inserted data using DbVisualizer I get
ä ö ü
in the 'name' column. When I try to edit the value in the
DbVisualizer, I get an error message
"unexpected end of input". (The SQL statements reads update voc.test
set name = 'ö' where id = 0).
Is this a JDBC driver problem? When I connect to an MySQL database
with the MySQL JDBC driver included in DbVisualizer I can insert
arbitrary characters, and they get properly displayed in the GUI.
Kind regards,
Markus
I tried running the SQLAlchemy ORM testsuite. The first time it ran
for a while and then mserver segfaulted. (It also grew to using
700+Megs of memory. Am assumming this is a monetdb issue, since after
each test the tables are usually dropped). The server now has issues.
It appears to start fine:
# MonetDB server v5.2.3, based on kernel v1.20.1
# Serving database 'demo'
# Compiled for i686-pc-linux-gnu/32bit with 32bit OIDs dynamically linked
# Copyright (c) 1993-2007 CWI, all rights reserved
# Visit http://monetdb.cwi.nl/ for further information
#warning: please don't forget to set your vault key!
#(see /root/MonetDB/etc/monetdb5.conf)
# Listening for connection requests on mapi:monetdb://127.0.0.1:50000/
# MonetDB/SQL module v2.20.3 loaded
When I try to connect with the command line client it complains repeatedly:
!ERROR: BUNdelete: access denied to tmp_537, aborting.
!ERROR: BUNdelete: access denied to tmp_537, aborting.
....
I'm having difficulty linking a custom external C function into MonetDB without crashing the database connection. If anyone knows of a documented end-to-end example of adding external functions compiled with GCC, please let me know. I've documented a minimal example below that crashes the connection for me each time.
Software:
Red Hat Linux knock-off CentOS, i386.
Most recent MonetDB download from Sourceforge (server 5.2.2, client 1.20.2, common 1.20.0).
GCC 4.1.1
1. Create a simple C file named te.c.
int
TEtest( )
{
return 4;
}
2. Compile file, move it to the proper directory, and create symlinks.
$ gcc -c -fPIC te.c -o te.o
$ gcc -shared -Wl,-soname,lib_te.so.0 -o lib_te.so.0.0.0 te.o (also tried with soname = libte.so.0, leaving output as lib_te.so.0.0.0)
$ cp lib_te.so.0.0.0 $mhome/lib/MonetDB5/lib
$ cd $mhome/lib/MonetDB5/lib
$ ln -s lib_te.so.0.0.0 lib_te.so.0
$ ln -s lib_te.so.0.0.0 lib_te.so
3. Create a MAL file, saved as $mhome/lib/MonetDB5/te.mal
module te;
command test( ):int
address TEtest();
command doesnotexist( ):int
address TEdoesnotexist;
4. Update mal_init.mal by adding the following 2 lines:
library te;
include te;
5. Start a random DB and try to invoke the test function. Merovingian log show database dino killed by signal 11 afterwards.
$ cd $mhome/bin
$ ./merovingian &
[1] xxxx
$ ./mclient -lmal --database=dino -h COS1
mal> te.test( );
MAPI = monetdb@COS1:50001
ACTION= read_line
QUERY = te.test();
ERROR = Connection terminated
If I instead invoke te.doesnotexist(), I get a different error (below.) This leads me to think that MonetDB is findng and loading my compiled .so file, but it wasn't compiled correctly or otherwise isn't providing what MonetDB expects.
mal> te.doesnotexist();
MAPI = monetdb@COS1:50001
QUERY = te.doesnotexist();
ERROR = !TypeException:user.main[1]:'te.doesnotexist' undefined in: _1:int := te.doesnotexist()
Any pointers are appreciated.
Thomas Schulte
How do I run non-ascii queries in python? Or from the command line
for that matter?
Naive tests on my part fail....
python:
RuntimeError: !syntax error, unexpected IDENT, expecting INTERSECT or
EXCEPT or UNION in: "select name from tables where system=false and
name=unitéble2"
commandline:
sql>select name from tables where system=false and name=unitéble2;
!syntax error, unexpected IDENT, expecting INTERSECT or EXCEPT or
UNION in: "select name from tables where system=false and
name=unitéble2"
Thanks
-matt
