users-list June 2008

users-list@monetdb.org

10 participants
15 discussions

by Matt Magoffin

Hello, I am interested in evaluating MonetDB/XQuery. I have some XML collections for a project I work on that are of 1-2 million individual XML documents for which I use XQuery to access for OLAP style reporting operations. This is a Java-based project, and while working on this project I developed a thin XQuery-centric API so that I could evaluate/use many different XQuery implementations, which my company released as an open source project called Xaj (http://sourceforge.net/projects/xaj). Currently I am using a Berkeley DB XML implementation, but am interested in exploring other options. I'd like to implement a MonetDB version of this API now, but I am having some difficulty finding out how to effectively access MonetDB through Java. So far I was able to use the MonetDB JDBC driver to construct XQuery statements and get XML results, and also I was able to construct MonetDB pf:add-doc() statements to add documents, but I am wondering if this is the best approach (and if so, how to add documents to a remote MonetDB server using the pf:add-doc() mechanism). The API I am trying to implement you can view here, to give you an idea what I'm trying to do: http://xaj.svn.sourceforge.net/viewvc/xaj/xaj/src/net/sf/xaj/XmlDb.java?rev… It was somewhat modeled after the defunct XML:DB API, but focused just on add/store/XQuery operations. Any help/advise would be greatly appreciated! -- m@

13 years, 11 months

[MonetDB-users] New MonetDB Release

by Stefan Manegold

Dear MonetDB users and developers, The MonetDB team at CWI is pleased to announce the new "Jun2008" release of the MonetDB suite of programs. For MonetDB/XQuery, this release marks the roll forward from the old "milprint_summer"-based Pathfinder compiler to the new "Algebra"-based Pathfinder compiler. MonetDB/SQL has been extended with initial support for OpenGIS via the "Geom[etry]" extension module, aka. MonetDB/SQL/GIS. Additionally, the configure defaults for compilation from sources on Unix(-like) systems as well as the Windows installers have been cleaned-up. For more information, please see the detailed release notes at http://monetdb.cwi.nl/Development/Releases/Version4.24/ IMPORTANT NOTE: Before upgrading to the latest "Jun2008" release from any previous release of MonetDB, please make a dump of your database, and rename/move/backup the database directories (dbfarm, sql_logs, xquery_logs). After the upgrade, restore your database, again. Once you have checked and confirmed the correctness of the restore, you can delete the above renamed/moved/backed-up old database directories. For more information, please see the detailed release notes at http://monetdb.cwi.nl/Development/Releases/Version4.24/ The new release consists of the following packages: MonetDB Common 1.24.0 MonetDB Clients 1.24.0 MonetDB4 Server 4.24.0 MonetDB4 XQuery 0.24.0 MonetDB5 Server 5.6.0 MonetDB5 SQL 2.24.0 MonetDB5 Geom 0.4.0 monetdb-1.8-jdbc.jar There is of course also a new "Super Source Ball", a tarball containing the combined sources. All this is available on our SourceForge project page. See https://sourceforge.net/project/showfiles.php?group_id=56967 for the downloadable files. Enjoy! The MonetDB development team. -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

14 years, 1 month

[MonetDB-users] Python on Windows

by Kees Varekamp

Does anyone know how to get the Python libs running on Win XP? Tried it with latest trunk and release versions of Monet, and with Python 2.4 and 2.5 File "C:\Python25\lib\MonetDB\MapiLib.py", line 7, in <module> import _MapiLib ImportError: DLL load failed: The specified module could not be found. _MapiLib.pyd is there alright. Is this a version problem? Thanks, Kees

14 years, 1 month

[MonetDB-users] MonetDB database size restrictions

by Sam Mason

Hi, I hope you don't get asked this question too often, I've had a look for FAQs but couldn't find anything obvious. I'm trying to load some data into MonetDB (built on a i386 Debian "etch" box from MonetDB-Feb2008-SuperBall.tar.bz2) and I'm getting strange problems. I've got a table with 11 columns (2 int, 5 varchar, 4 date) and about 38 million rows (the text file I want to pull in is 3.3GB). If I try COPYing the data in, Monet alternately stops making progress (the computer is reading from the disk slowly, so maybe it's just going to take a *long* time to finish) or crashes with the following message: !SQLException:importTable:failed to import table !ERROR: GDKload: cannot mmap(): name=24/2470, ext=tail.priv !OS: Cannot allocate memory !ERROR: GDKload failed: name=24/2470, ext=tail.priv If I try splitting the file into smaller pieces (about a gig each), things work and I end up with the data in the database. It would be nice to know why I needed to split the file up first. Now I've got the data in I tried running a few queries to make sure everything went OK. I'm getting some very strange results when doing aggregations. The following work: SELECT count(*) FROM source_livestock; +----------+ | count_ | +==========+ | 37898310 | +----------+ SELECT sex_code, count(*) FROM source_livestock GROUP BY sex_code; +----------+---------------+ | sex_code |count_sex_code | +==========+===============+ | M | 17505304 | | F | 20392997 | | null | 9 | +----------+---------------+ This last query came back amazingly quickly, about ten times faster than Postgres. Monet looks like a fairly amazing database if I can routinely expect queries to be this much faster. SELECT count(*); +--------------+ | single_value | +==============+ | 1 | +--------------+ but the following don't do what I'd expect: SELECT count(1); MAPI = monetdb@endemic:50001 ACTION= read_line QUERY = select count(1); ERROR = Connection terminated I'd expect to get a single row with a single value containing the number 1 from this query. And for the following query I'd expect to get the same, just with the result being zero. SELECT count(null); !syntax error, unexpected sqlNULL in: "select count(null" This is how I found the bug: SELECT sex_code, count(1) FROM source_livestock GROUP BY sex_code; !TypeException:user.s3_1[26]:'aggr.count_no_nil' undefined in: _36:bat[:any,:int] := aggr.count_no_nil(_35:bte, _31:bat[:oid,:oid], _32:bat[:oid,:oid]) !SQLException:sql.rsColumn:Cannot access descriptor !WARNING: BATdescriptor: range error 1 I've come to expect COUNT(*) and COUNT(1) as having the same semantics. But I may be missing something. SELECT sex_code, count(1+1) FROM source_livestock GROUP BY sex_code; !TypeException:user.s4_1[28]:'aggr.count_no_nil' undefined in: _39:bat[:any,:int] := aggr.count_no_nil(_38:bte, _32:bat[:oid,:oid], _33:bat[:oid,:oid]) +----------+---------------------+ | sex_code |sql_add_single_value | +==========+=====================+ | M | monetdb | +----------+---------------------+ This second one is particularly bad because I get an incorrect answer back. Any idea what I'm doing wrong? Thanks, Sam

14 years, 1 month

[MonetDB-users] MonetDB Server dies on connect

by Robert Schumacher

Hello, i was evaluating MonetDB. Got the Server running finr on Windows XP x64. I created about 12 Tabled by using a JDBC Client, then the Server died instantly. Since then when I start the Server it dies asap when I either start the Windows Client or perform a JDBC connection. I get no error log, no message, nothing. Has anybody an idea how to get MonetDB to stop crashing on connect? Thanks, Robert Robert Schumacher, Senior Software Developer prevero Systems GmbH Landsberger Straße 154 80339 München Tel +49 89 455 77 - 0 Fax +49 89 455 77 - 200 Registergericht: Amtsgericht München, HRB 143809 -- Geschäftsführer: Matthias Thurner Bitte beachten Sie unsere Veranstaltungshinweise unter www.prevero.de

14 years, 1 month

Re: [MonetDB-users] [Monetdb-developers] arrow keys not working in mclient in stable nightly

by Martin Kersten

Rt Ibmer wrote: > Hi - I use a SSH client to access my Fedora 8 core box. When I was using the RELEASE built I was able to use left/right/up/down/home/end keys within mclient to edit the line. For instance I could push up arrow to have it automatically cycle through the previous lines, and then use left arrow to move over to a part of the line and process to change it. > There have not been changes in that area. Running against the current also does not show the problem > That all worked fine under the RELEASE build. > > However now that I have built from source the nightly stable (as of a couple nights ago) I can no longer use these keys. For instance when I press UP arrow it simply adds "^[[A" to the line. Other arrow keys / home /end do something similar. > Nothing has changed in this area. I mostly use a Putty ssh client to access my machine and work with mclient. It works as expected. You may want to check the preference setting of the ssh client. > Not sure but I think something broke regarding this at some point between the released build and the stable nightly. Thanks. > > > > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-developers mailing list > Monetdb-developers(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-developers >

14 years, 1 month

Re: [MonetDB-users] replication in MonetDB

by Martin Kersten

Alexandru Toth wrote: > Hello, > > I'm the developer of Cubulus OLAP > (http://sourceforge.net/projects/cubulus/ ) . Hello Alexandru Welcome to the MonetDB user community. > > On the roadmap there is a hint about replication, coming in SQL 2.2 > (February 2008) release. Is it implemented ? If not, is it possible to use We have been experimenting with replication techniques. Some of the hooks have been included in the distribution. However, we are not yet satisfied and with the result. It remains on our R&D list to come up with a novel and efficient solution. > some existing software ex: http://dbreplicator.org/ Middleware solutions could potentially provide a short term solution. However, we don't know if they are powerful enough or require functionality currently not available in our JDBC driver. > > Or, is it possible to load data into "master", stop it, sync the > dbfarm to several slaves (into new directories), and re-start the > slaves with the new dbfarm path ? Of course, slaves are used as > read-only copies. ofcourse, provided you don't forget to copy the sql_logs as well. You can even instruct the server that the database is readonly. See the documentation : http://monetdb.cwi.nl/projects/monetdb/SQL/Documentation/Storage-Modes.html regards, Martin > > -Alex > >

14 years, 1 month

Re: [MonetDB-users] [Monetdb-developers] MonetDB Server dies on connect

by Martin Kersten

Sjoerd Mullender wrote: > Martin Kersten wrote: > >> Sjoerd, >> >> I was planning to try out the installers on my Vista machine. >> The previous version is v5.4.0 installed >> >> Uninstalling it gives the following error: >> There is a problem with this Windows Installer >> package. A program required for this install >> to complete could not be run. Contact your >> support personnel or package vendor. >> >> You want specific information, or should i manually >> remove the C:\Program Files\CWI/MonetDB5 >> > > Manually removing won't help unless you also edit the registry. > > My guess is that it complains about the uninstall of the ODBC driver. > Can you run > "C:\Program Files\CWI\MonetDB5\odbcinstall" /Uninstall > (I think that is the exact name) manually from a cmd window? > ran this program , which worked. However, uninstalling MonetDB still gives the same error.

14 years, 1 month

[MonetDB-users] New default configure options for MonetDB

by Stefan Manegold

Dear MonetDB developer and users, For convenience of both developers and users as well as to comply even more with open source standards, we now set/use the following defaults for the configure options --enable-strict, --enable-assert, --enable-debug, --enable-optimize (Unless specified differently, the same defaults are used for both the "Stable" release branches and the "Current" development trunk.) When compiling from CVS sources (as mainly done by developers): strict=yes assert=yes debug=yes optimize=no (*) When compiling from packaged/distributed sources (i.e., tarballs) (as mainly done by users): strict=no assert=no debug=no optimize=no (*) For building binary distributions (RPMs, MSIs): strict=no assert=no debug=no optimize=yes With nightly testing for the "Stable" release branches: strict=yes assert=no debug=no optimize=yes With nightly testing for the "Current" development trunk: strict=yes assert=yes debug=no optimize=yes (*) IMPORTANT NOTE: Since "--enable-optimize=yes" is no longer the default for any case except binary packages, it is *strongly recommended* to (re)compile everything from scratch, *explicitly configured* with "--enable-debug=no --enable-optimize=yes", in case you want/need to run any performance experiments with MonetDB! (Please note: "--enable-X=yes" is equivalent to "--enable-X", and "--enable-X=no" is equivalent to "--disable-X".) Don't hesitate to ask in case you have any questions! Enjoy! Stefan -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

14 years, 1 month

Re: [MonetDB-users] [Monetdb-developers] Fastest way to calculate median?

by Martin Kersten

Rt Ibmer wrote: > Hi - I have an int column with several million rows of data. What is the fastest way to write a query to get the median value of the column? Apparently there is no median() function in SQL. A web search revealed lots of various talk about using stored procedures and fancy functions. None of this looked like it would be very efficient. I searched these archives but didn't see a single mention of median. Thank you! > True, the complexity of the median() in essence requires a partial sort of the complete list. It is a member of a group of statistical functions. A feature request has been added to the list. > > > > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-developers mailing list > Monetdb-developers(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-developers >

14 years, 1 month

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

users-list June 2008