January 2007 - users-list

[MonetDB-users] BAT Memory Usage
by Guillermo Arbeiza 05 Jul '07

05 Jul '07

Hello everyone, We are evaluating Monet right now to store and analyze a huge quantity of data. We have several facts collection that range from 1 million row, 200 cols. to way over 300 million rows and over 10.000 columns. We are aware of memory limitations of current Monet version and looking forward to the new one but in the meantime we'd like to do some performance and load tests. In order to know which data we will be able to store, we need to know how much memory will use a BAT depending on the datatype and the number of elements we want to store in it. Is there a lineal relation between those variables (ie. 8 bytes oid, 8 bytes int => 16 bytes per element) or do Monet encode data using intervals or such? Thank you in advance and kudos for your big efforts, Guillermo Arbeiza, Open Sistemas de Información e Internet garbeiza(AT)opensistemas.com ---------------------------------------------------------------------- Mensaje enviado por el servidor de openSistemas (www.opensistemas.com)

3 2

[MonetDB-users] Rocket Stock Report
by heart universe 26 Jan '07

26 Jan '07

1 0

[MonetDB-users] SQL compilation problem
by Ying Zhang 26 Jan '07

26 Jan '07

Hello, Compiling SQL (stable branch, Fedora core 4) from scratch gives the following error: gcc -m64 -DHAVE_CONFIG_H -I. -I../../../../src/backends/monet5 -I../../.. -I../../../../src/backends/monet5 -I../../include -I../../../../src/backends/monet5/../../include -I../../common -I../../../../src/backends/monet5/../../common -I../../storage -I../../../../src/backends/monet5/../../storage -I../../server -I../../../../src/backends/monet5/../../server -I/net/andorea/export/scratch1/zhang/monet/dev/G.64.64.d.debug/include/MonetDB -I/net/andorea/export/scratch1/zhang/monet/dev/G.64.64.d.debug/include/MonetDB/common -I/net/andorea/export/scratch1/zhang/monet/dev/G.64.64.d.debug/include/MonetDB/gdk -I/net/andorea/export/scratch1/zhang/monet/dev/G.64.64.d.debug/include/MonetDB -I/net/andorea/export/scratch1/zhang/monet/dev/G.64.64.d.debug/include/MonetDB/mapilib -I/net/andorea/export/scratch1/zhang/monet/dev/G.64.64.d.debug/include/MonetDB5/atoms -I/net/andorea/export/scratch1/zhang/monet/dev/G.64.64.d.debug/include/MonetDB5/compiler -I/net/andorea/export/scratch1/zhang/monet/dev/G.64.64.d.debug/include/MonetDB5/kernel -I/net/andorea/export/scratch1/zhang/monet/dev/G.64.64.d.debug/include/MonetDB5/mal -I/net/andorea/export/scratch1/zhang/monet/dev/G.64.64.d.debug/include/MonetDB5/optimizer -I/net/andorea/export/scratch1/zhang/monet/dev/G.64.64.d.debug/include/MonetDB5/scheduler -I/net/andorea/export/scratch1/zhang/monet/dev/G.64.64.d.debug/include/MonetDB5 -DLIBSQL -std=c99 -g -c sql.c -fPIC -DPIC -o .libs/lib_sql_la-sql.o ../../../../src/backends/monet5/sql.mx: In function 'str_2_sqlblob': ../../../../src/backends/monet5/sql.mx:1763: error: 'TYPE_sqlblob' undeclared (first use in this function) ../../../../src/backends/monet5/sql.mx:1763: error: (Each undeclared identifier is reported only once ../../../../src/backends/monet5/sql.mx:1763: error: for each function it appears in.) ../../../../src/backends/monet5/sql.mx: In function 'SQLsqlblob_2_str': ../../../../src/backends/monet5/sql.mx:1784: warning: implicit declaration of function 'sqlblob_tostr' make[7]: *** [lib_sql_la-sql.lo] Error 1 make[7]: Leaving directory `/ufs/zhang/src/monet/dev/sql/build/src/backends/monet5' make[6]: *** [all] Error 2 make[6]: Leaving directory `/ufs/zhang/src/monet/dev/sql/build/src/backends/monet5' make[5]: *** [all-recursive] Error 1 make[5]: Leaving directory `/ufs/zhang/src/monet/dev/sql/build/src/backends' make[4]: *** [all] Error 2 make[4]: Leaving directory `/ufs/zhang/src/monet/dev/sql/build/src/backends' make[3]: *** [all-recursive] Error 1 make[3]: Leaving directory `/ufs/zhang/src/monet/dev/sql/build/src' make[2]: *** [all] Error 2 make[2]: Leaving directory `/ufs/zhang/src/monet/dev/sql/build/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/ufs/zhang/src/monet/dev/sql/build' make: *** [all] Error 2 The code is up-to-date Does anyone have a idea about this? Thanks! Jennie

2 1

[MonetDB-users] Compilation problem on electra
by Ying Zhang 26 Jan '07

26 Jan '07

Hi, I'm trying to install monet (stable branch) on electra (the 32 bits CWI machine running SunOS5.10), compilation of module MonetDB gives the following error: gcc -DHAVE_CONFIG_H -I. -I../../../src/gdk -I../.. -I../../../src/gdk -I../common -I../../../src/gdk/../common -DLIBGDK -D__EXTENSIONS__ -std=c99 -g -DHWCOUNTERS -DHW_SunOS -DHW_i86pc -D_REENTRANT -c gdk_atoms.c -fPIC -DPIC -o .libs/libbat_la-gdk_atoms.o ../../../src/gdk/gdk_atoms.mx: In function `fltFromStr': ../../../src/gdk/gdk_atoms.mx:1714: error: `__builtin_infinity' undeclared (first use in this function) ../../../src/gdk/gdk_atoms.mx:1714: error: (Each undeclared identifier is reported only once ../../../src/gdk/gdk_atoms.mx:1714: error: for each function it appears in.) ../../../src/gdk/gdk_atoms.mx:1717: error: invalid operands to binary == gmake[5]: *** [libbat_la-gdk_atoms.lo] Error 1 gmake[5]: Leaving directory `/export/scratch1/zhang/monet/dev/MonetDB/build/src/gdk' gmake[4]: *** [install] Error 2 gmake[4]: Leaving directory `/export/scratch1/zhang/monet/dev/MonetDB/build/src/gdk' gmake[3]: *** [install-recursive] Error 1 gmake[3]: Leaving directory `/export/scratch1/zhang/monet/dev/MonetDB/build/src' gmake[2]: *** [install] Error 2 gmake[2]: Leaving directory `/export/scratch1/zhang/monet/dev/MonetDB/build/src' gmake[1]: *** [install-recursive] Error 1 gmake[1]: Leaving directory `/export/scratch1/zhang/monet/dev/MonetDB/build' gmake: *** [install] Error 2 This code is a fresh check-out from Source Forge. I used the following configure options: $ ../configure --prefix=$INSTALL_DIR --enable-debug --disable-optimize --enable-assert --enable-bits=32 Does anyone have a idea about this problem? Thanks! Regards, Jennie

2 1

[MonetDB-users] New Release Branches --- * Please read carefully! *
by Stefan Manegold 21 Jan '07

21 Jan '07

Dear all, please read the following carefully till the end! To prepare the upcoming release of MonetDB, I just created the respective new release branches in CVS and set new version numbers accordingly: module branch version ---------- ----------------- ----------- buildtools MonetDB_1-16 - MonetDB MonetDB_1-16 1.16.0 clients Clients_1-16 1.16.0 MonetDB4 MonetDB_4-16 4.16.0 MonetDB5 MonetDB_5-0_beta1 5.0.0_beta1 template MonetDB_1-16 0.1.3 sql SQL_2-16 2.16.0 pathfinder XQuery_0-16 0.16.0 Please recall: All bug-fixes go to the new release branches (see below), only! All new features go to the development trunks, only! Any violation of these rules might be "punished" by a forced undo of the respective changes. As usually, we will take care of propagating bug-fixes from the new release branches to the development trunks on an (ir)regular basis or on kind request. Upgrading your MonetDB version =============================== Please read the following *before* starting any upgrade attempt! Binary installation & source tarballs ------------------------------------- In case you're using binary installation (Windows Installer or RPMs) or source tarballs, please be patient for a few more days. We plan to finish the release before end of this week, including the respective documentation. CVS sources ----------- In case you're using CVS sources, please note that there have been two major changes: 1) The former "MonetDB" CVS module has been split in three CVS modules, a) "MonetDB" still contains the old common source base, in particular src/common/ and src/gdk/ . b) The new "clients" CVS module contains the GDK-independent code base for the MAPI-based clients, i.e., the code from former MonetDB/src/mapi/ as well as from former sql/src/jdbc/ and sql/src/odbc/ . c) The new "MonetDB4" CVS module contains the well known MonetDB Version 4 server, including the MIL interpreter (src/monet/) and the MIL extension modules (src/modules/). 2) The formerly CWI-internal code base for the new MonetDB Version 5 server ("monet5") has become public on SourceForge as CVS module "MonetDB5" (next to "MonetDB4"). "MonetDB5" introduces the new MAL interpreter as well as various new features. See the upcoming release announcement for more details. In case you used to use the "monet5" code at/from CWI, you should read http://sourceforge.net/mailarchive/forum.php?thread_id=31481666&forum_id=39… for details how to upgrade to "MonetDB5" from SourceForge. The build a complete MonetDB4 (MIL) or MonetDB5 (MAL) server, you need to first get, compile and install buildtools, MonetDB, and clients in that order. Then you can compile and install MonetDB4, MonetDB5, or both. But *before* doing so, please read till the end of this message! Updating CVS existing sources ----------------------------- Please follow the subsequent steps in the given order! Before upgrading to the new release branches --- at least if you've been using MonetDB 4.14, MonetDB/SQL 2.14, MonetDB/XQuery 0.14 any older version, or MonetDB 4.15, MonetDB/SQL 2.15, MonetDB/XQuery 0.15 before the "repository split" --- it is mandatory to clean-up any previous installation of MonetDB. There are various ways to do so: In case you are using a distinctive prefix directory (the one you specified with the --prefix option of configure) only for MonetDB software, and your default database (<prefix>/var/MonetDB/dbfarm/) does not contain any crucial/important data, the easiest way is to remove the whole prefix directory (rm -r <prefix>). In case you installed MonetDB into a prefix directory that also contains other software (e.g., /usr/, /usr/local/, ~/local/, etc.), but you still have your ("old") build directories intact, call `make uninstall` in all build directories in *reverse* order of installation, e.g., ( cd <amdb-builddir> && make uninstall && make clean ) ( cd <pruning-builddir> && make uninstall && make clean ) ( cd <xml-builddir> && make uninstall && make clean ) ( cd <pathfinder-builddir> && make uninstall && make clean ) ( cd <sql-builddir> && make uninstall && make clean ) ( cd <template-builddir> && make uninstall && make clean ) ( cd <monet5-builddir> && make uninstall && make clean ) ( cd <MonetDB5-builddir> && make uninstall && make clean ) ( cd <MonetDB4-builddir> && make uninstall && make clean ) ( cd <clients-builddir> && make uninstall && make clean ) ( cd <MonetDB-builddir> && make uninstall && make clean ) ( cd <buildtools-builddir> && make uninstall && make clean ) In case none of the above two options applies to you please make sure that you at least remove the following directories: rm -r <prefix>/share/MonetDB rm -r <prefix>/include/MonetDB rm -r <prefix>/lib/MonetDB rm -r <prefix>/lib64/MonetDB Then, you should clean up your build directories (those where you called configure for the various parts of the code base): In case your build directories are different from your source directories (as recommended), you can simple delete (rm -r) them all completely. In case your build directories are identical with you source directories, and you haven't called `make clean` together with `make uninstall` before), call `make clean` in all build directories in *reverse* order of installation, e.g., ( cd <amdb-builddir> && make clean ) ( cd <pruning-builddir> && make clean ) ( cd <xml-builddir> && make clean ) ( cd <pathfinder-builddir> && make clean ) ( cd <sql-builddir> && make clean ) ( cd <template-builddir> && make clean ) ( cd <MonetDB5-builddir> && make clean ) ( cd <monet5-builddir> && make clean ) ( cd <MonetDB4-builddir> && make clean ) ( cd <clients-builddir> && make clean ) ( cd <MonetDB-builddir> && make clean ) ( cd <buildtools-builddir> && make clean ) Finally, you're ready to upgrade your existing CVS sources as follows (skip those parts you do not have yet): ( cd buildtools && cvs update -dP -r MonetDB_1-16 ) ( cd MonetDB && cvs update -dP -r MonetDB_1-16 ) ( cd clients && cvs update -dP -r MonetDB_1-16 ) ( cd MonetDB4 && cvs update -dP -r MonetDB_4-16 ) ( cd MonetDB5 && cvs update -dP -r MonetDB_5-0_beta1 ) ( cd template && cvs update -dP -r MonetDB_1-16 ) ( cd sql && cvs update -dP -r SQL_2-16 ) ( cd pathfinder && cvs update -dP -r XQuery_0-16 ) Getting new CVS sources ----------------------- Once you updated your existing CVS source, or in case you (want to) start with a complete new check-out, you can get the (missing) MonetDB source from the CVS repository on SourceForge as follows. (Skip those part that you already updated above and/or those parts that you do not want/need. Recall the following dependencies: MonetDB[45] require buildtools, MonetDB & clients; template & sql require at least one of MonetDB[45]; pathfinder requires MonetDB4.) Developers with ssh access replace <access> by :ext:<username>; others replace <access> by :pserver:anonymous for anonymous pserver access. cvs -d <access>@monetdb.cvs.sourceforge.net:/cvsroot/monetdb co -P -rMonetDB_1-16 buildtools MonetDB clients template cvs -d <access>@monetdb.cvs.sourceforge.net:/cvsroot/monetdb co -P -rMonetDB_4-16 MonetDB4 cvs -d <access>@monetdb.cvs.sourceforge.net:/cvsroot/monetdb co -P -rMonetDB_5-0_beta1 MonetDB5 cvs -d <access>@monetdb.cvs.sourceforge.net:/cvsroot/monetdb co -P -rSQL_2-16 sql cvs -d <access>@monetdb.cvs.sourceforge.net:/cvsroot/monetdb co -P -rXQuery_0-16 pathfinder Compilation and installation ---------------------------- Once all desired/required source are up-to-date you can compile and in stall the parts one by one as follows (recall, we recommend to use separate build directories). (Again, skip the parts that you don't want/need.) sourcebase=/my/source/base prefix=/my/target/prefix export PATH=$prefix/bin:$PATH export PYTHONPATH="$prefix/`python -c 'import distutils.sysconfig; print distutils.sysconfig.get_python_lib(0,0,"")'`" for i in buildtools MonetDB clients MonetDB4 MonetDB5 template sql pathfinder ; do source=$sourcebase/$i build=$source/_BUILD_ mkdir -p $build ( cd $source && ./bootstrap && ( cd $build && $source/configure --prefix=$prefix && make && make install ) ) done That should be it. Enjoy! In case you have any questions and.or problems please feel free to use the bug tracker at http://sourceforge.net/tracker/?group_id=56967, or the respective mailing lists https://lists.sourceforge.net/lists/listinfo/monetdb-developers and/or https://lists.sourceforge.net/lists/listinfo/monetdb-users . Kind regards The MonetDB developer team. -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 1

Re: [MonetDB-users] my retrospectio
by Cerberus Loredo 12 Jan '07

12 Jan '07

else. We watch them and keep them going mostly by habit-and for our Re: my retrospectio giants back and kidneys that opened his mouth wide with pain, forcing under their noses so they had to step back again. I got something Youre a good guy, Tremearne. My heartiest thanks. past the throne and into the opening. Madonette touched Steengos These can be tapped by satellite. now. to make it out. Sweating and trembling I dropped the replacement artifact temporooter orders. I have the feeling that I am going to enjoy being a civilian. Do they have their own doctors?

1 0

[MonetDB-users] print bookseller database
by in 10 Jan '07

10 Jan '07

To view does not currently have default page. Provides virus, protection when using delivered? Listing tsx july interim six months june unveils. Without is strictly forbidden subject civil andor criminal. Minimum windows mexp explorer media player more how. Process of being upgraded configured, please try this. Customer footprint canada latin systems enable control ip. Corp sirific innovation copyright copy reserved mac os? Funding introduces suite providers. Want hear email questions comments emerson wulling printer. Sales products, eg full matchnet etc we respect, privacy. Shrink sefys guide, bitcomet, ffdshow rev nero, burning rom. Quality digitally rights managed. America december adelphia protect worms, spam. Halo ii signals gaming, boom. Napster while, avoiding their, problems doesnt rely central server. Denial general mitigation camiant leverage cablelabs pcmm battle zombie. Gui linux available operating osx. Enabling industrys most powerful broadband, network solutions. Protect worms spam november. Ip realtime, dns servers fire secures million funding introduces. Writers digital video afterdawn. Threats liberty signs global purchase sandvines fast brazilian wightman! Statement on data, practices must read. Robert definitive work gtthen maybe tomorrow bob flecks.

1 0

Re: [MonetDB-users] about insert speed on Monetdb
by Fabian Groffen 09 Jan '07

09 Jan '07

Hi ypli, Sorry for the belated response to your question. I'm puzzled too on this performance drama. We will run again our experiments to check this behaviour. Is your application suitable to be run by us? In that case we would like to use your application to find out why the performance degrades. I assume you execute something like this: prepare(some query) while items left do: for each 1000 items do: set(arguments) addBatch() loop executeBatch() loop Note that I don't switch off auto-commit here. On 06-01-2007 00:23:36 +0800, yichengpingan123(a)sina.com wrote: > Renmin University China > > > Mr Fabian.Groffen: > > When I use the jdbc(PreparedStatment,setXXX,addBatch(),excuteBatch())to > insert data ,the time is more elapse,the speed is more slow.The first > one minute may about 700/s,the tenth minute may about > > 400/s,and then about 100/s. > > I'm puzzled. > > > With many thanks > > > ypli

1 0

[MonetDB-users] about the insert speed(useint jdbc)
by yichengpingan123＠sina.com 08 Jan '07

08 Jan '07

I have a problem when I insert data into Monetdb,the more time eclapse,the more insert speed slowe, At the begining,the speed may 2000/s ,and then may 1000/s,and then may 400/s,and last may 10/s. I used jdbc ( PrepareStatment ,addBatch,setXXX,excuteBatch )to connect Monetdb. Thank you very much ------------------------------------------------------------------- 惠普商用台式机dc5750,高性能，低功耗！( http://ad4.sina.com.cn/sina/limeng3/mail_zhuiyu/2006/mail_zhuiyu_20061225.h… ) =================================================================== 注册新浪2G免费邮箱（ http://mail.sina.com.cn/chooseMode.html ）

1 0

[MonetDB-users] ===2007年　年賀状===
by ◆mailplanet 01 Jan '07

01 Jan '07

はるかさんからmonetdb-users(a)lists.sourceforge.xn--net-s83b5g5f 新年のグリーティングカードが届いております。こちらをご確認下さい。 http://www.foreverss.com/p/planet ☆　mailplanetグリーティング　☆

1 0