March 2010 - developers-list

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB5/src/modules/kernel batcast.mx, Feb2010, 1.31.2.7, 1.31.2.8
by Stefan Manegold 20 Mar '10

20 Mar '10

On Sat, Mar 20, 2010 at 12:53:12PM +0000, Martin Kersten wrote: > Update of /cvsroot/monetdb/MonetDB5/src/modules/kernel > In directory sfp-cvsdas-1.v30.ch3.sourceforge.com:/tmp/cvs-serv10537 > > Modified Files: > Tag: Feb2010 > batcast.mx > Log Message: > Backport of possible dangerous double frees. No roll forward needed. While your revised code looks indeed more efficient to me (as it allows ATOMfromstr() --- well, at least strFromStr(); I cannot tell about other "extern" (varsized) atoms --- to reuse a allocated temporary buffer), I wonder whether/why the original experienced double frees? Called with v==NULL, ATOMfromstr() --- again, I refer only to strFromStr(); FromStr functions for other "extern" (varsized) atoms might behave differently; IMHO, we don't have any "contract" or "policies" for FromStr() implementations --- would allocate a buffer for the result, ATOMput() copies the value from the buffer to the BAT heap, and then the buffer is (was) freed. Looks very sound to me. What am I missing? I would be thankful for any enlightment. Stefan > Index: batcast.mx > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB5/src/modules/kernel/batcast.mx,v > retrieving revision 1.31.2.7 > retrieving revision 1.31.2.8 > diff -u -d -r1.31.2.7 -r1.31.2.8 > --- batcast.mx 18 Feb 2010 01:04:05 -0000 1.31.2.7 > +++ batcast.mx 20 Mar 2010 12:53:10 -0000 1.31.2.8 > @@ -248,6 +248,8 @@ > BAT *b,*bn; > BATiter bi; > BUN p, q, r = 0; > + ptr v = NULL; > + int len = 0; > > @:getBATdescriptor(bid, b, "batcalc.@1")@ > @:voidresultBAT(TYPE_@1,"batcalc.@1")@ > @@ -255,15 +257,12 @@ > > BATaccessBegin(b, USE_TAIL, MMAP_SEQUENTIAL); > BATloop(b, p, q) { > - ptr v = NULL; > - int len = 0; > - > ATOMfromstr(TYPE_@1, &v, &len, (char *) BUNtvar(bi, p)); > ATOMput(TYPE_@1, bn->T->vheap, Tloc(bn, r), v); > - if (ATOMextern(TYPE_@1)) > - GDKfree(v); > r++; > } > + if (ATOMextern(TYPE_@1)) > + GDKfree(v); > BATsetcount(bn, BATcount(b)); > bunins_failed: > BATaccessEnd(b, USE_TAIL, MMAP_SEQUENTIAL); > > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 |

2 3

Re: [Monetdb-developers] why open() before dlopen()?
by Mark Bucciarelli 19 Mar '10

19 Mar '10

On Fri, Mar 19, 2010 at 11:38 AM, Martin Kersten <Martin.Kersten(a)cwi.nl> wrote: > > Also portability over all platforms is point of concern, > there is some old old out there, e.g. HP-UX. > Ok, then I'll make the patch specific to OpenBSD unless someone here has a preferred solution. Thanks, m

2 2

[Monetdb-developers] why open() before dlopen()?
by Mark Bucciarelli 19 Mar '10

19 Mar '10

I'm working on fixing tracker #2971516. What I have learned is that libtool on OpenBSD does not create .so files without the version suffix because they use a different value of library_names_spec in libtool than Linux does. However, their dlopen() has some ld magic; for example, dlopen("libbat.so") and dlopen("/usr/local/lib/libbat.so.1.34") both work just fine. However the monetdb server tries to open() the .so first, and the open() method does not have the same ld magic as dlopen(). So my question is---why call open() at all? What value does it add to the dlopen() call? Seems like picking the right library version is a responsibility that should belong to ld. I'm contemplating a patch that skips the open() call for .so files. Thanks, m

1 0

[Monetdb-developers] IMPORTANT: changes are afoot
by Sjoerd Mullender 18 Mar '10

18 Mar '10

For a long time now we have had serious performance issues with the MonetDB CVS repository on Sourceforge. MonetDB B.V. has bought a machine that is to be used as the host for the public repository for MonetDB. This means that at some as yet undecided point in the near future (i.e. fairly soon) we will move the repository away from Sourceforge and to this new machine. In the process we will also convert the current CVS repository to a new and more modern type, namely Mercurial (also known as hg). Mercurial is a so called distributed version control system (DVCS). This means that every developer has his/her own complete copy of the repository and can make any changes they want to it. In order to share any results, developers with the right access rights can "push" their changes to the central repository, and anyone can "pull" changes from that repository to their own. Mercurial (like its competitors git and bazaar) is very good at managing this push/pull work, merging changes from different sources and keeping track of what has actually changed (as opposed to what the end result looks like, which is what CVS and Subversion (SVN) do). Read-only access to the Mercurial repository will of course be possible for anyone. This only requires the HTTP protocol. Read-write access to the repository will require an SSH public/private key pair (of which you will need to keep the private part, well, private; the public part will need to be sent to me--I will contact you for that). Because Mercurial is distributed, we can work in a slightly different way than we used to with CVS. A group of core developers will have read-write access. They can make their copy of the repository (or one of their copies) available for their students who can develop and push their changes to this copy. Once the core developer is satisfied with the changes he/she can push those changes to the main repository. This way of working has several big advantages: - Any developer can store changes (with appropriate log messages) in their copy of the repository, safeguarding those changes, even if they are not yet fully developed. - Once changes are mature enough to share, they can be shared. - We don't need to give lots and lots of people write access to the main repository. Currently there are a large number of people with write access to the repository at Sourceforge of whom I don't know whether they still need the access. The new machine will do more than just serving the repository. It will also be used as the MonetDB webserver, and we have plans to switch over to another bug tracking system (bugzilla) that is to run on this system. An interesting intro into Mercurial usage can be found here: <http://hginit.com/00.html>. -- Sjoerd Mullender

1 0

[Monetdb-developers] locate_file and error handling
by Mark Bucciarelli 17 Mar '10

17 Mar '10

Hi, As I read the source, I see lots of instances where return values are not checked or not handled; for example, strdup(). Most recently, this programming idiom triggered a null pointer dereference, which caused mserver to segfault (see tracker #2971516 [1]). In my experience, when the logic encounters a condition it can't recover from, exiting quickly with as much information as possible is very useful. Whe I read locate_file() method and see: char *mod_path = GDKgetenv("monet_mod_path"); if (mod_path == NULL) return NULL; I wonder---how can you recover from this? I want to submit a patch that says: if (mod_path == NULL) errx(1, "locate_file: GDBgetenv("monet_mod_path") is null"); But I'm not sure you guys will accept this. Would you? It's a large change to error handling style. What happens to me in #2971516 is simply the library file name is not found. But instead of a useful error message that tells me what directories where searched and what the file name is, I get ERR control[14875]: (local): failed to fork mserver: database 'test' has inconsistent state (running but dead), review merovingian's logfile for any peculiarities and an mserver5.core file. Thanks, m [1] https://sourceforge.net/tracker/index.php?func=detail&aid=2971516&group_id=…

1 0

[Monetdb-developers] Fw: Re: Fw: Re: Fw: Re: Fw: Re: related to jdbc
by anshul gangwar 17 Mar '10

17 Mar '10

Yes we are on monetdb/xquery The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

3 2

[Monetdb-developers] Fw: Re: Fw: Re: Fw: Re: related to jdbc
by anshul gangwar 15 Mar '10

15 Mar '10

hello , actually we are working on java based tool whose main focus is to visualize the plans used by the optimizer for final execution of query but in monetdb optimization is runtime so we have to somehow access that plan through jdbc and what are the maincomponents of the plan i.e can be represented as node in tree We are not able to use that , actually that is giving error MAPI = monetdb@localhost:50000 ACTION= mapi_start_talking ERROR = !xchange_challenge: frontend mal not found Dear Anshul, Op 15-03-10 05:42, anshul gangwar schreef: > We want to debug the monetdb server , in other words we want final > execution plan and query execution times etc. Then most likely you will love the stethoscope tool! It gives an awesome view in the MonetDB kernel *and* it provides you realtime information about the queries that have been executed. That is not all, you can just start and stop it without(!) restarting the server, while still making nifty graphs from this program. I think this is totally what you need. There is a lot documentation written for this tool, so you don't have to be a rocket scientist to use it :) <http://monetdb.cwi.nl/MonetDB/Documentation/Stethoscope.html> The last line on the page will give you an example commandline. If you did not yet change your default monetdb password you can use this one: stethoscope -umonetdb -Pmonetdb +tis algebra.* bat.* group.* sql.* aggr.* Please report back how useful this was for your debuging and use of MonetDB. We appreciate your interest. Yours Sincerely, Stefan de Konink -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAkudwdMACgkQYH1+F2Rqwn0+AwCfTg5FruAc6QDgOIT4ZJZ4bOWB hTAAnjkWylxX7JafSLI21qZFE6BMydoO =8rz4 -----END PGP SIGNATURE----- Your Mail works best with the New Yahoo Optimized IE8. Get it NOW! http://downloads.yahoo.com/in/internetexplorer/

2 1

[Monetdb-developers] Fw: Re: Fw: Re: related to jdbc
by anshul gangwar 15 Mar '10

15 Mar '10

We want to debug the monetdb server , in other words we want final execution plan and query execution times etc. > We are working on linux so any way through jdbc Just as a sidenote: JDBC vs OBDC is not necessarily a question of the OS; both JDBC and ODBC are available and supported on various OSs, among which Windows and Linux (there is UnixODBC for Linux). > Can we get the debugger output through jdbc Which debugger and/or debugger output are you referring to / talking about? Or in other words, what do you want to debug? Your JDBC client? The JDBC driver? The MonetDB server? The MonetDB kernel? Stefan > You can enable the ODBC debug output by your ODBC configuration. > > For example: > > Trace = Yes > TraceFile = /tmp/debug.log > ForceTrace = Yes > > > Stefan > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.14 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iEYEAREKAAYFAkudHEYACgkQYH1+F2Rqwn2n1ACeJkxR8di3h1McltuRXsDZRxri > V9UAoIchyOqm2pqGDrCWITtWUJngAkhg > =Qz4/ > -----END PGP SIGNATURE----- > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Monetdb-developers mailing list > Monetdb-developers(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-developers > > > > The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/ > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Monetdb-developers mailing list > Monetdb-developers(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-developers -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 | The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

2 1

[Monetdb-developers] related to jdbc
by anshul gangwar 14 Mar '10

14 Mar '10

hello, Can we get the debugger output through jdbc Your Mail works best with the New Yahoo Optimized IE8. Get it NOW! http://downloads.yahoo.com/in/internetexplorer/

3 2

[Monetdb-developers] Fw: Re: related to jdbc
by anshul gangwar 14 Mar '10

14 Mar '10

We are working on linux so any way through jdbc Can we get the debugger output through jdbc You can enable the ODBC debug output by your ODBC configuration. For example: Trace = Yes TraceFile = /tmp/debug.log ForceTrace = Yes Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREKAAYFAkudHEYACgkQYH1+F2Rqwn2n1ACeJkxR8di3h1McltuRXsDZRxri V9UAoIchyOqm2pqGDrCWITtWUJngAkhg =Qz4/ -----END PGP SIGNATURE----- ------------------------------------------------------------------------------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev _______________________________________________ Monetdb-developers mailing list Monetdb-developers(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. http://in.yahoo.com/

2 1