[Monetdb-developers] [Monetdb-pf-checkins] pathfinder configure.ag, 1.91, 1.92

Peter Boncz boncz at cwi.nl
Mon Jan 22 14:24:06 CET 2007


Hi Jan,

Thanks for the clarifications.

However, from what you tell me know, I do wonder whether it would not be a
better idea to go for the sandbox approach. For me, it is not an issue of
performance, but of dependability.

Given this choice, maybe we should exctually decouple the pathfinder
compiler altogether from the runtime, and fork off a process. That also
resolves the MT safeness issue.

Peter


-----Original Message-----
From: Jan Rittinger [mailto:rittinge at in.tum.de]
Sent: 22 January 2007 14:04
To: Peter Boncz
Cc: monetdb-developers at lists.sourceforge.net
Subject: Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder
configure.ag, 1.91, 1.92



On 01/22/2007 01:43 PM, Peter Boncz wrote with possible deletions:
> Hi Jan,
>
> As I have indicated, use of the Boehm garbage collector inside the MonetDB
> process is not really an option.
>
> If algebra plans can only be compiled with Boehm, than it will become very
> difficult to use pathfinder inside MonetDB/XQuery.

As I mentioned the embedded variant uses the old memory allocation
routine. It just will run out of memory for more complex queries. (But
applying less optimization phases will certainly avoid many memory
allocation problems).

>
> Did you try the option with memory sandboxes that at certain critical
stages
> can be cleared?

Yes, of course. In the check-in message I mentioned that the sandbox
idea works but is way slower than the Boehm GC (7min vs 3min -- and yes
I know it takes some time to compile the query :)).

Furthermore it requires a lot of changes in various places to ensure
that we address the correct sandbox and to make sure that we do not
throw away too much/less. This was one of the main reasons why we
decided to use the garbage collector instead.

Jan

>
> Peter
>
> -----Original Message-----
> From: monetdb-pf-checkins-bounces at lists.sourceforge.net
> [mailto:monetdb-pf-checkins-bounces at lists.sourceforge.net]On Behalf Of
> Jan Rittinger
> Sent: 22 January 2007 12:26
> To: monetdb-pf-checkins at lists.sourceforge.net
> Subject: [Monetdb-pf-checkins] pathfinder configure.ag,1.91,1.92
>
>
> Update of /cvsroot/monetdb/pathfinder
> In directory sc8-pr-cvs7.sourceforge.net:/tmp/cvs-serv7979
>
> Modified Files:
> 	configure.ag
> Log Message:
> -- Re-integrated Hans Boehms garbage collection library.
>
>    * This was necessary because more complex queries did run out of
memory.
>      (Before we did allocate memory and only freed it at the end. Snapshot
>      based freeing -- meaning after a full tree copy was made -- did help
>      but was a lot slower than using the garbage collection library.)
>
>    * The old memory allocation (without garbage collection) is still in
use
>      whenever pathfinder is used in embedded mode (e.g. if the xquery()
>      command in MonetDB is used).
>
>    * If the garbage collection library is not detected during configure
>      we avoid to generate an error and use the old memory allocation
> instead.
>
>    * The drawback of this change is that the compilation of simple queries
>      is now lot slower (e.g. XMark Q6 takes 400ms to compile).
>
>
>
> Index: configure.ag
> ===================================================================
> RCS file: /cvsroot/monetdb/pathfinder/configure.ag,v
> retrieving revision 1.91
> retrieving revision 1.92
> diff -u -d -r1.91 -r1.92
> --- configure.ag	21 Jan 2007 16:01:38 -0000	1.91
> +++ configure.ag	22 Jan 2007 11:25:59 -0000	1.92
> @@ -334,6 +334,70 @@
>  AC_SUBST(LIBXML2_LDADD, $LIBXML2_LDADD)
>  AM_CONDITIONAL(HAVE_LIBXML2, test x"$have_libxml2" != xno)
>
> +dnl See if we have the Hans Boehm garbage collector available
> +dnl
> +dnl First allow the user to specify a location on the command line.
> +dnl Further down see if we find the library and abort if not.
> +have_gc="auto"
> +AC_ARG_WITH(gc,
> +    [  --with-gc=DIR           Boehm garbage collector library is
installed
> in DIR],
> +    [have_gc="$withval"], [have_gc="auto"])
> +
> +if test "x$have_gc" != xauto; then
> +        LIBGC_CFLAGS="-I$have_gc/include"
> +        LIBGC_LIBS="-L$have_gc/lib"
> +fi
> +
> +if test "x$have_gc" = xauto; then
> +        LIBGC_CFLAGS=""
> +        LIBGC_LIBS=""
> +fi
> +
> +save_CPPFLAGS="$CPPFLAGS"
> +save_LDFLAGS="$LDFLAGS"
> +save_LIBS="$LIBS"
> +CPPFLAGS="$CPPFLAGS $LIBGC_CFLAGS"
> +LDFLAGS="$LDFLAGS $LIBGC_LIBS"
> +LIBS=""
> +
> +dnl If we have the dl and/or the pthread library, use it
> +dnl (dl and/or pthread are required for the garbage collection
> +dnl library on some systems.)
> +LIBGC_LDADD=""
> +AC_CHECK_LIB(dl, dlopen, [LIBGC_LDADD="$LIBGC_LDADD -ldl"])
> +AC_CHECK_LIB(pthread, pthread_create,
> [LIBGC_LDADD="$LIBGC_LDADD -lpthread"])
> +
> +AC_CHECK_HEADER([gc.h],
> +    [ dnl See if we can link in the gc library.
> +      dnl If we have found the dl and/or pthread library as well, also
use
> it.
> +      dnl If the gc library was not found, clear LIBGC_LDADD.
> +      AC_CHECK_LIB(gc, GC_malloc,
> +                   [LIBGC_LDADD="$LIBGC_LDADD -lgc"],
> +                   [LIBGC_LDADD=""], [$LIBGC_LDADD])],
> +    [ LIBGC_LDADD=""])
> +
> +
> +LIBS="$save_LIBS"
> +LDFLAGS="$save_LDFLAGS"
> +CPPFLAGS="$save_CPPFLAGS"
> +
> +if test "x$LIBGC_LDADD" = "x"; then
> +    AC_MSG_WARN([Unable to find the Hans Boehm garbage collector.
> +                 Please go get it from
> +                 http://www.hpl.hp.com/personal/Hans_Boehm/gc/
> +                 (Compilation will use memory allocation only strategy
> instead.)])
> +    AC_DEFINE(HAVE_GC, 0, [Define if you use garbage collection])
> +fi
> +if test "x$LIBGC_LDADD" != "x"; then
> +    AC_DEFINE(HAVE_GC, 1, [Define if you use garbage collection])
> +fi
> +
> +
> +dnl Propagate the variable to the Makefiles
> +AC_SUBST(LIBGC_CFLAGS, $LIBGC_CFLAGS)
> +AC_SUBST(LIBGC_LIBS, $LIBGC_LIBS)
> +AC_SUBST(LIBGC_LDADD, $LIBGC_LDADD)
> +
>  # have_pftijah=no
>  # AM_CONDITIONAL(HAVE_PFTIJAH, test "x$have_pftijah" == xyes)
>
>
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share
your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Monetdb-pf-checkins mailing list
> Monetdb-pf-checkins at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins
>
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share
your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Monetdb-developers mailing list
> Monetdb-developers at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/monetdb-developers
>

--
Jan Rittinger
Database Systems
Technische Universität München (Germany)
http://www-db.in.tum.de/~rittinge/





More information about the developers-list mailing list