Beste Harm,
you are running on Red Hat Enterprise Linux Server release 5.4, right?
Is your OS and hardware 64-bit or 32-bit?
Did you install pre-built binaries (RPMs) or did you compile MonetDB yourself? If the former, which RPMs did you get from where? (Official release or nightly build?) If the latter, did you compile from CVS sources or from a (released or nightly built?) super source ball, which configure setting did you use, and which compiler (incl. version) did you use?
Like Peter, I haven't yet managed to reproduce your problem with the given query. The aove information might help us to test in a setup that is as close to yours as possible ...
Stefan
On Mon, Dec 14, 2009 at 09:09:51AM +0100, Harm van Beek (DT) wrote:
Dear Peter,
Thanks for your reply!
I double-checked; the query sent earlier DOES run into the "too deep recursion" problem. One major difference with respect to earlier development is that we currently are using the RedHat Enterprise Linux (Tikanga) operation system on our servers. We know that this might cause the problem, however, as far as we can see the user limits match (or outnumber) the limits on the Fedora systems.
If you can point us where/how to check which limits are used in the calculation used for limiting recursion depth, we might be able to adapt the settings.
We installed the November release in our development environment and will soon start testing with it.
Kind regards,
Harm
-----Original Message----- From: Peter Boncz [mailto:P.Boncz@cwi.nl] Sent: Monday, December 14, 2009 12:25 AM To: Harm van Beek (DT) Cc: Communication channel for MonetDB users Subject: RE: [MonetDB-users] Fatal error: aborted too deep recursion
Hi Harm,
I had promised to get back to on this problem during the week; I did not quite make that, here I am trying to make what might be your definition of 'before the beginning of next week'. But apologies for the delay.
You provided a test query on which you saw the 'recursion' error occurring. We know this error can occur on certain very large queries (in the test web there is a query with 100 XPath steps which triggers it). As there are limits to any program, we accept that this bug will happen in very extreme cases. However, the query you provided did not seem very intimidating. Therefore it may be a serious issue.
However, your report so far has proven irreproducible for me.
I tried it on the latest release with the mclient utility to no avail. I also tried with the 'pf' program standalone, as well as with in-server (using the query wrapped in a 'xquery()' MIL command).
Are you sure this query triggers the error? Of course, I might go ahead and create an option for changing the recursion limit in the code. I held back from that because the risk is that instead of an error you might be getting a segmentation violation instead (after all, the fence is there for a reason). I could mitigate that risk by also giving the thread more stack space (does not work very easily on all architectures). Still, understanding the problem better will make the solution better, so getting this to reproduce would help.
Thanks in advance,
Peter
BTW did NFI try the Nov2009 release already? It is the one with the pf:cache pragmas that will help performance quite a bit.
I tested this with all 126 versions of MonetDB4 currently in Stefan's archive:
$ for i in `locate bin/pf | egrep -e 'pf$'`; do echo $i; $i /tmp/t 2>&1| fgrep ecurs; done /export/scratch0/manegold/BenQ/2008-05-15.F8/sda1/home/manegold /monetdb-curr ent/program-i686/bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda1/usr/bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda1/var/www/cgi-b in/MonetDB/bi n/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda2/Win/home/mane gold/Developm ent/Current/.GNU.32.32.d.--enable-debug.PREFIX./bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda2/Win/home/mane gold/Developm ent/Update/.GNU.32.32.d.--enable-debug.PREFIX./bin/pf /export/scratch0/manegold/BenQ/2008-05-15.F8/sda3/SuSE/FC5/home /manegold/mon etdb-current/program-i686/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP1/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Aug2009-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all/prefix.db g.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP1/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.opt.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.opt.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Feb2009-SuperB all-SP2/prefi x.opt.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.db g.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.op t.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.op t.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all/prefix.op t.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.opt.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.opt.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP1/prefi x.opt.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.opt.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.opt.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-May2009-SuperB all-SP2/prefi x.opt.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP1/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP1/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP2/prefi x.dbg.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP2/prefi x.dbg.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2008-SuperB all-SP2/prefi x.dbg.oid32/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2009-SuperB all/prefix.db g.32bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2009-SuperB all/prefix.db g.64bit/bin/pf /export/scratch0/manegold/Monet/Releases/MonetDB-Nov2009-SuperB all/prefix.db g.oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix./bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-bits~32/bi n/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-bits~32_-- enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-oid32/bin/ pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-oid32_--en able-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--disable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--disable-assert_CC~icc/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--disable-assert_CC~icc_--enable -probxml/bin/ pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-debug_--enable-optimize_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-optimize_--disable-debug_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-optimize_--disable-debug_--disable-assert_--enable-probx ml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-optimize_--enable-debug_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix.--enab le-strict_--d isable-optimize_--enable-debug_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Candidate/prefix._--ena ble-probxml/b in/pf /export/scratch0/manegold/Monet/Testing/Current/prefix./bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -bits~32/bin/ pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -bits~32.pf_M 5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -bits~32_--en able-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -oid32.pf_M5X Q/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -oid32_--enab le-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert_CC~icc/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert_CC~icc.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--disable-assert_CC~icc_--enable-p robxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--enable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-debug_--enable-optimize_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--disable-debug_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--disable-debug_--disable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--disable-debug_--disable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--enable-debug_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--enable-debug_--enable-assert.pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix.--enable -strict_--dis able-optimize_--enable-debug_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix..pf_M5XQ/bin/pf /export/scratch0/manegold/Monet/Testing/Current/prefix._--enabl e-probxml/bin /pf /export/scratch0/manegold/Monet/Testing/Stable/prefix./bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- bits~32/bin/p f /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- bits~32.pf_NF I/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- bits~32_--ena ble-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- oid32.pf_NFI/ bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- oid32_--enabl e-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_CC~icc/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_CC~icc.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--disable-assert_CC~icc_--enable-pr obxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-bits~32/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-bits~32_-- enable-probxm l/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-oid32/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-oid32_--en able-probxml/ bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-debug_--enable-optimize_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--disable-debug_--disable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--disable-debug_--disable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--disable-debug_--disable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--enable-debug_--enable-assert/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--enable-debug_--enable-assert.pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix.--enable- strict_--disa ble-optimize_--enable-debug_--enable-assert_--enable-probxml/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix..pf_NFI/bin/pf /export/scratch0/manegold/Monet/Testing/Stable/prefix._--enable -probxml/bin/ pf /export/scratch0/manegold/MonetDB_4-12/.GNU.64.64.d..PREFIX./PA THFINDER/bin/ pf /export/scratch0/manegold/MonetDB_Release_Build/prefix/bin/pf /export/scratch0/manegold/tmp/BenQ_FC6/home/manegold/monetdb-cu rrent/program -i686/bin/pf /export/scratch0/manegold/tmp/BenQ_FC6/var/www/cgi-bin/MonetDB/bin/pf /usr/bin/pf
-----Original Message----- From: Harm van Beek(DT) [mailto:harm@holmes.nl] Sent: vrijdag 27 november 2009 14:49 To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Hi Peter,
We indeed execute a large query (which is constructed by progra mcode, so hard to change). You can find the query at the bottom of my reply.
I understand that you introduced the limits to not run out of stack space. However, since we are in full control of the system on which the query is executed, we might be able to adapt the limits to our environment. Can you give us directions on where to set this?
Thanks again,
Harm
-- query:
element result { for $name in distinct-values( ( let $var1 := let $var2 := let $var3 := let $var4 := let $var5 := let $var6 := let $var7 := let $var8 := let $var9 := let $var10 := let $var11 := let $var12 := doc('document.xml')//properties/.. return $var12[ (self::registry) ] return $var11/ancestor-or-self::file[1]/properties/.. return $var10[ properties/(name)[ contains( lower-case(.), 'system' ) ] ] return $var9/child::registry/properties/.. return $var8/child::key/properties/.. return $var7/child::key/properties/.. return $var6/child::key/properties/.. return $var5[ properties/(name)[ . = 'Enum' ] ] return $var4/descendant-or-self::key/properties/.. return $var3[ properties/(name)[ contains( lower-case(.), 'usb' ) ] ] return $var2/descendant-or-self::value/properties/.. return $var1[ properties/(name)[ . = 'LocationInformation' ] ] )/properties/*/name() ) return element name { $name } }
-----Original Message----- From: Peter Boncz [mailto:P.Boncz@cwi.nl] Sent: Friday, November 27, 2009 10:26 AM To: 'Communication channel for MonetDB users' Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Hi Harm,
The 'too deep recursion' error is typically triggered by very large queries that send the pathfinder compiler into a very deep (and time consuming) search. The error is given to prevent a crash due to lack of stack space. Recently, I do recall that the limits were tightened as there are some operating systems in our nightly test farm that give their processes little stack space, and could still cause pathfinder to crash due to lack of stack space.
So, it depends. The limit when to break off search is arbitrary and based currently on the lowest common denominator among the systems we used. We could potentially increase it (and possibly fix the settings on our testing machines to allow more stack space). We could even introduce a user-controller limit.
However, this phenomenon typically happens only on strange queries (e.fg doing tens or hundreds XPath steps after each other).
Could you share what query produced such an error for you?
Best,
Peter Boncz
-----Original Message----- From: Martin Kersten [mailto:Martin.Kersten@cwi.nl] Sent: Thursday, November 26, 2009 4:28 PM To: Communication channel for MonetDB users Subject: Re: [MonetDB-users] Fatal error: aborted too deep recursion
Harm van Beek(DT) wrote:
During the development of our tool, using Xquery on
MonetDB, we ran onto
the following error:
ERROR = !fatal error: aborted too deep recursion
From previous posts (I guess) I can conclude that this
relates to an
Xquery that simply is too big (too many operators) to
create a good plan
for it.
All systems have limits that either cause an emergency stop.
In this case,
the operating system has such a limit.
I tried several MonetDB releases (May2009-SP2,
Aug2009-SP2, Nov2009-DB0)
on Red Hat Enterprise Linux Server release 5.4 (Tikanga),
all showing
the same result.
Does anyone have any directions on how to solve this?
Yes. Programming cleverness comes into play. Especially if
you use a tool
to generate queries, then that tool should be a little more
conservative
in the kind of queries generated.
Can we reconfigure the compiler to fix this (or recompile it with options)?
No. At some point the optimization space becomes just too big. People that can spare the time to find the optimal settings
for a larger
application space are more then welcome to try it out on the
source code
made available.
regards, Martin
Thanks in advance!
Harm van Beek Netherlands Forensic Institute
Let Crystal Reports handle the reporting - Free Crystal
Reports 2008 30-
Day
trial. Simplify your report design, integration and
deployment - and
focus on
what you do best, core application coding. Discover
what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
Let Crystal Reports handle the reporting - Free Crystal
Reports 2008 30-Day
trial. Simplify your report design, integration and
deployment - and focus
on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.426 / Virus Database: 270.14.83/2526 - Release
Date: 11/25/09
19:43:00
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment
- and focus on
what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
-- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment
- and focus
on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.426 / Virus Database: 270.14.83/2526 - Release Date: 11/26/09 19:42:00
Return on Information: Google Enterprise Search pays you back Get the facts. http://p.sf.net/sfu/google-dev2dev _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users