July 2008 - developers-list

[Monetdb-developers] Monet-in-a-File
by Stefan de Konink 19 Jul '08

19 Jul '08

Hi, By a thought that I have expressed to Niels as alternative for [the-idea-that-i'm-not-going-mention-here]. I was wondering how much work it would take to make MonetDB to work directly on fixed size files. I'm not going into details but anyone with a bit computer experience would know what fixed size files are in an operation system perspective. Would it be feasible to memory map directly to a disk instead, or would this potentially harm the performance with respect to transactions? With the storage method that I have in mind the backend would make absolutely sure the file is stored bit correct. Stefan ps. After writing this mail I have realized this could be applied on the two different projectes we discussed.

1 0

Re: [Monetdb-developers] tables/databases simultaneous updates & extended-stored-procedures
by SOL ZADEH 19 Jul '08

19 Jul '08

Thanks Martin; let me explain better: if table update are too costly (slow etc) then could try insertion instead (having/including the timestamps as an index) providing the server can cope with very frequent table insertions. Is there any benchmark tests in relation to table insertions? Regarding : >> Concurrent transactions can freely read a consistent copy of the database. does this mean "dirty reads" ? Regards ----- Original Message ---- From: Martin Kersten <Martin.Kersten(a)cwi.nl> To: SOL ZADEH <solzadeh(a)yahoo.com> Cc: Niels Nes <Niels.Nes(a)cwi.nl>; monetdb-developers(a)lists.sourceforge.net Sent: Saturday, July 19, 2008 2:22:16 PM Subject: Re: [Monetdb-developers] tables/databases simultaneous updates & extended-stored-procedures SOL ZADEH wrote: > > Thanks Neils for your prompt reply. > > Is there a quantified limitation on "insertions" on scenarios I > mentioned below (multiple tables/database)? > I.e. Does very frequent insertions on 2 or more tables (on same > database or other database on the same server) > in any way effect the performance of the other queries > (select,insert,update) on the "same" > or "other" tables (i.e. is parallel processing possible & locking > performance etc). > Regards Trying to interpret you question what it means. MonetDB does not lock tables or databases during update. Concurrent transactions can freely read a consistent copy of the database. All interference comes from sharing the same CPU, memory and IO channels. > > ----- Original Message ---- > From: Niels Nes <Niels.Nes(a)cwi.nl> > To: SOL ZADEH <solzadeh(a)yahoo.com> > Cc: monetdb-developers(a)lists.sourceforge.net > Sent: Saturday, July 19, 2008 8:49:55 AM > Subject: Re: [Monetdb-developers] tables/databases simultaneous > updates & extended-stored-procedures > > On Fri, Jul 18, 2008 at 05:13:54PM -0700, SOL ZADEH wrote: > > Hi > > > > > > I have a couple of questions and really appreciate if any one > > could answer them or instruct me to the relevant forum topics: > > > > > > 1- what is the limitations (speed, threshold etc) on > > simultaneous tables /databases update i.e. via multiple odbc > > connection lets say (also any other faster API to interface > > with the MonetDB ?) > MonetDB research concentrated mostly in low update/high query loads. The > update implementation is still quite capable, but no reall performance > experiments on updates have been conducted. Next to odbc we have > mapi and jdbc APIs available. > > > > > > 2- Is there a way to have the triggers to generate > > real-world (non-db) events i.e. similar to the Extended Stored > > Procedure in SqlServer or the way Oracle allows you to run Java > > code inside the stored procedures. > MonetDB can be extended using C-code and that code could be run within > the triggers. > > Niels > > > > Thanks > > > > ------------------------------------------------------------------------- > > This SF.Net email is sponsored by the Moblin Your Move Developer's > challenge > > Build the coolest Linux based applications with Moblin SDK & win > great prizes > > Grand prize is a trip for two to an Open Source event anywhere in > the world > > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > <http://moblin-contest.org/redirect.php?banner_id=100&url=/> > > _______________________________________________ > > Monetdb-developers mailing list > > Monetdb-developers(a)lists.sourceforge.net > <mailto:Monetdb-developers@lists.sourceforge.net> > > https://lists.sourceforge.net/lists/listinfo/monetdb-developers > > > -- > > Niels Nes, Centre for Mathematics and Computer Science (CWI) > Kruislaan 413, 1098 SJ Amsterdam, The Netherlands > room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312 > url: http://www.cwi.nl/~niels <http://www.cwi.nl/%7Eniels> e-mail: > Niels.Nes(a)cwi.nl <mailto:Niels.Nes@cwi.nl> > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > ------------------------------------------------------------------------ > > _______________________________________________ > Monetdb-developers mailing list > Monetdb-developers(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-developers >

1 0

Re: [Monetdb-developers] tables/databases simultaneous updates & extended-stored-procedures
by SOL ZADEH 19 Jul '08

19 Jul '08

Thanks Neils for your prompt reply. Is there a quantified limitation on "insertions" on scenarios I mentioned below (multiple tables/database)? I.e. Does very frequent insertions on 2 or more tables (on same database or other database on the same server) in any way effect the performance of the other queries (select,insert,update) on the "same" or "other" tables (i.e. is parallel processing possible & locking performance etc). Regards ----- Original Message ---- From: Niels Nes <Niels.Nes(a)cwi.nl> To: SOL ZADEH <solzadeh(a)yahoo.com> Cc: monetdb-developers(a)lists.sourceforge.net Sent: Saturday, July 19, 2008 8:49:55 AM Subject: Re: [Monetdb-developers] tables/databases simultaneous updates & extended-stored-procedures On Fri, Jul 18, 2008 at 05:13:54PM -0700, SOL ZADEH wrote: > Hi > > > I have a couple of questions and really appreciate if any one > could answer them or instruct me to the relevant forum topics: > > > 1- what is the limitations (speed, threshold etc) on > simultaneous tables /databases update i.e. via multiple odbc > connection lets say (also any other faster API to interface > with the MonetDB ?) MonetDB research concentrated mostly in low update/high query loads. The update implementation is still quite capable, but no reall performance experiments on updates have been conducted. Next to odbc we have mapi and jdbc APIs available. > > > 2- Is there a way to have the triggers to generate > real-world (non-db) events i.e. similar to the Extended Stored > Procedure in SqlServer or the way Oracle allows you to run Java > code inside the stored procedures. MonetDB can be extended using C-code and that code could be run within the triggers. Niels > > Thanks > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > Monetdb-developers mailing list > Monetdb-developers(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-developers -- Niels Nes, Centre for Mathematics and Computer Science (CWI) Kruislaan 413, 1098 SJ Amsterdam, The Netherlands room C0.02, phone ++31 20 592-4098, fax ++31 20 592-4312 url: http://www.cwi.nl/~niels e-mail: Niels.Nes(a)cwi.nl

2 1

[Monetdb-developers] tables/databases simultaneous updates & extended-stored-procedures
by SOL ZADEH 19 Jul '08

19 Jul '08

Hi I have a couple of questions and really appreciate if any one could answer them or instruct me to the relevant forum topics: 1- what is the limitations (speed, threshold etc) on simultaneous tables /databases update i.e. via multiple odbc connection lets say (also any other faster API to interface with the MonetDB ?) 2- Is there a way to have the triggers to generate real-world (non-db) events i.e. similar to the Extended Stored Procedure in SqlServer or the way Oracle allows you to run Java code inside the stored procedures. Thanks

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime Makefile.ag, , 1.91, 1.92
by Ying Zhang 18 Jul '08

18 Jul '08

On Fri, Jul 18, 2008 at 01:25:23PM +0000, Sjoerd Mullender wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv28620 > > Modified Files: > Makefile.ag > Log Message: > XRPC server needs TYPE_logger. > Now it compiles on Windows (64 bit). Thanks Sjoerd! > > > U Makefile.ag > Index: Makefile.ag > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/Makefile.ag,v > retrieving revision 1.91 > retrieving revision 1.92 > diff -u -d -r1.91 -r1.92 > --- Makefile.ag 17 Jul 2008 21:28:28 -0000 1.91 > +++ Makefile.ag 18 Jul 2008 13:25:21 -0000 1.92 > @@ -124,7 +124,7 @@ > ./lib_pf_support ./lib_pathfinder \ > $(PF_LIBS) $(SOCKET_LIBS) \ > $(MONETDB_LIBS) -lbat -lstream $(MONETDB4_LIBS) -lmonet $(PTHREAD_LIBS) \ > - $(MONETDB4_MODS) -l_mapi -l_streams -l_builtin -l_ascii_io > + $(MONETDB4_MODS) -l_logger -l_mapi -l_streams -l_builtin -l_ascii_io > } > > headers_mil = { > > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins

1 0

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB4/src/mapi/Tests All, , 1.2, 1.3
by Stefan Manegold 18 Jul '08

18 Jul '08

On Fri, Jul 18, 2008 at 11:46:19AM +0000, Sjoerd Mullender wrote: > Update of /cvsroot/monetdb/MonetDB4/src/mapi/Tests > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv19788 > > Modified Files: > All > Log Message: > Test can only be run if compiled with HAVE_PYTHON (else Mapi.py does > not get installed). > This reminds me of a "minor"(?) flaw in Mtest.py: Mapi.py gets installed in case clients was configured with python avialable, but Mtest.py set "HAVE_PYTHON" if MonetDB was configured with python avialable. Hence, in case clients and MonetDB are not configured equally, the conditional does not reflect the actual status. The same hold for all static/compile-time conditionals currently supported by Mtest.py. To solve this problem, we could consider deriving all conditionals only at runtime as Mtest.py already does, e.g., for all MonetDB/Java related conditional. ... once we will have some spare time ... Stefan > > U All > Index: All > =================================================================== > RCS file: /cvsroot/monetdb/MonetDB4/src/mapi/Tests/All,v > retrieving revision 1.2 > retrieving revision 1.3 > diff -u -d -r1.2 -r1.3 > --- All 12 Jan 2007 17:19:23 -0000 1.2 > +++ All 18 Jul 2008 11:46:14 -0000 1.3 > @@ -1,3 +1,3 @@ > monet_BATs > -manyMapiClients > +HAVE_PYTHON?manyMapiClients > utf_8_C > > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > Monetdb-checkins mailing list > Monetdb-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-checkins > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

1 0

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime xrpc_const.mx, , NONE, 1.1 Makefile.ag, , 1.89, 1.90 serialize_dflt.mx, , 1.49, 1.50 xrpc_client.mx, , 1.49, 1.50 xrpc_common.mx, , 1.6, 1.7 xrpc_server.mx, , 1.80, 1.81
by Stefan Manegold 17 Jul '08

17 Jul '08

On Thu, Jul 17, 2008 at 04:42:42PM +0000, Ying Zhang wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv1165 > > Modified Files: > Makefile.ag serialize_dflt.mx xrpc_client.mx xrpc_common.mx > xrpc_server.mx > Added Files: > xrpc_const.mx > Log Message: > Since xrpc_server and xrpc_client are not independent of each other (the > server needs the client to handle nested xrpc queries), better put them > into one lib. This way, we also get rid of the problem that > xrpc_common.mx is included in two libs. The "xrpc_server" & "xrpc_client" used to be not only "libs" but also *modules*. Replacing two modules xrpc_server & xrpc_client by one module "xrpc" requires some more work: ======== $ Mserver --dbinit='module(pathfinder);' !WARNING: GDKlockHome: created directory /ufs/manegold/_/scratch0/Monet/Testing/Current/prefix.--enable-strict_--disable-optimize_--enable-debug_--enable-assert/var/MonetDB4/dbfarm/demo/ !WARNING: GDKlockHome: ignoring empty or invalid .gdk_lock. !WARNING: BBPdir: initializing BBP. # MonetDB Server v4.25.0 # based on GDK v1.25.0 # Copyright (c) 1993-2008, CWI. All rights reserved. # Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs; dynamically linked. # Visit http://monetdb.cwi.nl/ for further information. !ERROR: TBL_openmod: DL_open: library not found. !ERROR: TBL_loadmod: module load _xrpc_server cancelled. !ERROR: moduleClient: module(_xrpc_server) load error. !ERROR: TBL_openmod: DL_open: library not found. !ERROR: TBL_loadmod: module load _xrpc_client cancelled. !ERROR: moduleClient: module(_xrpc_client) load error. # PF/Tijah module v0.5.0 loaded. http://dbappl.cs.utwente.nl/pftijah !ERROR: interpret_params: append(param 1): invalid BAT. MonetDB> ======== Stefan > Also separate xrpc constant strings into a new header file, so that > serialize_dflt kan include it. > > > > U xrpc_common.mx > Index: xrpc_common.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/xrpc_common.mx,v > retrieving revision 1.6 > retrieving revision 1.7 > diff -u -d -r1.6 -r1.7 > --- xrpc_common.mx 17 Jul 2008 14:56:19 -0000 1.6 > +++ xrpc_common.mx 17 Jul 2008 16:42:38 -0000 1.7 > @@ -24,134 +24,12 @@ > @' $Id$ > @f xrpc_common > @a Ying Zhang > -@t Includes header files, definitions shared by the XRPC server, XRPC client and serializer > +@t Includes struct and functions shared by the XRPC server and XRPC client > > @h > #ifndef XRPC_COMMON_H > #define XRPC_COMMON_H > > -/* HTTP defines */ > -#define XRPC_DOC_CALLBACK "/xrpc/doc" > -#define XRPC_ADM_CALLBACK "/xrpc/admin" > -#define XRPC_WSAT_CALLBACK "/xrpc/wsat" > -#define XRPC_REQ_CALLBACK "/xrpc" > - > -#define ERR403 "403 Forbidden" > -#define ERR404 "404 Bad Request" > -#define ERR408 "408 Request Timeout" > -#define ERR500 "500 Internal Server Error" > -#define ERR504 "504 Gateway Timeout" > - > -#define OUT_OF_MEM "Internal Receiver Error: out-of memory" > -#define NOT_WELL_FORMED "Request XML message not well-formed" > -#define MAX_NR_PARAMS 1024 > - > -#define HTTP_200_OK\ > - "HTTP/1.1 200 OK\r\n"\ > - "Content-type: text/xml; charset=\"utf-8\"\r\n\r\n" > - > -/* Namespaces */ > -#define MXQ_ADMIN "http://monetdb.cwi.nl/XQuery/admin/" > -#define SOAP_NS "http://www.w3.org/2003/05/soap-envelope" > -#define XDT_NS "http://www.w3.org/2005/xpath-datatypes" > -#define XS_NS "http://www.w3.org/2001/XMLSchema" > -#define XSI_NS "http://www.w3.org/2001/XMLSchema-instance" > -#define XRPC_NS "http://monetdb.cwi.nl/XQuery" > -#define XRPC_LOC "http://monetdb.cwi.nl/XQuery/XRPC.xsd" > -#define WSCOOR_NS "http://docs.oasis-open.org/ws-tx/wscoor/2006/06" > -#define WSAT_NS "http://docs.oasis-open.org/ws-tx/wsat/2006/06" > - > -#define XRPC_REQ_QN XRPC_NS"|request" > -#define XRPC_RES_QN XRPC_NS"|response" > - > -/* XRPC SOAP snippets */ > -#define SOAP_ENVELOPE\ > - "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n"\ > - "<env:Envelope\n"\ > - " xmlns:env=\"" SOAP_NS "\"\n"\ > - " xmlns:xrpc=\"" XRPC_NS "\"\n"\ > - " xmlns:xdt=\"" XDT_NS "\" \n"\ > - " xmlns:xs=\"" XS_NS "\"\n"\ > - " xmlns:xsi=\"" XSI_NS "\"\n"\ > - " xsi:schemaLocation=\"" XRPC_NS " " XRPC_LOC "\">\n" > - > -#define XRPC_WS_QID\ > - " <xrpc:header xrpc:mode=\"%s\">\n"\ > - " <wscoor:CoordinationContext\n"\ > - " xmlns:wscoor=\""WSCOOR_NS"\"\n"\ > - " env:mustUnderstand=\"true\">\n"\ > - " <wscoor:Identifier>%s</wscoor:Identifier>\n"\ > - " <wscoor:Expires>"LLFMT"</wscoor:Expires>\n"\ > - " <wscoor:CoordinationType>" WSAT_NS "</wscoor:CoordinationType>\n"\ > - " </wscoor:CoordinationContext>\n"\ > - " </xrpc:header>\n" > - > -#define XRPC_REQ_BODY_OPEN\ > - "<env:Body>\n"\ > - " <xrpc:request\n"\ > - " xrpc:module=\"%s\"\n"\ > - " xrpc:location=\"%s\"\n"\ > - " xrpc:method=\"%s\"\n"\ > - " xrpc:arity=\""LLFMT"\"\n"\ > - " xrpc:iter-count=\""LLFMT"\"\n"\ > - " xrpc:updCall=\"%s\"\n"\ > - " xrpc:caller=\"" > -#define XRPC_REQ_BODY\ > - XRPC_REQ_BODY_OPEN "%s:"LLFMT"\">\n" > -#define XRPC_REQ_BODY_FIRST_CALLER\ > - XRPC_REQ_BODY_OPEN "query\">\n" > - > -#define XRPC_WSAT_REQ SOAP_ENVELOPE\ > - "<env:Header>"\ > - XRPC_WS_QID \ > - "</env:Header>\n"\ > - "<env:Body><wsat:%s/></env:Body>\n"\ > - "</env:Envelope>" > - > -#define XRPC_WSAT_RES SOAP_ENVELOPE\ > - "<env:Header>\n"\ > - XRPC_WS_QID \ > - " <xrpc:partipant>%s,%s:%s:%s,"LLFMT",%d</xrpc:participant>\n"\ > - "</env:Header>\n"\ > - "<env:Body><wsat:%s/></env:Body>\n"\ > - "</env:Envelope>" > - > -#define XRPC_RES_BODY\ > - "<env:Body>\n"\ > - " <xrpc:response\n"\ > - " xrpc:module=\"%s\"\n"\ > - " xrpc:method=\"%s\">\n" > - > -#define XRPC_HTTP_CALL\ > - " <xrpc:call>"\ > - " <xrpc:sequence>"\ > - " <xrpc:atomic-value xsi:type=\"xs:string\">%s</xrpc:atomic-value>"\ > - " </xrpc:sequence>"\ > - " </xrpc:call>" > - > -#define XRPC_PUT_CALL\ > - " <xrpc:call>"\ > - " <xrpc:sequence>"\ > - " <xrpc:atomic-value xsi:type=\"xs:string\">%s</xrpc:atomic-value>"\ > - " </xrpc:sequence>"\ > - " <xrpc:sequence>"\ > - " <xrpc:element>%s</xrpc:element>"\ > - " </xrpc:sequence>"\ > - " </xrpc:call>" > - > -#define XRPC_REQ_FOOTER\ > - " </xrpc:request>\n"\ > - "</env:Body>\n"\ > - "</env:Envelope>" > - > -#ifndef HOST_NAME_MAX > -#ifdef _POSIX_HOST_NAME_MAX > -#define HOST_NAME_MAX _POSIX_HOST_NAME_MAX > -#else > -#define HOST_NAME_MAX 255 > -#endif > -#endif > - > typedef struct { > char *qid; > char *caller; > @@ -188,6 +66,7 @@ > #include "pf_config.h" > #include "pathfinder.h" > #include "xrpc_common.h" > +#include "xrpc_const.h" > > XRPCreq_t * > XRPCreq_new( > > U xrpc_client.mx > Index: xrpc_client.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/xrpc_client.mx,v > retrieving revision 1.49 > retrieving revision 1.50 > diff -u -d -r1.49 -r1.50 > --- xrpc_client.mx 17 Jul 2008 14:56:19 -0000 1.49 > +++ xrpc_client.mx 17 Jul 2008 16:42:38 -0000 1.50 > @@ -398,6 +398,7 @@ > #include "shredder.h" > #include "serialize.h" > #include "xrpc_common.h" > +#include "xrpc_const.h" > #include "xrpc_client.h" > > int > > U serialize_dflt.mx > Index: serialize_dflt.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/serialize_dflt.mx,v > retrieving revision 1.49 > retrieving revision 1.50 > diff -u -d -r1.49 -r1.50 > --- serialize_dflt.mx 17 Jul 2008 14:56:19 -0000 1.49 > +++ serialize_dflt.mx 17 Jul 2008 16:42:38 -0000 1.50 > @@ -41,7 +41,7 @@ > #include "serialize_null.h" > #include "pathfinder.h" > #include "pf_support.h" > -#include "xrpc_common.h" /* *_NS defs */ > +#include "xrpc_const.h" > > /* a lot of characters, static strings, and their sizes > (the idea is to reuse constant character pointers during > > U Makefile.ag > Index: Makefile.ag > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/Makefile.ag,v > retrieving revision 1.89 > retrieving revision 1.90 > diff -u -d -r1.89 -r1.90 > --- Makefile.ag 15 Jul 2008 14:14:15 -0000 1.89 > +++ Makefile.ag 17 Jul 2008 16:42:38 -0000 1.90 > @@ -49,11 +49,6 @@ > ll_sibling.mx > } > > -lib_xrpc_common = { > - NOINST > - SOURCES = xrpc_common.mx > -} > - > lib__pf_support = { > DIR = libdir/MonetDB4 > SOURCES = \ > @@ -103,25 +98,16 @@ > $(MONETDB4_MODS) -l_logger -l_streams -l_builtin -l_ascii_io -l_algebra -l_sys -l_constant -l_mapi > } > > -lib__xrpc_client = { > - DIR = libdir/MonetDB4 > - SOURCES = xrpc_client.mx > - LIBS = \ > - libserialize libxrpc_common \ > - ./lib_pf_support ./lib_pathfinder \ > - $(PF_LIBS) $(SOCKET_LIBS) \ > - $(MONETDB_LIBS) -lbat -lstream $(MONETDB4_LIBS) -lmonet \ > - $(MONETDB4_MODS) -l_streams -l_builtin -l_ascii_io > -} > - > -lib__xrpc_server = { > +lib__xrpc = { > DIR = libdir/MonetDB4 > SOURCES = \ > + xrpc_const.mx \ > + xrpc_common.mx \ > shttpd.c shttpd.h \ > - xrpc_server.mx > + xrpc_server.mx \ > + xrpc_client.mx > LIBS = \ > - libxrpc_common \ > - ./lib_pf_support ./lib_pathfinder \ > + libserialize ./lib_pf_support ./lib_pathfinder \ > $(PF_LIBS) $(SOCKET_LIBS) \ > $(MONETDB_LIBS) -lbat -lstream $(MONETDB4_LIBS) -lmonet $(PTHREAD_LIBS) \ > $(MONETDB4_MODS) -l_mapi -l_streams -l_builtin -l_ascii_io > > U xrpc_server.mx > Index: xrpc_server.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/xrpc_server.mx,v > retrieving revision 1.80 > retrieving revision 1.81 > diff -u -d -r1.80 -r1.81 > --- xrpc_server.mx 17 Jul 2008 15:54:23 -0000 1.80 > +++ xrpc_server.mx 17 Jul 2008 16:42:39 -0000 1.81 > @@ -177,6 +177,7 @@ > #include "serialize.h" > #include "shttpd.h" > #include "xrpc_common.h" > +#include "xrpc_const.h" > > static int rpcd_running = 0; > static int timing = 0; > > --- NEW FILE: xrpc_const.mx --- > @' Copyright Notice: > @' ----------------- > @' > @' The contents of this file are subject to the Pathfinder Public License > @' Version 1.1 (the "License"); you may not use this file except in > @' compliance with the License. You may obtain a copy of the License at > @' http://monetdb.cwi.nl/Legal/PathfinderLicense-1.1.html > @' > @' Software distributed under the License is distributed on an "AS IS" > @' basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See > @' the License for the specific language governing rights and limitations > @' under the License. > @' > @' The Original Code is the Pathfinder system. > @' > @' The Original Code has initially been developed by the Database & > @' Information Systems Group at the University of Konstanz, Germany and > @' is now maintained by the Database Systems Group at the Technische > @' Universitaet Muenchen, Germany. Portions created by the University of > @' Konstanz and the Technische Universitaet Muenchen are Copyright (C) > @' 2000-2005 University of Konstanz and (C) 2005-2008 Technische > @' Universitaet Muenchen, respectively. All Rights Reserved. > @' > @' $Id: xrpc_const.mx,v 1.1 2008/07/17 16:42:39 yingying Exp $ > @f xrpc_const > @a Ying Zhang > @t Defines XRPC string constants > > @h > #ifndef XRPC_CONST_H > #define XRPC_CONST_H > > /* HTTP defines */ > #define XRPC_DOC_CALLBACK "/xrpc/doc" > #define XRPC_ADM_CALLBACK "/xrpc/admin" > #define XRPC_WSAT_CALLBACK "/xrpc/wsat" > #define XRPC_REQ_CALLBACK "/xrpc" > > #define ERR403 "403 Forbidden" > #define ERR404 "404 Bad Request" > #define ERR408 "408 Request Timeout" > #define ERR500 "500 Internal Server Error" > #define ERR504 "504 Gateway Timeout" > > #define OUT_OF_MEM "Internal Receiver Error: out-of memory" > #define NOT_WELL_FORMED "Request XML message not well-formed" > #define MAX_NR_PARAMS 1024 > > #define HTTP_200_OK\ > "HTTP/1.1 200 OK\r\n"\ > "Content-type: text/xml; charset=\"utf-8\"\r\n\r\n" > > /* Namespaces */ > #define MXQ_ADMIN "http://monetdb.cwi.nl/XQuery/admin/" > #define SOAP_NS "http://www.w3.org/2003/05/soap-envelope" > #define XDT_NS "http://www.w3.org/2005/xpath-datatypes" > #define XS_NS "http://www.w3.org/2001/XMLSchema" > #define XSI_NS "http://www.w3.org/2001/XMLSchema-instance" > #define XRPC_NS "http://monetdb.cwi.nl/XQuery" > #define XRPC_LOC "http://monetdb.cwi.nl/XQuery/XRPC.xsd" > #define WSCOOR_NS "http://docs.oasis-open.org/ws-tx/wscoor/2006/06" > #define WSAT_NS "http://docs.oasis-open.org/ws-tx/wsat/2006/06" > > #define XRPC_REQ_QN XRPC_NS"|request" > #define XRPC_RES_QN XRPC_NS"|response" > > /* XRPC SOAP snippets */ > #define SOAP_ENVELOPE\ > "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n"\ > "<env:Envelope\n"\ > " xmlns:env=\"" SOAP_NS "\"\n"\ > " xmlns:xrpc=\"" XRPC_NS "\"\n"\ > " xmlns:xdt=\"" XDT_NS "\" \n"\ > " xmlns:xs=\"" XS_NS "\"\n"\ > " xmlns:xsi=\"" XSI_NS "\"\n"\ > " xsi:schemaLocation=\"" XRPC_NS " " XRPC_LOC "\">\n" > > #define XRPC_WS_QID\ > " <xrpc:header xrpc:mode=\"%s\">\n"\ > " <wscoor:CoordinationContext\n"\ > " xmlns:wscoor=\""WSCOOR_NS"\"\n"\ > " env:mustUnderstand=\"true\">\n"\ > " <wscoor:Identifier>%s</wscoor:Identifier>\n"\ > " <wscoor:Expires>"LLFMT"</wscoor:Expires>\n"\ > " <wscoor:CoordinationType>" WSAT_NS "</wscoor:CoordinationType>\n"\ > " </wscoor:CoordinationContext>\n"\ > " </xrpc:header>\n" > > #define XRPC_REQ_BODY_OPEN\ > "<env:Body>\n"\ > " <xrpc:request\n"\ > " xrpc:module=\"%s\"\n"\ > " xrpc:location=\"%s\"\n"\ > " xrpc:method=\"%s\"\n"\ > " xrpc:arity=\""LLFMT"\"\n"\ > " xrpc:iter-count=\""LLFMT"\"\n"\ > " xrpc:updCall=\"%s\"\n"\ > " xrpc:caller=\"" > #define XRPC_REQ_BODY\ > XRPC_REQ_BODY_OPEN "%s:"LLFMT"\">\n" > #define XRPC_REQ_BODY_FIRST_CALLER\ > XRPC_REQ_BODY_OPEN "query\">\n" > > #define XRPC_WSAT_REQ SOAP_ENVELOPE\ > "<env:Header>"\ > XRPC_WS_QID \ > "</env:Header>\n"\ > "<env:Body><wsat:%s/></env:Body>\n"\ > "</env:Envelope>" > > #define XRPC_WSAT_RES SOAP_ENVELOPE\ > "<env:Header>\n"\ > XRPC_WS_QID \ > " <xrpc:partipant>%s,%s:%s:%s,"LLFMT",%d</xrpc:participant>\n"\ > "</env:Header>\n"\ > "<env:Body><wsat:%s/></env:Body>\n"\ > "</env:Envelope>" > > #define XRPC_RES_BODY\ > "<env:Body>\n"\ > " <xrpc:response\n"\ > " xrpc:module=\"%s\"\n"\ > " xrpc:method=\"%s\">\n" > > #define XRPC_HTTP_CALL\ > " <xrpc:call>"\ > " <xrpc:sequence>"\ > " <xrpc:atomic-value xsi:type=\"xs:string\">%s</xrpc:atomic-value>"\ > " </xrpc:sequence>"\ > " </xrpc:call>" > > #define XRPC_PUT_CALL\ > " <xrpc:call>"\ > " <xrpc:sequence>"\ > " <xrpc:atomic-value xsi:type=\"xs:string\">%s</xrpc:atomic-value>"\ > " </xrpc:sequence>"\ > " <xrpc:sequence>"\ > " <xrpc:element>%s</xrpc:element>"\ > " </xrpc:sequence>"\ > " </xrpc:call>" > > #define XRPC_REQ_FOOTER\ > " </xrpc:request>\n"\ > "</env:Body>\n"\ > "</env:Envelope>" > > #ifndef HOST_NAME_MAX > #ifdef _POSIX_HOST_NAME_MAX > #define HOST_NAME_MAX _POSIX_HOST_NAME_MAX > #else > #define HOST_NAME_MAX 255 > #endif > #endif > > #endif /* XRPC_CONST_H */ > > > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 4

Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/common stream.mx, , 1.166, 1.167
by Ying Zhang 17 Jul '08

17 Jul '08

On Jul 15, 2008, at 08:58 , Niels Nes wrote: > Update of /cvsroot/monetdb/MonetDB/src/common > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv8300 > > Modified Files: > stream.mx > Log Message: > hopefully fixing the mapi hangs on macos Hi Niels, Unfortunately, the hanging problem still exists, on Mac OS 10.5 By the way, 'mclient -lmil' doesn't hang like 'mclient -lx'. I don't know how 'mclient -lsql' behaves. Regards, Jennie

2 1

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime xrpc_common.mx, , 1.4, 1.5 xrpc_server.mx, , 1.77, 1.78
by Stefan Manegold 17 Jul '08

17 Jul '08

Jennie, in fact, only xprc_admin and xrpc_trusted are currently(?) used/required as global C variable in xrpc_server.c , all other BATs are indeed share as (global) MIL variables and/or via the BAT names. Hence, the attached patch would IMHO clean-up the code even more. (I.e., reduce use of / need for global variables as much as possible.) Stefan On Thu, Jul 17, 2008 at 10:51:38AM +0000, Ying Zhang wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv24940 > > Modified Files: > xrpc_common.mx xrpc_server.mx > Log Message: > Try to fix bug > > [ 2020269 ] PF/runtime fails to link on Darwin (MacOS X) > > remove declaratin of BATs xrpc_* from xrpc_common.mx > xrpc_trusted and xrpc_admin are not shared BATs (only used by > xrpc_server), the others are shared via MIL, so should no be (re)defined > in xrpc_common.mx > > > > U xrpc_common.mx > Index: xrpc_common.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/xrpc_common.mx,v > retrieving revision 1.4 > retrieving revision 1.5 > diff -u -d -r1.4 -r1.5 > --- xrpc_common.mx 15 Jul 2008 14:14:21 -0000 1.4 > +++ xrpc_common.mx 17 Jul 2008 10:51:33 -0000 1.5 > @@ -182,14 +182,6 @@ > XRPCreq_t *xrpc_parse_message(char *err, int errlen, BAT *shredBAT, > BAT *participants); > > -BAT *xrpc_qids; > -BAT *xrpc_timeouts; > -BAT *xrpc_statuses; > -BAT *xrpc_locks; > -BAT *xrpc_wsbats; > -BAT *xrpc_trusted; > -BAT *xrpc_admin; > - > #endif /* XRPC_COMMON_H */ > > @c > > U xrpc_server.mx > Index: xrpc_server.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/xrpc_server.mx,v > retrieving revision 1.77 > retrieving revision 1.78 > diff -u -d -r1.77 -r1.78 > --- xrpc_server.mx 15 Jul 2008 14:14:23 -0000 1.77 > +++ xrpc_server.mx 17 Jul 2008 10:51:33 -0000 1.78 > @@ -194,6 +194,12 @@ > static MT_Sema xrpc_commit_sema; > static xrpc_commit_t *xrpc_commit_active = NULL; > > +/* shared BATs */ > +BAT *xrpc_qids = NULL, *xrpc_statuses = NULL, *xrpc_timeouts = NULL; > +BAT *xrpc_locks = NULL, *xrpc_wsbats = NULL; > +/* only used by xrpc_server */ > +BAT *xrpc_trusted = NULL, *xrpc_admin = NULL, *xrpc_user = NULL; > + > int CMDmy_hostname(char **res) > { > int ret = 0, len = HOST_NAME_MAX > 255 ? HOST_NAME_MAX : 255; > @@ -871,9 +877,6 @@ > BATseqbase(xrpc_@1, 1); > BBPrename(xrpc_@1->batCacheid, "xrpc_@1"); > @c > -BAT *xrpc_qids = NULL, *xrpc_statuses = NULL, *xrpc_timeouts = NULL, *xrpc_locks = NULL, *xrpc_wsbats = NULL; > -BAT *xrpc_trusted = NULL, *xrpc_admin = NULL, *xrpc_user = NULL; > - > > bat* xrpc_prelude(void) { > @:xrpc_bat(qids,str)@ > @@ -881,6 +884,7 @@ > @:xrpc_bat(timeouts,lng)@ > @:xrpc_bat(locks,lock)@ > @:xrpc_bat(wsbats,bat)@ > + > @:xrpc_bat(trusted,str)@ > @:xrpc_bat(admin,str)@ > @:xrpc_bat(user,str)@ > > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > > -- | Dr. Stefan Manegold | mailto:Stefan.Manegold@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4312 |

2 4

Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtime xrpc_server.mx, XQuery_0-24, 1.68.2.4, 1.68.2.5
by Sjoerd Mullender 14 Jul '08

14 Jul '08

Propagation prevented. On 2008-07-14 17:52, Ying Zhang wrote: > Update of /cvsroot/monetdb/pathfinder/runtime > In directory sc8-pr-cvs16.sourceforge.net:/tmp/cvs-serv9021 > > Modified Files: > Tag: XQuery_0-24 > xrpc_server.mx > Log Message: > only in stable: fixes in dealing with whitespace characters between > elements. > > Please *do not* propagate this change > > > > > U xrpc_server.mx > Index: xrpc_server.mx > =================================================================== > RCS file: /cvsroot/monetdb/pathfinder/runtime/xrpc_server.mx,v > retrieving revision 1.68.2.4 > retrieving revision 1.68.2.5 > diff -u -d -r1.68.2.4 -r1.68.2.5 > --- xrpc_server.mx 4 Jun 2008 11:57:58 -0000 1.68.2.4 > +++ xrpc_server.mx 14 Jul 2008 15:52:24 -0000 1.68.2.5 > @@ -756,8 +756,11 @@ > } > argcnt[i][j]++; > nr_args++; > + next_tpe_node_pre += (pre_kindT[next_tpe_node_pre] == TEXT ? 1 : 0); > } /* end loop 'xrpc:<type>' */ > + next_seq_node_pre += (pre_kindT[next_seq_node_pre] == TEXT ? 1 : 0); > } /* end loop 'xrpc:sequence' */ > + next_call_node_pre += (pre_kindT[next_call_node_pre] == TEXT ? 1 : 0); > } /* end loop 'xrpc:call' */ > frag_root = BATsetaccess(frag_root, BAT_READ); > > > > ------------------------------------------------------------------------- > Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! > Studies have shown that voting for your favorite open source project, > along with a healthy diet, reduces your potential for chronic lameness > and boredom. Vote Now at http://www.sourceforge.net/community/cca08 > _______________________________________________ > Monetdb-pf-checkins mailing list > Monetdb-pf-checkins(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins -- Sjoerd Mullender

1 0