[Monetdb-developers] [XQ] Mixing update/docmgmt/query
Hi,
I know that the XQUF prohibits it, but for writing robust functions, it is often necessary to mix query and update/docmgmt functionality. Some examples:
* Expression that queries/updates document, but if the document does not exist, adds an empty version of it: let $name := "foo.xml" let $emptydoc := <root/> let $doc := if (empty(pf:documents()[.=$name])) then (put($emptydoc,$name),$emptydoc) else doc("foo.xml") return $doc//bar * Expression that adds to a log-file: let $log := doc("log.xml") let $expr := ...some expression... return (do insert <entry>...{$expr}...<entry> as last into $log,$expr)
Each of these patterns gives you a "!type error: err:XUST0101: illegal mix of updating and non-updating expressions" error.
Isn't is possible to somehow allow such expressions? It is against the standard, so it probably requires some server configuration parameter, but it would be really really useful to have it. Mixing queries with update/docmgmt is known to cause many semantical problems, but only if these updates are visible within the query. Looking at the implementation based on pending update lists, then each of these patterns is harmless. If you declare that no updates are visible within the transaction (which is against the standard), then allowing these queries is I guess not hard: it simply means that with updating expressions and document management expressions you do return a result.
Another thing I've been thinking about in this context: If I send two SOAP-messages to a server on its XRPC-port, am I guaranteed that they are served in the same order? I could perhaps, with a lot of duplication of code, separate update/docmgmt from a query, i.e., implement these things with two requests to the server. I really dislike such a solution, though.
Looking forward to comments :-)
Maurice.
On 03/18/2008 02:43 PM, Keulen, M. van (Maurice) wrote with possible deletions:
Hi,
I know that the XQUF prohibits it, but for writing robust functions, it is often necessary to mix query and update/docmgmt functionality. Some examples:
Hi Maurice,
I don't think what you suggest is necessary. If you want to query with side effects -- which is certainly not what XQuery was initially introduced for -- and you want to simulate XQueryP then you can *always* use updating queries and return your results in a special result document.
* Expression that queries/updates document, but if the document does not exist, adds an empty version of it: let $name := "foo.xml" let $emptydoc := <root/> let $doc := if (empty(pf:documents()[.=$name])) then (put($emptydoc,$name),$emptydoc) else doc("foo.xml") return $doc//bar
Here you expect, that the document management stuff is done just in time and thus with side effects! Your query fails if the document management stuff is done after the query part.
* Expression that adds to a log-file: let $log := doc("log.xml") let $expr := ...some expression... return (do insert <entry>...{$expr}...<entry> as last into $log,$expr)
You can also return a ``nested'' sequence and query the result to split up your query result and the log files. In my eyes you once more try to ``cheat'' to have more than a single result stream.
Jan
Jan Rittinger wrote:
On 03/18/2008 02:43 PM, Keulen, M. van (Maurice) wrote with possible deletions:
Hi,
I know that the XQUF prohibits it, but for writing robust functions, it is often necessary to mix query and update/docmgmt functionality. Some examples:
Hi Maurice,
I don't think what you suggest is necessary. If you want to query with side effects -- which is certainly not what XQuery was initially introduced for -- and you want to simulate XQueryP then you can *always* use updating queries and return your results in a special result document.
What you mean is to do the updates and instead of returning the query result, store it in the database as well ... and then use a second query to retrieve the result? That only works if I am sure that the first query is completed before the second one starts. Furthermore, an update and a docmgmt operation also do not mix, so you need three queries:
* (docmgmt) one to make sure that there is a document in the database for the result (if not, it adds it) * (update) one to do the updates and insert the result in the result document * (query) one to get the result document
A lot of overhead if you ask me.
Furthermore, I access the database by sending SOAP-messages to the XRPC-interface ... in other words: asynchronous communication. So, waiting for a query to complete is hard ... and you cannot send different queries in one SOAP-message (only multiple calls for the same query, but with other parameters).
Jennie: my SOAP-messages do not come from MonetDB/XQuery, but from outside (i.e., some scripting language).
* Expression that queries/updates document, but if the document does not exist, adds an empty version of it: let $name := "foo.xml" let $emptydoc := <root/> let $doc := if (empty(pf:documents()[.=$name])) then (put($emptydoc,$name),$emptydoc) else doc("foo.xml") return $doc//barHere you expect, that the document management stuff is done just in time and thus with side effects! Your query fails if the document management stuff is done after the query part.
No, not true. If "foo.xml" is not in the database yet, I want to add it, but simply continue with $emptydoc. The query does not depend on the document to be added immediately.
* Expression that adds to a log-file: let $log := doc("log.xml") let $expr := ...some expression... return (do insert <entry>...{$expr}...<entry> as last into$log,$expr)
You can also return a ``nested'' sequence and query the result to split up your query result and the log files. In my eyes you once more try to ``cheat'' to have more than a single result stream.
What do you mean by "nested" sequence? Due to the typing rules, an expression is either an updating expression (without query result) or not (with query result). This implies I cannot within one query do an update /and/ return a result. This is exactly the thing I want to do. I do not want two result streams, I just want a query to have one query result but with side-effects to the database.
Maurice.
On Mar 18, 2008, at 15:38 , Keulen, M. van (Maurice) wrote:
Jan Rittinger wrote:
On 03/18/2008 02:43 PM, Keulen, M. van (Maurice) wrote with possible deletions:
Hi,
I know that the XQUF prohibits it, but for writing robust functions, it is often necessary to mix query and update/docmgmt functionality. Some examples:
Hi Maurice,
I don't think what you suggest is necessary. If you want to query with side effects -- which is certainly not what XQuery was initially introduced for -- and you want to simulate XQueryP then you can *always* use updating queries and return your results in a special result document.
What you mean is to do the updates and instead of returning the query result, store it in the database as well ... and then use a second query to retrieve the result? That only works if I am sure that the first query is completed before the second one starts. Furthermore, an update and a docmgmt operation also do not mix, so you need three queries: • (docmgmt) one to make sure that there is a document in the database for the result (if not, it adds it) • (update) one to do the updates and insert the result in the result document • (query) one to get the result document A lot of overhead if you ask me.
Furthermore, I access the database by sending SOAP-messages to the XRPC-interface ... in other words: asynchronous communication. So, waiting for a query to complete is hard ... and you cannot send different queries in one SOAP-message (only multiple calls for the same query, but with other parameters).
Jennie: my SOAP-messages do not come from MonetDB/XQuery, but from outside (i.e., some scripting language).
Then, an order can only be guaranteed if you, in your scripts, send a message, wait for result, and send the next message. In all other cases, unfortunately, no guarantee, as an xrpc server immediately start handling a request when it's received.
Jennie
* Expression that queries/updates document, but if thedocument does not exist, adds an empty version of it: let $name := "foo.xml" let $emptydoc := <root/> let $doc := if (empty(pf:documents()[.=$name])) then (put($emptydoc,$name),$emptydoc) else doc("foo.xml") return $doc//bar
Here you expect, that the document management stuff is done just in time and thus with side effects! Your query fails if the document management stuff is done after the query part.
No, not true. If "foo.xml" is not in the database yet, I want to add it, but simply continue with $emptydoc. The query does not depend on the document to be added immediately.
* Expression that adds to a log-file: let $log := doc("log.xml") let $expr := ...some expression... return (do insert <entry>...{$expr}...<entry> as last into$log,$expr)
You can also return a ``nested'' sequence and query the result to split up your query result and the log files. In my eyes you once more try to ``cheat'' to have more than a single result stream.
What do you mean by "nested" sequence? Due to the typing rules, an expression is either an updating expression (without query result) or not (with query result). This implies I cannot within one query do an update and return a result. This is exactly the thing I want to do. I do not want two result streams, I just want a query to have one query result but with side-effects to the database.
Maurice.
Dr.Ir. M. van Keulen - Assistant Professor, Data Management Technology Univ. of Twente, Dept of EEMCS, POBox 217, 7500 AE Enschede, Netherlands Email: m.vankeulen@utwente.nl, Phone: +31 534893688, Fax: +31 534892927 Room: ZI 3039, WWW: http://www.cs.utwente.nl/~keulen
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/_____________________... Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
Ying Zhang wrote:
Jennie: my SOAP-messages do not come from MonetDB/XQuery, but from outside (i.e., some scripting language).
Then, an order can only be guaranteed if you, in your scripts, send a message, wait for result, and send the next message. In all other cases, unfortunately, no guarantee, as an xrpc server immediately start handling a request when it's received.
Jennie
I thought so already. This is why I don't like to have to send multiple requests to do one thing (transaction). I'd rather be able to send one request to both do, say, an update and return a result, or add a document and return a result. The system so far doesn't allow me to do this, because I cannot define one query (or function in a module) that does both because of typing restrictions, and I also cannot send two separate queries in one request because of restrictions in the XRPC protocol.
Any thoughts about achieving what I want?
Maurice.
On Mar 18, 2008, at 14:43 , Keulen, M. van (Maurice) wrote:
Hi,
I know that the XQUF prohibits it, but for writing robust functions, it is often necessary to mix query and update/docmgmt functionality. Some examples: • Expression that queries/updates document, but if the document does not exist, adds an empty version of it: let $name := "foo.xml" let $emptydoc := <root/> let $doc := if (empty(pf:documents()[.=$name])) then (put($emptydoc, $name),$emptydoc) else doc("foo.xml") return $doc//bar • Expression that adds to a log-file: let $log := doc("log.xml") let $expr := ...some expression... return (do insert <entry>...{$expr}...<entry> as last into $log,$expr) Each of these patterns gives you a "!type error: err:XUST0101: illegal mix of updating and non-updating expressions" error.
Isn't is possible to somehow allow such expressions? It is against the standard, so it probably requires some server configuration parameter, but it would be really really useful to have it. Mixing queries with update/docmgmt is known to cause many semantical problems, but only if these updates are visible within the query. Looking at the implementation based on pending update lists, then each of these patterns is harmless. If you declare that no updates are visible within the transaction (which is against the standard), then allowing these queries is I guess not hard: it simply means that with updating expressions and document management expressions you do return a result.
Another thing I've been thinking about in this context: If I send two SOAP-messages to a server on its XRPC-port, am I guaranteed that they are served in the same order? I could perhaps, with a lot of duplication of code, separate update/docmgmt from a query, i.e., implement these things with two requests to the server. I really dislike such a solution, though.
Hi Maurice,
I assume you are talking about soap msgs triggered by execute-at statements in your query, then, it depends on where those execute-at statements are in the query.
If they are in a sequence expression: (execute at {peer1} {fcn1()}, execute at {peer1} {fcn2()}), then fcn1() will always be evaluated firstly, and then fcn2().
If they are in a for-expression: for $dst in ("peer1", "peer2") return execute at {$dst} {fcn1()}, then the request messages will be sent *in parallel* to peer1 and peer2. Hence, if peer1 and peer2 happen to be the same peer (e.g. if you address the same peer by its hostname, and its ip-address, the xrpc client won't know that the destination is actually the same peer, because xrpc client only does simple string comparison to find unique destination), the order in which the messages are handled depends on which one arrives first. But the order of the results of the for-loop is guaranteed to conform the iteration number.
Regards,
Jennie
Looking forward to comments :-)
Maurice.
--
Dr.Ir. M. van Keulen - Assistant Professor, Data Management Technology Univ. of Twente, Dept of EEMCS, POBox 217, 7500 AE Enschede, Netherlands Email: m.vankeulen@utwente.nl, Phone: +31 534893688, Fax: +31 534892927 Room: ZI 3039, WWW: http://www.cs.utwente.nl/~keulen
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/_____________________... Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
participants (3)
-
Jan Rittinger -
Keulen, M. van (Maurice) -
Ying Zhang