MonetDB-extend: reverse/README.rst annotate

annotate reverse/README.rst @ 57:d77cd888a2f2 default tip

We should say something about how to return from the bulk function.

author	Sjoerd Mullender <sjoerd@acm.org>
date	Thu, 24 Mar 2022 18:37:39 +0100
parents	68263b10998e
children

rev	line source
3 de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	1 .. This Source Code Form is subject to the terms of the Mozilla Public
de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	2 .. License, v. 2.0. If a copy of the MPL was not distributed with this
de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	3 .. file, You can obtain one at http://mozilla.org/MPL/2.0/.
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	4 ..
40 e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	5 .. Copyright 2013-2021 MonetDB B.V.
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	6
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	7 .. This document is written in reStructuredText (see
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	8 http://docutils.sourceforge.net/ for more information).
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	9 Use ``rst2html.py`` to convert this file to HTML.
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	10
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	11 ===========================================
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	12 Implementing a Simple User-Defined Function
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	13 ===========================================
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	14
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	15 :Author: Sjoerd Mullender
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	16 :Organization: CWI, MonetDB B.V.
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	17 :Abstract: In this short tutorial we describe how to create a simple
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	18 UDF for MonetDB/SQL.
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	19
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	20 Introduction
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	21 ------------
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	22
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	23 In this directory we show how to make a simple user-defined function
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	24 (UDF) that can be used in MonetDB/SQL. The function is implemented in
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	25 C. In order to make the function available to SQL, we need to provide
3 de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	26 the correct C interface, a MAL interface, and an SQL interface. We
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	27 will discuss them all, starting from SQL and going down towards the
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	28 actual implementation.
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	29
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	30 We want to create a function that allows us to write something like
42 b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	31 this:
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	32
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	33 .. code-block:: sql
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	34
49 0e5e08bd133b Rename "reverse" function to "revstr" to avoid clash. Sjoerd Mullender <sjoerd@acm.org> parents: 42 diff changeset	35 SELECT revstr(strcol) FROM table;
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	36
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	37 where ``table`` is an SQL table with a column called ``strcol`` which
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	38 is of type ``VARCHAR`` (or any other string type).
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	39
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	40 Implementation
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	41 --------------
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	42
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	43 We will first create an interface to do a simple one-value-at-a-time
42 b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	44 (scalar) operation:
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	45
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	46 .. code-block:: sql
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	47
49 0e5e08bd133b Rename "reverse" function to "revstr" to avoid clash. Sjoerd Mullender <sjoerd@acm.org> parents: 42 diff changeset	48 SELECT revstr('string');
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	49
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	50 The SQL catalog will need to be extended with a definition of the
49 0e5e08bd133b Rename "reverse" function to "revstr" to avoid clash. Sjoerd Mullender <sjoerd@acm.org> parents: 42 diff changeset	51 ``revstr`` function as follows:
42 b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	52
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	53 .. code-block:: sql
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	54
49 0e5e08bd133b Rename "reverse" function to "revstr" to avoid clash. Sjoerd Mullender <sjoerd@acm.org> parents: 42 diff changeset	55 CREATE FUNCTION revstr(src STRING) RETURNS STRING
0e5e08bd133b Rename "reverse" function to "revstr" to avoid clash. Sjoerd Mullender <sjoerd@acm.org> parents: 42 diff changeset	56 EXTERNAL NAME revstr.revstr;
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	57
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	58 The statement tells the SQL system that there is a function called
49 0e5e08bd133b Rename "reverse" function to "revstr" to avoid clash. Sjoerd Mullender <sjoerd@acm.org> parents: 42 diff changeset	59 ``revstr`` which takes a ``STRING`` argument and produces a
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	60 ``STRING`` result. The function is implemented using the MAL
49 0e5e08bd133b Rename "reverse" function to "revstr" to avoid clash. Sjoerd Mullender <sjoerd@acm.org> parents: 42 diff changeset	61 interface ``revstr.revstr``. Note that ``STRING`` is equivalent to
3 de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	62 ``CHARACTER LARGE OBJECT`` or ``CLOB``.
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	63
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	64 This statement will normally be executed once when the database is
40 e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	65 created, after which it is part of the SQL catalog. How this is
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	66 accomplished exactly we will leave until later in this tutorial. For
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	67 now let it suffice to note that the SQL query is encoded as a C string
42 b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	68 and stored in the variable ``reverse_sql``:
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	69
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	70 .. code-block:: c
40 e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	71
49 0e5e08bd133b Rename "reverse" function to "revstr" to avoid clash. Sjoerd Mullender <sjoerd@acm.org> parents: 42 diff changeset	72 static char reverse_sql[] = "CREATE FUNCTION revstr(src STRING)"
0e5e08bd133b Rename "reverse" function to "revstr" to avoid clash. Sjoerd Mullender <sjoerd@acm.org> parents: 42 diff changeset	73 " RETURNS STRING EXTERNAL NAME revstr.revstr;";
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	74
7 38931ec299f8 Some updates. Sjoerd Mullender <sjoerd@acm.org> parents: 6 diff changeset	75 At the SQL side we don't have to do anything more.
38931ec299f8 Some updates. Sjoerd Mullender <sjoerd@acm.org> parents: 6 diff changeset	76
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	77 The SQL layer produces an intermediate code called MAL (which stands
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	78 for MonetDB Assembly Language). The external name above refers to a
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	79 command that is defined in MAL, so now that we have the SQL interface,
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	80 we need to create the MAL interface.
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	81
42 b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	82 The MAL interface of the function looks like this:
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	83
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	84 .. code-block::
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	85
49 0e5e08bd133b Rename "reverse" function to "revstr" to avoid clash. Sjoerd Mullender <sjoerd@acm.org> parents: 42 diff changeset	86 module revstr;
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	87
49 0e5e08bd133b Rename "reverse" function to "revstr" to avoid clash. Sjoerd Mullender <sjoerd@acm.org> parents: 42 diff changeset	88 command revstr(ra1:str):str
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	89 address UDFreverse
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	90 comment "Reverse a string";
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	91
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	92 The SQL engine uses the convention that a bulk variant of a scalar
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	93 operation (i.e., a variant that works on a complete column and
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	94 produces a column as opposed to a function that works on a single
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	95 value and produces a single value) has the same name but is located in
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	96 a module with the string ``bat`` prepended. So, the bulk version of
49 0e5e08bd133b Rename "reverse" function to "revstr" to avoid clash. Sjoerd Mullender <sjoerd@acm.org> parents: 42 diff changeset	97 the ``revstr.revstr`` function can also be created:
42 b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	98
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	99 .. code-block::
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	100
49 0e5e08bd133b Rename "reverse" function to "revstr" to avoid clash. Sjoerd Mullender <sjoerd@acm.org> parents: 42 diff changeset	101 module batrevstr;
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	102
49 0e5e08bd133b Rename "reverse" function to "revstr" to avoid clash. Sjoerd Mullender <sjoerd@acm.org> parents: 42 diff changeset	103 command revstr(b:bat[:str]):bat[:str]
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	104 address UDFBATreverse
3 de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	105 comment "Reverse a column of strings";
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	106
40 e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	107 This MAL code also needs to be encoded in the C source. This is done as
42 b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	108 follows:
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	109
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	110 .. code-block:: c
40 e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	111
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	112 static mel_func reverse_init_funcs[] = {
49 0e5e08bd133b Rename "reverse" function to "revstr" to avoid clash. Sjoerd Mullender <sjoerd@acm.org> parents: 42 diff changeset	113 command("revstr", "revstr", UDFreverse, false,
40 e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	114 "Reverse a string",
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	115 args(1,2, arg("",str),arg("ra1",str))),
49 0e5e08bd133b Rename "reverse" function to "revstr" to avoid clash. Sjoerd Mullender <sjoerd@acm.org> parents: 42 diff changeset	116 command("batrevstr", "revstr", UDFBATreverse, false,
40 e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	117 "Reverse a BAT of strings",
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	118 args(1,2, batarg("",str),batarg("b",str))),
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	119 { .imp=NULL } /* sentinel */
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	120 };
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	121
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	122 A C array with elements of type ``mel_func`` is created, and each MAL
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	123 command is one element of this array. The array ends with a sentinel, a
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	124 value that is "empty" and can thus be recognized as the end of the
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	125 array.
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	126
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	127 Each element in the array is an instance of the macro ``command`` which
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	128 has a bunch of arguments. In order, they are: MAL module name (string),
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	129 MAL function name (string), C function (pointer to function), a Boolean
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	130 flag to indicate whether this is an "unsafe" operation (one with side
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	131 effects), a comment (string--not currently used but must be present), a
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	132 description of the function arguments. The function arguments are
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	133 encoded using the ``args`` macro with the following arguments. The
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	134 number of return values (MAL functions can return 1 or more values), the
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	135 total number of arguments (i.e. the sum of return values and input
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	136 arguments), and then for each return argument and each input argument a
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	137 description of the argument itself. Each argument is an instance of a
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	138 macro. There are various forms, but here we use two. ``arg`` describes
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	139 a scalar argument and has two parameters, the MAL name of the argument
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	140 and the MAL type. ``batarg`` describes a BAT argument and also has two
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	141 parameters, the MAL name of the argument and the MAL type of the
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	142 elements of the BAT.
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	143
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	144 Note that implementing a bulk version is optional. If it does not
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	145 exist, the scalar version will be called multiple times. However,
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	146 calling the scalar version multiple (sometimes very many) times incurs
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	147 significant overhead, hence it is usually a good idea to implement the
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	148 bulk version.
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	149
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	150 Now we come to the actual implementation of the feature.
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	151
49 0e5e08bd133b Rename "reverse" function to "revstr" to avoid clash. Sjoerd Mullender <sjoerd@acm.org> parents: 42 diff changeset	152 The MAL interfaces of the scalar and bulk versions of the ``revstr``
42 b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	153 function translates to the following C interfaces:
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	154
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	155 .. code-block:: c
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	156
40 e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	157 static char UDFreverse(char retval, const char *arg);
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	158 static char UDFBATreverse(bat retval, const bat *arg);
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	159
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	160 The return value of the C functions is normally ``MAL_SUCCEED`` which
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	161 translates to ``NULL``. If an error occurs, the return should be a
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	162 freshly allocated string that contains the error message. The string
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	163 must be allocated with the function ``GDKmalloc`` or one of its
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	164 variants (``GDKzalloc``, ``GDKstrdup``) since it will be freed with
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	165 ``GDKfree`` when the interpreter is done with the message. Most of
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	166 the time this can be done by calling the macro ``throw`` which is
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	167 defined as ``return createException``, so calling ``throw`` with
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	168 arguments will actually call ``createException`` with those same
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	169 arguments and return the result. The function ``createException`` has
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	170 three or more arguments, the first is ``MAL``, the second is the name
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	171 of the MAL function, the third is a ``printf`` format string, and
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	172 remaining arguments are values that are used by the format string. A
42 b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	173 minimal example is:
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	174
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	175 .. code-block:: c
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	176
40 e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	177 static char UDFreverse(char retval, const char *arg)
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	178 {
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	179 (void) retval; (void) arg; /* we're not using these */
49 0e5e08bd133b Rename "reverse" function to "revstr" to avoid clash. Sjoerd Mullender <sjoerd@acm.org> parents: 42 diff changeset	180 throw(MAL, "revstr.revstr", "Not yet implemented");
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	181 }
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	182
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	183 MAL commands can return any number of values. These values are
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	184 returned by the C function in the first arguments of the C function,
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	185 hence the two ``retval`` arguments. The arguments of the MAL
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	186 interface follow the return values in the argument list of the C
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	187 function (the ``arg`` arguments). We have added the ``const`` keyword
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	188 to indicate that the arguments will not be altered.
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	189
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	190 The MonetDB code usually uses the C type ``str`` which is defined to
42 b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	191 be ``char *``, so you could define the functions also as:
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	192
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	193 .. code-block:: c
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	194
40 e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	195 static str UDFreverse(str retval, const str arg);
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	196 static str UDFBATreverse(bat retval, const bat arg);
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	197
1 ccdd5feff344 Some updates to the tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 0 diff changeset	198 Note that the definitions are not entirely equivalent. The target of
ccdd5feff344 Some updates to the tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 0 diff changeset	199 the ``const`` keyword is different for the first function.
ccdd5feff344 Some updates to the tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 0 diff changeset	200
5 c3f23ea91e48 Updates for current MonetDB release. Sjoerd Mullender <sjoerd@acm.org> parents: 4 diff changeset	201 The type ``bat`` is defined as ``int``, so you may see ``int`` in some
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	202 places in the MonetDB source code in equivalent positions.
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	203
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	204 These functions must be located in a dynamically loadable module
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	205 (``.so`` file on Linux, ``.dll`` on Windows), and this module must
52 22b5ec33dd19 Fix cmake build for monetdbe-extend. Aris Koning <aris.koning@monetdbsolutions.com> parents: 50 diff changeset	206 have the name ``lib_reverse.so`` (or ``lib_reverse.dll``). The
40 e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	207 functions are only directly referenced from the ``reverse_init_funcs``
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	208 array that we have defined above, so the functions are declared as
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	209 ``static`` functions.
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	210
3 de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	211 Scalar Version
de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	212 ~~~~~~~~~~~~~~
de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	213
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	214 We will now first focus on the implementation of the scalar function
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	215 and return to the bulk version later.
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	216
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	217 The input of the function ``UDFreverse`` is a (NULL-terminated)
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	218 string. The function is called with a pointer to the string pointer,
3 de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	219 so ``arg`` is the actual string and ``*arg`` the first byte of the
de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	220 string.
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	221
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	222 The result of the operation is also a (NULL-terminated) string. Since
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	223 the caller does not know what the size of the result will be, it
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	224 provides a pointer to where the result is to be put. The callee is
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	225 responsible for allocating the necessary space. This means that we
42 b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	226 need to do something like this:
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	227
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	228 .. code-block:: c
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	229
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	230 *retval = GDKmalloc(size);
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	231 // copy data into *retval
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	232
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	233 In the case of this function, calculating the needed space is easy,
42 b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	234 although we need to do error checking as well:
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	235
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	236 .. code-block:: c
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	237
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	238 retval = GDKmalloc(strlen(arg) + 1);
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	239 if (*retval == NULL)
49 0e5e08bd133b Rename "reverse" function to "revstr" to avoid clash. Sjoerd Mullender <sjoerd@acm.org> parents: 42 diff changeset	240 throw(MAL, "revstr.revstr", MAL_MALLOC_FAIL);
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	241 // reverse the string in arg into retval
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	242 return MAL_SUCCEED;
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	243
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	244 In the actual algorithm we have also taken into account that strings
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	245 in MonetDB are always stored in the UTF-8 encoding. In addition, our
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	246 implementation checks for the special value ``str_nil`` which is the C
3 de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	247 representation of the SQL ``NULL`` value for strings (which in MAL is
de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	248 called ``nil:str``).
de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	249
de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	250 Bulk Version
de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	251 ~~~~~~~~~~~~
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	252
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	253 The bulk version gets as input a pointer to a BAT identifier (a value
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	254 of type ``bat``). It also returns a BAT identifier of a newly created
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	255 BAT through the output pointer.
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	256
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	257 It is important to realize that BATs are reference counted with two
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	258 reference counters. There is a logical reference count (``lref``)
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	259 and a physical reference count (``ref``). C code is only allowed to
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	260 look at the actual data when the physical reference count is greater
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	261 than zero and the BAT is loaded into (virtual) memory. A newly
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	262 created BAT has a physical reference count of one and a logical
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	263 reference count of zero. Before returning a new BAT, the physical
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	264 reference count must be converted to a logical reference count.
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	265
7 38931ec299f8 Some updates. Sjoerd Mullender <sjoerd@acm.org> parents: 6 diff changeset	266 When a C function is called by the MAL interpreter, it passes a
38931ec299f8 Some updates. Sjoerd Mullender <sjoerd@acm.org> parents: 6 diff changeset	267 logical reference to any BAT parameters. This means that the C
38931ec299f8 Some updates. Sjoerd Mullender <sjoerd@acm.org> parents: 6 diff changeset	268 function must first make sure that the BAT gets a physical reference
38931ec299f8 Some updates. Sjoerd Mullender <sjoerd@acm.org> parents: 6 diff changeset	269 and is loaded into memory. We start with doing just that. We do that
38931ec299f8 Some updates. Sjoerd Mullender <sjoerd@acm.org> parents: 6 diff changeset	270 by calling ``BATdescriptor`` which increments the physical reference
42 b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	271 count and loads the BAT into memory (if it wasn't there already):
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	272
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	273 .. code-block:: c
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	274
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	275 BAT *b;
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	276 b = BATdescriptor(*arg);
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	277 if (b == NULL)
49 0e5e08bd133b Rename "reverse" function to "revstr" to avoid clash. Sjoerd Mullender <sjoerd@acm.org> parents: 42 diff changeset	278 throw(MAL, "batrevstr.revstr", RUNTIME_OBJECT_MISSING);
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	279
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	280 When we're done with this BAT, we will need to decrement the physical
42 b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	281 reference count again. We do that by calling ``BBPunfix``:
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	282
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	283 .. code-block:: c
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	284
3 de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	285 BBPunfix(b->batCacheid);
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	286
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	287 Note that ``b->batCacheid`` is equal to ``*arg``.
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	288
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	289 We need to create the result BAT ourselves. We know the type, and we
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	290 know that the size of the BAT will be the same as the input BAT.
42 b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	291 Hence we can use this code:
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	292
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	293 .. code-block:: c
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	294
13 a3465119dc5b Updated comments for (not so) new MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 12 diff changeset	295 bn = COLnew(b->hseqbase, TYPE_str, BATcount(b), TRANSIENT);
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	296
14 5a5167adae4a Updates and corrections for current MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 13 diff changeset	297 The arguments of ``COLnew`` are the head seqbase, the type of the
5a5167adae4a Updates and corrections for current MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 13 diff changeset	298 column, the initial size (in number of entries) of the to-be-allocated
13 a3465119dc5b Updated comments for (not so) new MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 12 diff changeset	299 BAT, and ``TRANSIENT`` to indicate that this BAT is temporary.
a3465119dc5b Updated comments for (not so) new MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 12 diff changeset	300 ``COLnew`` guarantees that there is space for at least the specified
a3465119dc5b Updated comments for (not so) new MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 12 diff changeset	301 number of elements, or it returns ``NULL``. Since we call
a3465119dc5b Updated comments for (not so) new MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 12 diff changeset	302 ``BUNappend`` to add entries to the BAT, we're actually not concerned
a3465119dc5b Updated comments for (not so) new MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 12 diff changeset	303 about the size of the new BAT (``BUNappend`` takes care of resizing if
a3465119dc5b Updated comments for (not so) new MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 12 diff changeset	304 necessary), but from an efficiency point of view, it's better to
a3465119dc5b Updated comments for (not so) new MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 12 diff changeset	305 create the BAT with the required size (growing a BAT can be
14 5a5167adae4a Updates and corrections for current MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 13 diff changeset	306 expensive). We must set the head sequence base of the new BAT to be
5a5167adae4a Updates and corrections for current MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 13 diff changeset	307 the same as that of the input BAT.
5a5167adae4a Updates and corrections for current MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 13 diff changeset	308
5a5167adae4a Updates and corrections for current MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 13 diff changeset	309 Note that for variable-sized types (such as the strings we are dealing
5a5167adae4a Updates and corrections for current MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 13 diff changeset	310 with here), ``BUNappend`` can still fail due to not enough memory,
5a5167adae4a Updates and corrections for current MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 13 diff changeset	311 even though we supposedly allocated enough. The strings have to be
5a5167adae4a Updates and corrections for current MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 13 diff changeset	312 stored somewhere, and ``COLnew`` has no way of knowing how large the
40 e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	313 total area for the strings must be, so ``BUNappend`` may have to grow
14 5a5167adae4a Updates and corrections for current MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 13 diff changeset	314 the memory area for the strings, and that can fail.
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	315
42 b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	316 Iterating through the source BAT is done using a standard mechanism:
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	317
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	318 .. code-block:: c
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	319
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	320 BATiter bi;
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	321 BUN p, q;
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	322 bi = bat_iterator(b);
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	323 BATloop(b, p, q) {
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	324 ...
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	325 }
55 68263b10998e Updated: added bat_iterator_end. Sjoerd Mullender <sjoerd@acm.org> parents: 52 diff changeset	326 bat_iterator_end(&bi);
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	327
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	328 ``BATloop`` is a macro that translates to a C ``for`` loop, using the
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	329 first argument to set the bounds, the second argument to iterate
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	330 through the BAT, and the third argument as the loop limit. The second
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	331 argument can be used inside the body as an argument to
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	332 e.g. ``BUNtail``.
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	333
55 68263b10998e Updated: added bat_iterator_end. Sjoerd Mullender <sjoerd@acm.org> parents: 52 diff changeset	334 We make use of a BAT iterator (type ``BATiter``) which is initialized
68263b10998e Updated: added bat_iterator_end. Sjoerd Mullender <sjoerd@acm.org> parents: 52 diff changeset	335 with a call to ``bat_iterator(b)``. Since this call increments a
68263b10998e Updated: added bat_iterator_end. Sjoerd Mullender <sjoerd@acm.org> parents: 52 diff changeset	336 reference count, we need to match the initialization with a call to
68263b10998e Updated: added bat_iterator_end. Sjoerd Mullender <sjoerd@acm.org> parents: 52 diff changeset	337 ``bat_iterator_end(&bi)``. The calls to ``bat_iterator`` and
68263b10998e Updated: added bat_iterator_end. Sjoerd Mullender <sjoerd@acm.org> parents: 52 diff changeset	338 ``bat_iterator_end`` cannot fail. The need for ``bat_iterator_end`` is
68263b10998e Updated: added bat_iterator_end. Sjoerd Mullender <sjoerd@acm.org> parents: 52 diff changeset	339 new since the Jul2021 release (11.41.X).
68263b10998e Updated: added bat_iterator_end. Sjoerd Mullender <sjoerd@acm.org> parents: 52 diff changeset	340
14 5a5167adae4a Updates and corrections for current MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 13 diff changeset	341 The body of the loop first retrieves the current value from the
42 b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	342 column:
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	343
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	344 .. code-block:: c
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	345
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	346 src = (const char *) BUNtail(bi, p);
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	347
a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	348 We then use this string in the same way as in the scalar function.
42 b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	349 The reversed string in ``dst`` is appended to the result BAT:
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	350
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	351 .. code-block:: c
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	352
40 e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	353 BUNappend(bn, dst, false);
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	354
40 e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	355 The third argument to ``BUNappend`` must always be ``false``.
0 a1080ed7fe4d Created an example UDF for a SQL reverse() function. Sjoerd Mullender <sjoerd@acm.org> parents: diff changeset	356
40 e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	357 ``BUNappend`` returns ``GDK_SUCCEED`` for success or ``GDK_FAIL`` for
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	358 failure. ``BUNappend`` is marked in the include file as a function of
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	359 which the result must be checked. It is a good convention to always
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	360 check whether the result is (not) equal to ``GDK_SUCCEED`` so that if in
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	361 the future different errors are returned, the code keeps working.
3 de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	362
de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	363 BAT Properties
de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	364 --------------
de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	365
4 d84d840b4f86 Updates to text. Sjoerd Mullender <sjoerd@acm.org> parents: 3 diff changeset	366 MonetDB makes extensive use of a number of property flags that can be
14 5a5167adae4a Updates and corrections for current MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 13 diff changeset	367 set on BATs. It is crucial that these property flags don't lie. When
5a5167adae4a Updates and corrections for current MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 13 diff changeset	368 the server is started with the ``-d10`` option, the server checks
5a5167adae4a Updates and corrections for current MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 13 diff changeset	369 these property flags and exits with a failed assertion when a flag is
5a5167adae4a Updates and corrections for current MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 13 diff changeset	370 set incorrectly (or the server issues a warning when it was built with
5a5167adae4a Updates and corrections for current MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 13 diff changeset	371 assertions disabled).
3 de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	372
4 d84d840b4f86 Updates to text. Sjoerd Mullender <sjoerd@acm.org> parents: 3 diff changeset	373 Property flags are Boolean flags, i.e. they are either true (set) or
d84d840b4f86 Updates to text. Sjoerd Mullender <sjoerd@acm.org> parents: 3 diff changeset	374 false (not set). When a property flag is not set, it means that
d84d840b4f86 Updates to text. Sjoerd Mullender <sjoerd@acm.org> parents: 3 diff changeset	375 either the property doesn't hold or it is unknown whether the property
d84d840b4f86 Updates to text. Sjoerd Mullender <sjoerd@acm.org> parents: 3 diff changeset	376 holds.
3 de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	377
4 d84d840b4f86 Updates to text. Sjoerd Mullender <sjoerd@acm.org> parents: 3 diff changeset	378 The property flags are
3 de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	379
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	380 ``tsorted``
3 de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	381 the column is sorted in ascending order
de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	382
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	383 ``trevsorted``
3 de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	384 the column is sorted in descending order
de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	385
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	386 ``tkey``
3 de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	387 all values in the column are distinct
de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	388
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	389 ``tnonil``
3 de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	390 there are no nil values in the column
de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	391
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	392 ``tnil``
3 de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	393 there are nil values in the column (this property isn't used
de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	394 internally)
de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	395
7 38931ec299f8 Some updates. Sjoerd Mullender <sjoerd@acm.org> parents: 6 diff changeset	396 In addition to these flags there are a few other properties:
38931ec299f8 Some updates. Sjoerd Mullender <sjoerd@acm.org> parents: 6 diff changeset	397
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	398 ``tnosorted``
7 38931ec299f8 Some updates. Sjoerd Mullender <sjoerd@acm.org> parents: 6 diff changeset	399 the location that proofs the column is not sorted
38931ec299f8 Some updates. Sjoerd Mullender <sjoerd@acm.org> parents: 6 diff changeset	400
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	401 ``tnorevsorted``
7 38931ec299f8 Some updates. Sjoerd Mullender <sjoerd@acm.org> parents: 6 diff changeset	402 the location that proofs the column is not reverse sorted
38931ec299f8 Some updates. Sjoerd Mullender <sjoerd@acm.org> parents: 6 diff changeset	403
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	404 ``tnokey[0]`` and ``tnokey[1]``
7 38931ec299f8 Some updates. Sjoerd Mullender <sjoerd@acm.org> parents: 6 diff changeset	405 two locations that together proof not all values are distinct
38931ec299f8 Some updates. Sjoerd Mullender <sjoerd@acm.org> parents: 6 diff changeset	406
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	407 The property flags ``tsorted`` and ``trevsorted`` may both be set at
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	408 the same time. When they are, it implies that all values are equal to
3 de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	409 each other.
de61daddf5ab Updated copyrights and code, added comments, extended tutorial. Sjoerd Mullender <sjoerd@acm.org> parents: 1 diff changeset	410
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	411 When the ``tsorted`` property is unset, the ``tnosorted`` property is
7 38931ec299f8 Some updates. Sjoerd Mullender <sjoerd@acm.org> parents: 6 diff changeset	412 a position in the BAT where the previous value is not less than or
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	413 equal to the position itself. If the ``tnosorted`` value is 0, we
7 38931ec299f8 Some updates. Sjoerd Mullender <sjoerd@acm.org> parents: 6 diff changeset	414 truly don't know whether the BAT is sorted. Similarly for the
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	415 ``trevsorted`` and ``tnorevsorted`` properties. The two ``tnokey``
7 38931ec299f8 Some updates. Sjoerd Mullender <sjoerd@acm.org> parents: 6 diff changeset	416 values are either both 0 (we really don't know), or two distinct
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	417 locations whose values are equal. The ``tno``... values must be
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	418 zero if the corresponding property is set, they may be zero if the
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	419 corresponding property in unset.
7 38931ec299f8 Some updates. Sjoerd Mullender <sjoerd@acm.org> parents: 6 diff changeset	420
4 d84d840b4f86 Updates to text. Sjoerd Mullender <sjoerd@acm.org> parents: 3 diff changeset	421 Note that most of the properties are true for an empty column, hence
14 5a5167adae4a Updates and corrections for current MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 13 diff changeset	422 when ``COLnew`` returns, all property flags except for ``nil`` are set
4 d84d840b4f86 Updates to text. Sjoerd Mullender <sjoerd@acm.org> parents: 3 diff changeset	423 (there are no nils in an empty column). This means that as soon as
d84d840b4f86 Updates to text. Sjoerd Mullender <sjoerd@acm.org> parents: 3 diff changeset	424 you start adding data to a column, you must deal with the property
42 b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	425 flags. The simplest solution is to just clear all properties:
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	426
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	427 .. code-block:: c
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	428
28 e925d55b369b Updated to Aug2018 (11.31.X) version of MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 19 diff changeset	429 bn->tsorted = bn->trevsorted = false;
e925d55b369b Updated to Aug2018 (11.31.X) version of MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 19 diff changeset	430 bn->tkey = false;
e925d55b369b Updated to Aug2018 (11.31.X) version of MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 19 diff changeset	431 bn->tnil = false;
e925d55b369b Updated to Aug2018 (11.31.X) version of MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 19 diff changeset	432 bn->tnonil = false;
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	433 bn->tnosorted = bn->tnorevsorted = 0;
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	434 bn->tnokey[0] = bn->tnokey[1] = 0;
4 d84d840b4f86 Updates to text. Sjoerd Mullender <sjoerd@acm.org> parents: 3 diff changeset	435
d84d840b4f86 Updates to text. Sjoerd Mullender <sjoerd@acm.org> parents: 3 diff changeset	436 Note also that the function ``BUNappend`` maintains the property flags
14 5a5167adae4a Updates and corrections for current MonetDB. Sjoerd Mullender <sjoerd@acm.org> parents: 13 diff changeset	437 as best it can. That is why in the example we didn't need to do
4 d84d840b4f86 Updates to text. Sjoerd Mullender <sjoerd@acm.org> parents: 3 diff changeset	438 anything with the property flags.
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	439
57 d77cd888a2f2 We should say something about how to return from the bulk function. Sjoerd Mullender <sjoerd@acm.org> parents: 55 diff changeset	440 Returning the Result
d77cd888a2f2 We should say something about how to return from the bulk function. Sjoerd Mullender <sjoerd@acm.org> parents: 55 diff changeset	441 --------------------
d77cd888a2f2 We should say something about how to return from the bulk function. Sjoerd Mullender <sjoerd@acm.org> parents: 55 diff changeset	442
d77cd888a2f2 We should say something about how to return from the bulk function. Sjoerd Mullender <sjoerd@acm.org> parents: 55 diff changeset	443 When the return BAT has been created and filled, it needs to be
d77cd888a2f2 We should say something about how to return from the bulk function. Sjoerd Mullender <sjoerd@acm.org> parents: 55 diff changeset	444 returned. This is done by assigning the BAT id to the space the result
d77cd888a2f2 We should say something about how to return from the bulk function. Sjoerd Mullender <sjoerd@acm.org> parents: 55 diff changeset	445 pointer points to. In addition, we need to deal with the reference
d77cd888a2f2 We should say something about how to return from the bulk function. Sjoerd Mullender <sjoerd@acm.org> parents: 55 diff changeset	446 counting mentioned before. The input BAT's reference that we got from
d77cd888a2f2 We should say something about how to return from the bulk function. Sjoerd Mullender <sjoerd@acm.org> parents: 55 diff changeset	447 ``BATdescriptor`` should be released with ``BBPunfix`` and one physical
d77cd888a2f2 We should say something about how to return from the bulk function. Sjoerd Mullender <sjoerd@acm.org> parents: 55 diff changeset	448 reference to the new BAT needs to be converted to one logical reference.
d77cd888a2f2 We should say something about how to return from the bulk function. Sjoerd Mullender <sjoerd@acm.org> parents: 55 diff changeset	449 This is done by a call to BBPkeepref. And finally, the function should
d77cd888a2f2 We should say something about how to return from the bulk function. Sjoerd Mullender <sjoerd@acm.org> parents: 55 diff changeset	450 return and indicate that no errors occurred. This is done by returning
d77cd888a2f2 We should say something about how to return from the bulk function. Sjoerd Mullender <sjoerd@acm.org> parents: 55 diff changeset	451 the value ``MAL_SUCCEED`` (which is just a ``NULL`` pointer).
d77cd888a2f2 We should say something about how to return from the bulk function. Sjoerd Mullender <sjoerd@acm.org> parents: 55 diff changeset	452
d77cd888a2f2 We should say something about how to return from the bulk function. Sjoerd Mullender <sjoerd@acm.org> parents: 55 diff changeset	453 .. code-block:: c
d77cd888a2f2 We should say something about how to return from the bulk function. Sjoerd Mullender <sjoerd@acm.org> parents: 55 diff changeset	454
d77cd888a2f2 We should say something about how to return from the bulk function. Sjoerd Mullender <sjoerd@acm.org> parents: 55 diff changeset	455 BBPunfix(b->batCacheid);
d77cd888a2f2 We should say something about how to return from the bulk function. Sjoerd Mullender <sjoerd@acm.org> parents: 55 diff changeset	456 BBPkeepref(bn->batCacheid);
d77cd888a2f2 We should say something about how to return from the bulk function. Sjoerd Mullender <sjoerd@acm.org> parents: 55 diff changeset	457 *retval = bn->batCacheid;
d77cd888a2f2 We should say something about how to return from the bulk function. Sjoerd Mullender <sjoerd@acm.org> parents: 55 diff changeset	458 return MAL_SUCCEED;
d77cd888a2f2 We should say something about how to return from the bulk function. Sjoerd Mullender <sjoerd@acm.org> parents: 55 diff changeset	459
40 e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	460 Informing the Server
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	461 --------------------
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	462
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	463 So far we have created the necessary C code that can be called by the
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	464 interpreter when the appropriate SQL query is executed. However, we
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	465 still need to tell the server that this code actually exists. We have
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	466 already hinted at this, but here we will finish that part.
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	467
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	468 Once the ``.so`` or ``.ddl`` file has been created and installed, the
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	469 server needs to be told about it. This is done be calling the server
42 b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	470 with an extra argument:
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	471
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	472 .. code-block:: sh
40 e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	473
52 22b5ec33dd19 Fix cmake build for monetdbe-extend. Aris Koning <aris.koning@monetdbsolutions.com> parents: 50 diff changeset	474 mserver5 --loadmodule=reverse ...
40 e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	475
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	476 where ``...`` represents any other arguments not covered by this
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	477 tutorial.
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	478
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	479 When the server gets this ``--loadmodule`` argument, it loads the
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	480 library. And here we use a trick that is available in dynamically
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	481 loaded libraries. We tell the system to automatically execute some code
42 b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	482 when the library is loaded:
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	483
b67deab098f9 Use code-block directives. Sjoerd Mullender <sjoerd@acm.org> parents: 40 diff changeset	484 .. code-block:: c
40 e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	485
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	486 #include "mal_import.h"
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	487 #include "sql_import.h"
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	488 #ifdef _MSC_VER
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	489 #undef read
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	490 #pragma section(".CRT$XCU",read)
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	491 #endif
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	492 LIB_STARTUP_FUNC(init_reverse)
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	493 {
49 0e5e08bd133b Rename "reverse" function to "revstr" to avoid clash. Sjoerd Mullender <sjoerd@acm.org> parents: 42 diff changeset	494 mal_module("revstr", NULL, reverse_init_funcs);
0e5e08bd133b Rename "reverse" function to "revstr" to avoid clash. Sjoerd Mullender <sjoerd@acm.org> parents: 42 diff changeset	495 sql_register("revstr", reverse_sql);
40 e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	496 }
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	497
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	498 The ``LIB_STARTUP_FUNC`` macro is defined in one of the include files.
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	499 It has an argument which is a name that should be unique. The
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	500 convention is ``init_`` followed by the name of the module. This macro
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	501 is the start of a function definition, so it is followed by the body of
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	502 the function that is executed when the library is loaded.
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	503
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	504 The function calls two functions. The first, ``mal_module``, registers
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	505 the MAL functions that we have defined. The arguments are the name of
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	506 the module, an array of elements of type ``mel_atom`` (not used here),
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	507 and an array of elements of type ``mel_func`` which contains our MAL
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	508 functions.
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	509
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	510 The second function, ``sql_register`` registers the SQL query that needs
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	511 to be executed to enter the SQL function into the catalog.
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	512
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	513 Makefile
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	514 --------
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	515
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	516 To bring all of this together, we have a ``Makefile``. It uses the
40 e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	517 ``pkgconf`` command to find the location of the MonetDB installation and
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	518 the arguments needed to compile the module. (If you don't have
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	519 ``pkgconf``, you may be able to replace it with ``pkg-config``.) This
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	520 Makefile works on Fedora Linux if you have the package
40 e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	521 ``MonetDB-SQL-server5-devel`` with all its dependencies installed
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	522 (available starting in the Oct2020-SP2 bugfix release), and on
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	523 Debian/Ubuntu if you have the package ``monetdb5-sql-dev`` with all its
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	524 dependencies installed (available starting in the Oct2020-SP2 bugfix
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	525 release). The file may need to be changed for other systems. Note that
e70b12c15507 Updated for Oct2020 version. Sjoerd Mullender <sjoerd@acm.org> parents: 28 diff changeset	526 even in the Oct2020-SP5 release there are some include files missing.
19 499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	527
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	528 A Note About Names
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	529 ------------------
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	530
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	531 The name BAT originally stood for Binary Association Table. This
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	532 suggests, correctly, that there were actually two values per row in a
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	533 BAT that were associated. The two values where the head and tail
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	534 values of, what was called, a Binary UNit or BUN. The head and
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	535 tail columns of a BAT had independent types, but it turned out that
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	536 most of the time we used the type oid (Object IDentifier) for the
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	537 head column, and the values were, most of the time, consecutive.
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	538 Since then we have made sure that the head values were always of type
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	539 oid and consecutive, which meant that we could do away with the
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	540 complete head column. The only thing that we still needed (and still
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	541 need) is the first value of the old head column. This value is called
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	542 hseqbase (head sequence base). There are still many vestiges of the
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	543 old head and tail columns, especially when accessing values in what
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	544 used to be the tail column and now is the only column. So we have
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	545 functions such as ``BUNtail`` that have nothing to do anymore with a
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	546 tail. Also, the term BUN was repurposed to being the name we have
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	547 given to the index of the C array that holds the values of what used
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	548 to be the tail column. For more information see the blog post
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	549 `MonetDB goes headless`__.
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	550
499debebda5d Various updates. Sjoerd Mullender <sjoerd@acm.org> parents: 16 diff changeset	551 __ https://www.monetdb.org/blog/monetdb-goes-headless

Mercurial > hg > MonetDB-extend

annotate reverse/README.rst @ 57:d77cd888a2f2 default tip