MAL Modules

MAL Modules mk Tue, 04/28/2020 - 16:31

The  MAL module signatures are collected in individual files named .../lib(64)/<modulename>.mal.  The server is bootstrapped by processing a MAL script, called .../lib/MonetDB5/mal_init.mal, with the names of the modules to be loaded. For each module file encountered, the object library lib_<modulename>.so is searched for in the location identified by monet_mod_path=exec_prefix/lib/MonetDB5:exec_prefix/lib/MonetDB5/lib:exec_prefix/lib/MonetDB5/bin. Loading further relies on the Linux policy to search for the module location in the following order: 1) the colon-separated list of directories in the user's LD_LIBRARY_PATH, 2) the libraries specified in /etc/ and 3) /usr/lib followed by /lib.

The MAL module should be compiled with -rdynamic and -ldl (Linux). This enables loading the routines and finding out the address of a particular routine.  Since all modules libraries are loaded completely with GLOBAL visibility, it suffices to provide the internal function name. In case an attempt to link to an address fails, a final attempt is made to locate the *.o file in the current directory.

A module loading conflict emerges if a function is redefined. A duplicate load is simply ignored by keeping track of modules already loaded.


Alarm mk Tue, 04/28/2020 - 18:24

Timers and Timed Interrupts

This module handles various signaling/timer functionalities. The parameterless routines, usec, time and ctime provide access to the CPU clock. They return an integer and string, respectively. The functions are considered unsafe, because they affect the timing experienced by the user.

command alarm.sleep{unsafe}(secs:int):void
address ALARMsleep
comment "Sleep a few seconds";

command alarm.usec{unsafe}() :lng
address ALARMusec
comment "Return time in microseconds.";

command alarm.time{unsafe}() :int
address ALARMtime
comment "Return time in milliseconds.";

command alarm.epoch{unsafe}() :int
address ALARMepoch
comment "Return the current time as UNIX epoch.";

command alarm.ctime{unsafe}() :str
address ALARMctime
comment "Return the current time as a C-time string.";

command alarm.prelude():void
address ALARMprelude
comment "Initialize alarm module."


Algebra mk Tue, 04/28/2020 - 18:26

8.37 BAT Algebra

This modules contains the most common algebraic BAT manipulation commands. We call them algebra, because all operations take values as parameters, and produce new result values, but do not modify their parameters. Unlike the previous Monet versions, we reduce the number of functions returning a BAT reference. This was previously needed to simplify recursive bat-expression and manage reference counts. In the current version we return only a BAT identifier when a new bat is being created.

All parameters to the modules are passed by reference. In particular, this means that string values are passed to the module layer as (str *) and we have to de-reference them before entering the gdk library. This calls for knowlegde on the underlying BAT typs`s


BAT mk Tue, 04/28/2020 - 18:31

8.39 Binary Association Tables

This module contains the commands and patterns to manage Binary Association Tables (BATs). The relational operations you can execute on BATs have the form of a neat algebra, described in

But a database system needs more that just this algebra, since often it is crucial to do table-updates (this would not be permitted in a strict algebra).

All commands needed for BAT updates, property management, basic I/O, persistence, and storage options can be found in this module.

All parameters to the modules are passed by reference. In particular, this means that string values are passed to the module layer as (str *) and we have to de-reference them before entering the gdk library. (Actual a design error in gdk to differentiate passing int/str) This calls for knowledge on the underlying BAT types`s

BAT Extensions

BAT Extensions mk Tue, 04/28/2020 - 18:33

The kernel libraries are unaware of the MAL runtime semantics. This calls for declaring some operations in the MAL module section and register them in the kernel modules explicitly.

A good example of this borderline case are BAT creation operations, which require a mapping of the type identifier to the underlying implementation type.

pattern, tt:any_1) :bat[:oid,:any_1]
address CMDBATnew
comment "Creates a new empty transient BAT, with head- and tail-types as indicated.";

pattern, tt:any_1, size:int) :bat[:oid,:any_1]
address CMDBATnew
comment "Creates a new BAT with sufficient space.";

pattern, tt:any_1, size:lng) :bat[:oid,:any_1]
address CMDBATnew
comment "Creates a new BAT and allocate space.";

pattern bat.new_persistent(ht:oid, tt:any_1) :bat[:oid,:any_1]
address CMDBATnew_persistent
comment "Creates a new empty transient BAT in the persistent farm, with head- and tail-types as indicated.";

pattern bat.new_persistent(ht:oid, tt:any_1, size:int) :bat[:oid,:any_1]
address CMDBATnew_persistent
comment "Creates a new BAT in the persistent farm with sufficient space.";

pattern bat.new_persistent(ht:oid, tt:any_1, size:lng) :bat[:oid,:any_1]
address CMDBATnew_persistent
comment "Creates a new BAT in the persistent farm and allocate space.";

pattern[:oid,:any_1] ) :bat[:oid,:any_1]
address CMDBATnewDerived;
pattern[:oid,:any_1], size:lng) :bat[:oid,:any_1]
address CMDBATnewDerived
comment "Create a new BAT by structure inheritance.";

address CMDBATderivedByName
comment "Localize a bat by name and produce a clone.";

pattern bat.partition(b:bat[:oid,:any_1]):bat[:oid,:any_1]...
address CMDBATpartition
comment "Create a serie of slices over the BAT argument. The BUNs are distributed evenly.";

address CMDBATderivedByName
comment "Localize a bat by name and produce a clone.";

pattern bat.partition(b:bat[:oid,:any_1]):bat[:oid,:any_1]...
address CMDBATpartition
comment "Create a serie of slices over the BAT argument. The BUNs are distributed evenly.";

pattern bat.partition(b:bat[:oid,:any_1], pieces:int, n:int):bat[:oid,:any_1]
address CMDBATpartition2
comment "Create the n-th slice over the BAT broken into severral pieces.";

command bat.imprints(b:bat[:oid,:bte]) :void
address CMDBATimprints;
command bat.imprints(b:bat[:oid,:sht]) :void
address CMDBATimprints;
command bat.imprints(b:bat[:oid,:int]) :void
address CMDBATimprints;
command bat.imprints(b:bat[:oid,:lng]) :void
address CMDBATimprints;
command bat.imprints(b:bat[:oid,:flt]) :void
address CMDBATimprints;
command bat.imprints(b:bat[:oid,:dbl]) :void
address CMDBATimprints
comment "Check/create an imprint index on the BAT";

command bat.imprintsize(b:bat[:oid,:bte]):lng
address CMDBATimprintsize;
command bat.imprintsize(b:bat[:oid,:sht]):lng
address CMDBATimprintsize;
command bat.imprintsize(b:bat[:oid,:int]):lng
address CMDBATimprintsize;
command bat.imprintsize(b:bat[:oid,:lng]):lng
address CMDBATimprintsize;
command bat.imprintsize(b:bat[:oid,:flt]):lng
address CMDBATimprintsize;
command bat.imprintsize(b:bat[:oid,:dbl]):lng
address CMDBATimprintsize
comment "Return the size of the imprints";


BBP mk Tue, 04/28/2020 - 18:34

The BBP module provides inspection routines over the BAT buffer pool.

The remainder of this module contains operations that rely on the MAL runtime setting, but logically belong to the kernel/bat module.

module bbp;

command open():void

pattern releaseAll():void

pattern iterator(nme:str):lng

address CMDbbpiterator comment "Locate the next element in the box.";

pattern prelude():void

address CMDbbpprelude comment "Initialize the bbp box.";

pattern bind(name:str):bat[:any_1,:any_2]

address CMDbbpbind comment "Locate the BAT using its logical name";

command getHeadType() :bat[:int,:str]

address CMDbbpHeadType comment "Map a BAT into its head type";

command getTailType() :bat[:int,:str]

address CMDbbpTailType comment "Map a BAT into its tail type";

command getNames() :bat[:int,:str]

address CMDbbpNames comment "Map BAT into its bbp name";

command getRNames() :bat[:int,:str]

address CMDbbpRNames comment "Map a BAT into its bbp physical name";

command getName( b:bat[:any_1,:any_2]):str

address CMDbbpName comment "Map a BAT into its internal name";

command getCount() :bat[:int,:lng]

address CMDbbpCount comment "Create a BAT with the cardinalities of all known BATs";

command getRefCount() :bat[:int,:int]

address CMDbbpRefCount comment "Create a BAT with the (hard) reference counts";

command getLRefCount() :bat[:int,:int]

address CMDbbpLRefCount comment "Create a BAT with the logical reference counts";

command getLocation() :bat[:int,:str]

address CMDbbpLocation comment "Create a BAT with their disk locations";

command getHeat() :bat[:int,:int]

address CMDbbpHeat comment "Create a BAT with the heat values";

command getDirty() :bat[:int,:str]

address CMDbbpDirty comment "Create a BAT with the dirty/ diffs/clean status";

command getStatus() :bat[:int,:str]

address CMDbbpStatus comment "Create a BAT with the disk/load status";

command getKind():bat[:int,:str]

address CMDbbpKind comment "Create a BAT with the persistency status";

command getRefCount(b:bat[:any_1,:any_2]) :int

address CMDgetBATrefcnt comment "Utility for debugging MAL interpreter";

command getLRefCount(b:bat[:any_1,:any_2]) :int

address CMDgetBATlrefcnt comment "Utility for debugging MAL interpreter";

command getDiskSpace() :lng

address CMDbbpDiskSpace comment "Estimate the amount of disk space occupied by dbfarm";

command getPageSize():int

address CMDgetPageSize comment "Obtain the memory page size";


Calculator mk Tue, 04/28/2020 - 18:42

8.53 Basic arithmetic

This module is an extended version of the V4 arithmetic module. It implements the arithmetic operations on the built-in types, chr, bte, sht, int, flt, dbl and lng. All combinations are implemented. Limited combinations are implemented for bit, oid and str.

[binary operators]
The implemented operators are first of all all comparison that return a TRUE/FALSE value (bit values), i.e. <=, <, ==, !=, >=, and >=.

The module also implements the operators +, -, * and /. The rules for the return types operators is as follows. If one of the input types is a floating point the result will be a floating point. The largest type of the input types is taken.

The max and min functions return the maximum and minimum of the two input parameters.

[unary operators]
This module also implements the unary abs() function, which calculates the absolute value of the given input parameter, as well as the - unary operator.

The inv unary operation calculates the inverse of the input value. An error message is given when the input value is zero.

[bitwise operators]
For integers there are some additional operations. The % operator implements the congruent modulo operation. The << and >> are the left and right bit shift. The or, and, xor and not for integers are implemented as bitwise boolean operations.
[boolean operators]
The or, and, xor and not for the bit atomic type in MIL (this corresponds to what is normally called boolean) are implemented as the logic operations.
[random numbers]
This module also contains the rand and srand functions. The srand() function initializes the random number generator using a seed value. The subsequent calls to rand() are pseudo random numbers (with the same seed the sequence can be repeated).

The general interpretation for the NIL value is "unknown". This semantics mean that any operation that receives at least one NIL value, will produce a NIL value in the output for sure.

The only exception to this rule are the "==" and "!=" equality test routines (it would otherwise become rather difficult to test whether a value is nil).


Clients mk Tue, 04/28/2020 - 18:45

8.8 Client Management

Each online client is represented with an entry in the clients table. The client may inspect his record at run-time and partially change its properties. The administrator sees all client records and has the right to adjust global properties.

module clients;

pattern setListing(flag:int):int
address CLTsetListing comment "Turn on/off echo of MAL instructions: 2 - show mal instruction, 4 - show details of type resolutoin, 8 - show binding information.";
pattern setHistory(s:str)
address CLTsetHistory comment "Designate console history file for readline.";
pattern getId():int
address CLTgetClientId comment "Return a number that uniquely represents the current client.";
pattern getInfo( ):bat[:str,:str]
address CLTInfo comment "Pseudo bat with client attributes.";
pattern getScenario():str
address CLTgetScenario comment "Retrieve current scenario name.";
pattern setScenario(msg:str):str
address CLTsetScenario comment "Switch to other scenario handler, return previous one.";
pattern quit():void
address CLTquit comment "Terminate the client session.";
pattern quit(idx:int):void
address CLTquit comment "Terminate the session for a single client using a soft error. It is the privilige of the console user.";

Administrator operations

command getLogins( ):bat[:int,:str]
address CLTLogin comment "Pseudo bat of client login time.";
command getLastCommand( ):bat[:int,:str]
address CLTLastCommand comment "Pseudo bat of client's last command time.";
command getActions( ):bat[:int,:int]
address CLTActions comment "Pseudo bat of client's command counts.";
command getTime( ):bat[:int,:lng]
address CLTTime comment "Pseudo bat of client's total time usage(in usec).";
command getUsers( ):bat[:int,:str]
address CLTusers comment "Pseudo bat of users logged in.";
pattern stop(id:int)
address CLTstop comment "Stop the query execution at the next eligble statement.";
pattern suspend(id:int):void
address CLTsuspend comment "Put a client process to sleep for some time. It will simple sleep for a second at a time, until the awake bit has been set in its descriptor";
command wakeup(id:int):void
address CLTwakeup comment "Wakeup a client process";
pattern setTimeout(q:int,s:int):void
address CLTsetTimeout comment "Abort a query after q seconds (q=0 means run undisturbed). The session timeout aborts the connection after spending too many seconds on query processing.";
pattern getTimeout()(q:int,s:int)
address CLTgetTimeout comment "A query is aborted after q seconds (q=0 means run undisturbed). The session timeout aborts the connection after spending too many seconds on query processing.";
command shutdown(forced:bit):void
address CLTshutdown comment "Close all client connections. If forced=false the clients are moved into FINISHING mode, which means that the process stops at the next cycle of the scenario. If forced=true all client processes are immediately killed";


Debugger mk Tue, 04/28/2020 - 18:47

8.13 MAL debugger interface

This module provides access to the functionality offered by the MonetDB debugger and interpreter status. It is primarilly used in interactive sessions to activate the debugger at a given point. Furthermore, the instructions provide the necessary handle to generate information for post-mortum analysis.

To enable ease of debugging and performance monitoring, the MAL interpreter comes with a hardwired gdb-like text-based debugger. A limited set of instructions can be included in the programs themselves, but beware that debugging has a global effect. Any concurrent user will be affected by breakpoints being set.

The prime scheme to inspect the MAL interpreter status is to use the MAL debugger directly. However, in case of automatic exception handling it helps to be able to obtain BAT versions of the critical information, such as stack frame table, stack trace, and the instruction(s) where an exception occurred. The inspection typically occurs in the exception handling part of the MAL block.

Beware, a large class of internal errors can not easily captured this way. For example, bus-errors and segmentation faults lead to premature termination of the process. Similar, creation of the post-mortum information may fail due to an inconsistent state or insufficient resources.

module mdb;

pattern start():void
address MDBstart
comment "Start interactive debugger";
pattern start(clientid:int):void
address MDBstart
comment "Start interactive debugger on a client";
pattern start(mod:str,fcn:str):void
address MDBstartFactory
comment "Start interactive debugger on a running factory";

pattern stop():void
address MDBstop
comment "Stop the interactive debugger.";

pattern inspect(mod:str,fcn:str):void
address MDBinspect
comment "Run the debugger on a specific function";

command modules():bat[:oid,:str]
address CMDmodules
comment "List available modules";

pattern setTrap(mod:str, fcn:str, b:bit):void
address MDBtrapFunction
comment "Suspend upon a call to the MAL function.";

pattern setTrap(idx:int):void
address mdbTrapClient
comment "Call debugger for a specific process.";

pattern setTrace(b:bit):void
address MDBsetTrace
comment "Turn on/off tracing of current routine";

pattern setTrace(b:str):void
address MDBsetVarTrace
comment "Turn on/off tracing of a variable ";

pattern setCatch(b:bit):void
address MDBsetCatch
comment "Turn on/off catching exceptions";

command getDebug():int
address MDBgetDebug
comment "Get the kernel debugging bit-set. See the MonetDB configuration file for details";

command setDebug(flg:str):int
address MDBsetDebugStr
comment "Set the kernel debugging bit-set and return its previous value. The recognized options are: threads, memory, properties,
io, transactions, modules, algorithms, estimates.";
command setDebug(flg:int):int
address MDBsetDebug
comment "Set the kernel debugging bit-set and return its previous value.";

command getException(s:str):str
address MDBgetExceptionVariable
comment "Extract the variable name from the exception message";
command getReason(s:str):str
address MDBgetExceptionReason
comment "Extract the reason from the exception message";
command getContext(s:str):str
address MDBgetExceptionContext
comment "Extract the context string from the exception message";

pattern list():void
address MDBlist
comment "Dump the current routine on standard out.";
pattern listMapi():void
address MDBlistMapi
comment "Dump the current routine on standard out with Mapi prefix.";
pattern list(M:str,F:str):void
address MDBlist3
comment "Dump the routine M.F on standard out.";
pattern List():void
address MDBlistDetail
comment "Dump the current routine on standard out.";
pattern List(M:str,F:str):void
address MDBlist3Detail
comment "Dump the routine M.F on standard out.";
pattern var():void
address MDBvar
comment "Dump the symboltable of current routine on standard out.";
pattern var(M:str,F:str):void
address MDBvar3
comment "Dump the symboltable of routine M.F on standard out.";
pattern lifespan(M:str,F:str):void
address MDBlifespan
comment "Dump the current routine lifespan information on standard out.";

pattern grab():void
address mdbGrab
comment "Call debugger for a suspended process.";
pattern trap():void
address mdbTrap
comment "A suspended process for debugging.";

pattern dot(s:str):void
address MDBshowFlowGraph
comment "Dump the data flow of the current routine in a format recognizable by the command 'dot' to the file s";

pattern dot(M:str,F:str,s:str):void
address MDBshowFlowGraph
comment "Dump the data flow of the function M.F in a format recognizable by the command 'dot' on the file s";

pattern getStackDepth():int
address MDBStkDepth
comment "Return the depth of the calling stack.";

pattern getStackFrame(i:int)(:bat[:oid,:str] ,:bat[:oid,:str])
address MDBgetStackFrameN;
pattern getStackFrame()(:bat[:oid,:str] ,:bat[:oid,:str])
address MDBgetStackFrame
comment "Collect variable binding of current (n-th) stack frame.";
pattern getStackTrace()(:bat[:oid,:str], :bat[:oid,:str])
address MDBStkTrace;

pattern dump()
address MDBdump
comment "Dump instruction, stacktrace, and stack";

pattern getDefinition():bat[:oid,:str]
address MDBgetDefinition
comment "Returns a string representation of the current function with typing information attached";


Factories mk Tue, 04/28/2020 - 18:52

8.9 Factory management

The factory infrastructure can be inspected and steered with the commands provided here.

module factories;

command getPlants()(mod:bat[:oid,:str], fcn:bat[:oid,:str])
address FCTgetPlants comment "Retrieve the names for all active factories.";
command getCaller():int
address FCTgetCaller comment "Retrieve the unique identity of the factory caller.";
command getOwners():bat[:oid,:str]
address FCTgetOwners comment "Retrieve the factory owners table.";
command getArrival():bat[:oid,:timestamp]
address FCTgetArrival comment "Retrieve the time stamp the last call was made.";
command getDeparture():bat[:oid,:timestamp]
address FCTgetDeparture comment "Retrieve the time stamp the last answer was returned.";
pattern shutdown(m:str, f:str):void
address FCTshutdown comment "Close a factory.";


Groups mk Tue, 04/28/2020 - 19:23

8.55 The group module

This module contains the primitives to construct, derive, and perform statistical operations on BATs representing groups. The default scheme in Monet is to assume the head to represent the group identifier and the tail an element in the group.

Groups play an important role in datamining, where they are used to construct cross-tables. Such cross tables over a single BAT are already supported by the histogram function. This module provides extensions to support identification of groups in a (multi-)dimensional space.

The module implementation has a long history. The first implementation provided several alternatives to produce/derive the grouping. A more complete (and complex) scheme was derived during its extensive use in the context of the Data Distilleries product. The current implementation is partly a cleanup of this code-base, but also enables provides better access to the intermediate structures produced in the process, i.e. the histogram and the sub-group mapping. They can be used for various optimization schemes at the MAL level.

The prime limitation of the current implementation is that an underlying database of oid->any BATs is assumed. This enables representation of each group using an oid, and the value representation of the group can be accordingly be retrieved easily. An optimized implementation in which we use positional integer id's (as embodied by Monet's void type) is also available.

This limitation on (v)oid-headers is marginal. The primitive GRPsplit produces for any BAT two copies with both a (v)oid header.

8.55.1 Algorithms

There are several approaches to build a cross table. The one chosen here is aimed at incremental construction, such that re-use of intermediates becomes possible. Starting with the first dimension, a BAT is derived to represent the various groups, called a GRP BAT or cross-table BAT.

8.55.2 Cross Table (GRP)

A cross table is an <oid,oid> BAT where the first (head) denotes a tuple in the cross table and the second (tail) marks all identical lists. The tail-oids contain group identifiers; that is, this value is equal iff two tuples belong to the same group. The group identifiers are chosen from the domain of the tuple-identifiers. This simplifies getting back to the original tuples, when talking about a group. If the tuple-oid of 'John' is chosen as a group-id, you might view this as saying that each member of the group is 'like John' with respect to the grouping-criterion.

Successively the subgroups can be identified by modifying the GRP BAT or to derive a new GRP BAT for the subgroups. After all groups have been identified this way, a BAT histogram operation can be used to obtain the counts of each data cube. Other aggregation operations using the MIL set aggregate construct (bat) can be used as well; note for instance that histogram == (b.reverse()).

The Monet interface module specification is shown below. Ideally we should defined stronger type constraints, e.g. command[,:any_1]

The group macro is split along three dimensions:


[type:] Type specific implementation for selecting the right hash function and data size etc.;
[clustered:] The select the appropriate algorithm, i.e., with or without taking advantage of an order of values in the parent groups;
[physical properties:] Values , choosing between a fixed predefined and a custom hashmask. Custom allows the user to determine the size of the hashmask (and indirectly the estimated size of the result). The hashmask is 2^n - 1 where n is given by the user, or 1023 otherwise, and the derived result size is 4 ... 2^n.

Further research should point out whether fitting a simple statistical model (possibly a simple mixture model) can help choose these parameters automatically; the current idea is that the user (which could be a domain-specific extension of the higher-level language) knows the properties of the data, especially for IR in which the standard grouping settings differ significantly from the original datamining application.


I/O mk Tue, 04/28/2020 - 18:55

8.11 Input/Output module

The IO module provides simple ascii-io rendering options. It is modeled after the tuple formats, but does not attempt to outline the results. Instead, it is geared at speed, which also means that some functionality regarding the built-in types is duplicated from the atoms definitions.

A functional limited form of formatted printf is also provided. It accepts at most one variable. A more complete approach is the tablet module.

The commands to load and save a BAT from/to an ASCII dump are efficient, but work only for binary tables.

module io;
pattern stdin():bstream
address io_stdin comment "return the input stream to the database client";
pattern stderr():streams
address io_stderr comment "return the error stream for the database console";
pattern stdout():streams
address io_stdout comment "return the output stream for the database client";
pattern print(val:any_1,lst:any...):void
address IOprint_val comment "Print a MAL value tuple .";
pattern print(b1:bat[:any_1,:any]...):void
address IOtable comment "BATs are printed with '#' for legend lines, and the BUNs on seperate lines between brackets, containing each to comma separated values (head and tail). If multiple BATs are passed for printing, print() performs an implicit natural join, producing a multi attribute table.";
pattern ftable( filep:streams, b1:bat[:any_1,:any], b:bat[:any_1,:any]... ):void
address IOftable comment "Print an n-ary table to a file.";
pattern print(order:int,b:bat[:any_1,:any], b2:bat[:any_1,:any]...):void
address IOotable comment "The same as normal table print, but enforces to use the order of BAT number [1..argc] to do the printing.";
pattern table(b1:bat[:any_1,:any], b2:bat[:any_1,:any]...):void
address IOttable comment "Print an n-ary table. Like print, but does not print oid column";
pattern table(order:int, b1:bat[:any_1,:any], b2:bat[:any_1,:any]...):void
address IOtotable comment "Print an n-ary table.";
pattern ftable(fp:streams, order:int, b1:bat[:any_1,:any], b:bat[:any_1,:any]...):void
address IOfotable comment "Print an n-ary table to a file.";
pattern print(val:any_1):void
address IOprint_val comment "Print a MAL value tuple .";
pattern print(val:bat[:any_1,:any_2]):void
address IOprint_val comment "Print a MAL value tuple .";
pattern prompt(val:any_1):void
address IOprompt_val comment "Print a MAL value without brackets.";
pattern printf(fmt:str,val:any...):void
address IOprintf comment "Select default format ";
pattern printf(fmt:str):void
address IOprintf comment "Select default format ";
pattern printf(filep:streams,fmt:str,val:any...):void
address IOprintfStream comment "Select default format ";
pattern printf(filep:streams,fmt:str):void
address IOprintfStream comment "Select default format ";
command data(fname:str):str
address IOdatafile comment "Signals receipt of tuples in a file fname. It returns the name of the file, if it still exists.";
command export(b:bat[:any_1,:any_2], filepath:str):bit
address IOexport comment "Export a BAT as ASCII to a file. If the 'filepath' is not absolute, it is put into the .../dbfarm/$DB directory. Success of failure is indicated.";
command import(b:bat[:any_1,:any_2], filepath:str) :bat[:any_1,:any_2]
address IOimport comment "Import a BAT from an ASCII dump. The new tuples are *inserted* into the parameter BAT. You have to create it! Its signature must match the dump, else parsing errors will occur and FALSE is returned.";


Imprints mk Tue, 04/28/2020 - 18:56

Imprints are a novel secondary index structure to speed up scans over large tables described in the ACM SIGMOD paper "Column imprints: a secondary index structure". They are automatically created over columns of fixed type and carry a maximal storage overhead of 12%. An effective compression scheme brings this often down to much less storage.

command bat.imprints(b:bat[:oid,:bte]) :void
address CMDBATimprints;
command bat.imprints(b:bat[:oid,:sht]) :void
address CMDBATimprints;
command bat.imprints(b:bat[:oid,:int]) :void
address CMDBATimprints;
command bat.imprints(b:bat[:oid,:lng]) :void
address CMDBATimprints;
command bat.imprints(b:bat[:oid,:flt]) :void
address CMDBATimprints;
command bat.imprints(b:bat[:oid,:dbl]) :void
address CMDBATimprints
comment "Check for existence or create an imprint index on the BAT.";

command bat.imprintsize(b:bat[:oid,:bte]):lng
address CMDBATimprintsize;
command bat.imprintsize(b:bat[:oid,:sht]):lng
address CMDBATimprintsize;
command bat.imprintsize(b:bat[:oid,:int]):lng
address CMDBATimprintsize;
command bat.imprintsize(b:bat[:oid,:lng]):lng
address CMDBATimprintsize;
command bat.imprintsize(b:bat[:oid,:flt]):lng
address CMDBATimprintsize;
command bat.imprintsize(b:bat[:oid,:dbl]):lng
address CMDBATimprintsize
comment "Return the storage size of the imprints index structure.";


Inspect mk Tue, 04/28/2020 - 18:58

8.10 Inspection

This module introduces a series of commands that provide access to information stored within the interpreter data structures. It's primary use is debugging. In all cases, the pseudo BAT operation is returned that should be garbage collected after being used.

The main performance drain would be to use a pseudo BAT directly to successively access it components. This can be avoided by first assigning the pseudo BAT to a variable.

module inspect;

command getWelcome():str
address INSPECTgetWelcome comment "Return the server message of the day string";
pattern getDefinition(mod:str,fcn:str) :bat[:str,:str]
address INSPECTgetDefinition comment "Returns a string representation of a specific function.";
pattern getSignature(mod:str,fcn:str) :bat[:str,:str]
address INSPECTgetSignature comment "Returns the function signature(s).";
pattern getAddress(mod:str,fcn:str) :bat[:str,:str]
address INSPECTgetAddress comment "Returns the function signature(s).";
pattern getComment(mod:str,fcn:str) :bat[:str,:str]
address INSPECTgetComment comment "Returns the function help information.";
pattern getSource(mod:str,fcn:str):str
address INSPECTgetSource comment "Return the original input for a function.";
pattern getKind():bat[:oid,:str]
address INSPECTgetkind comment "Obtain the instruction kind.";
pattern getModule():bat[:oid,:str]
address INSPECTgetAllModules comment "Obtain the function name.";
pattern getFunction():bat[:oid,:str]
address INSPECTgetAllFunctions comment "Obtain the function name.";
pattern getSignatures():bat[:oid,:str]
address INSPECTgetAllSignatures comment "Obtain the function signatures.";
pattern getAddresses():bat[:oid,:str]
address INSPECTgetAllAddresses comment "Obtain the function address.";
pattern getSize():lng
address INSPECTgetSize comment "Return the storage size for the current function (in bytes).";
pattern getSize(mod:str):bat[:str,:lng]
address INSPECTgetModuleSize comment "Return the storage size for a module (in bytes).";
pattern getSize(mod:str,fcn:str):lng
address INSPECTgetFunctionSize comment "Return the storage size for a function (in bytes).";
pattern getType(v:bat[:any_1,:any_2]) (ht:str, tt:str)
address INSPECTtypeName comment "Return the concrete type of a variable (expression).";
pattern getType(v:any_1) :str
address INSPECTtypeName comment "Return the concrete type of a variable (expression).";
command getTypeName(v:int):str
address INSPECTtypename comment "Get the type name associated with a type id.";
pattern getTypeIndex(v:bat[:any_1,:any_2]) (ht:int, tt:int)
address INSPECTtypeIndex comment "Return the type index of a BAT head and tail.";
pattern getTypeIndex(v:any_1):int
address INSPECTtypeIndex comment "Return the type index of a variable. For BATs, return the type index for its tail.";
pattern equalType(l:any, r:any):bit
address INSPECTequalType comment "Return true if both operands are of the same type";
command getAtomNames():bat[:int,:str]
address INSPECTatom_names comment "Collect a BAT with the atom names.";
command getAtomSuper():bat[:int,:str]
address INSPECTatom_sup_names comment "Collect a BAT with the atom names.";
command getAtomSizes():bat[:int,:int]
address INSPECTatom_sizes comment "Collect a BAT with the atom sizes.";
command getEnvironment():bat[:str,:str]
address INSPECTgetEnvironment comment "Collect the environment variables.


Iterators mk Tue, 04/28/2020 - 19:08

BAT Iterators

Many low level algorithms rely on an iterator to break a collection into smaller pieces. Each piece is subsequently processed by a block. For very large BATs it may make sense to break it into chunks and process them separately to solve a query. An iterator pair is provided to chop a BAT into fixed size elements. Each chunk is made available as a BATview. It provides read-only access to an underlying BAT. Adjusting the bounds is cheap, once the BATview descriptor has been constructed.

The smallest granularity is a single (oid,value) pair, which can be used to realize an iterator over the individual BAT elements. For larger sized chunks, the operators return a view.

All iterators require storage space to administer the location of the next element. The BAT iterator module uses a simple lng variable, which also acts as a cursor for barrier statements.

The larger chunks produced are currently static, i.e. their size is a parameter of the call. Dynamic chunk sizes are interesting for time-series query processing. (See another module)

module iterator;

command new(b:bat[:oid,:any_2], size:lng) (:lng,:bat[:oid,:any_2])
address ITRnewChunk
comment "Create an iterator with fixed granule size. The result is a view.";

command next(b:bat[:oid,:any_2], size:lng) (:lng, :bat[:oid,:any_2])
address ITRnextChunk
comment "Produce the next chunk for processing.";

pattern new(b:bat[:oid,:any_2]) (h:oid, t:any_2)
address ITRbunIterator
comment "Process the buns one by one extracted from a void table.";

pattern next(b:bat[:oid,:any_2]) (h:oid, t:any_2)
address ITRbunNext
comment "Produce the next bun for processing.";

command next(step:oid,last:oid):oid
address ITRnext_oid;
command next(step:sht,last:sht):sht
address ITRnext_sht;
command next(step:int,last:int):int
address ITRnext_int;
command next(step:lng,last:lng):lng
address ITRnext_lng;
command next(step:flt,last:flt):flt
address ITRnext_flt;
command next(step:dbl,last:dbl):dbl
address ITRnext_dbl
comment "Advances the iterator with a fixed value";

Language Extension

Language Extension mk Tue, 04/28/2020 - 19:24

8.12 Language Extensions

Iterators over scalar ranges are often needed, also at the MAL level. The barrier and control primitives are sufficient to mimic them directly.

The modules located in the kernel directory should not rely on the MAL datastructures. That's why we have to deal with some bat operations here and delegate the signature to the proper module upon loading.

Running a script is typically used to initialize a context. Therefore we need access to the runtime context. For the call variants we have to determine an easy way to exchange the parameter/return values.

module language;

pattern call(s:str):void
address CMDcallString comment "Evaluate a MAL string program.";
pattern call(s:bat[:oid,:str]):void
address CMDcallBAT comment "Evaluate a program stored in a BAT.";
pattern source(f:str):void
address CMDevalFile comment "Merge the instructions stored in the file with the current program.";

command raise(msg:str) :str
address CMDraise
comment "Raise an exception labeled
    with a specific message.";
command assert(v:bit,term:str):void
address MALassertBit;
command assert(v:sht,term:str):void
address MALassertSht;
command assert(v:int,term:str):void
address MALassertInt;
command assert(v:lng,term:str):void
address MALassertLng;
command assert(v:str,term:str):void
address MALassertStr;
command assert(v:oid,term:str):void
address MALassertOid;
pattern assert(v:any_1,pname:str,oper:str,val:any_2):void
address MALassertTriple
comment "Assertion test.";

pattern dataflow():bit
address MALstartDataflow
comment "The current guarded block is executed using dataflow control. ";

pattern sink(v:any...):void
address MALgarbagesink
comment "Variables to be considered together when triggering garbage collection.
Used in the dataflow blocks to avoid early release of values.";

pattern pass(v:any_1):any_1
address MALpass
comment "Cheap instruction to disgard storage while retaining the dataflow dependency";

pattern register(m:str,f:str,code:str,help:str):void
address CMDregisterFunction
comment"Compile the code string to MAL and register it as a function.";



Logger mk Tue, 04/28/2020 - 19:25

8.57 The Transaction Logger

In the philosophy of MonetDB, transaction management overhead should only be paid when necessary. Transaction management is for this purpose implemented as a separate module and applications are required to obey the transaction policy, e.g. obtaining/releasing locks.

This module is designed to support efficient logging of the SQL database. Once loaded, the SQL compiler will insert the proper calls at transaction commit to include the changes in the log file.

The logger uses a directory to store its log files. One master log file stores information about the version of the logger and the transaction log files. This file is a simple ascii file with the following format: 6DIGIT-VERSION\n[log file number \n]*]* The transaction log files have a binary format, which stores fixed size logformat headers (flag,nr,bid), where the flag is the type of update logged. The nr field indicates how many changes there were (in case of inserts/deletes). The bid stores the bid identifier.

The key decision to be made by the user is the location of the log file. Ideally, it should be stored in fail-safe environment, or at least the log and databases should be on separate disk columns.

This file system may reside on the same hardware as the database server and therefore the writes are done to the same disk, but could also reside on another system and then the changes are flushed through the network. The logger works under the assumption that it is called to safeguard updates on the database when it has an exclusive lock on the latest version. This lock should be guaranteed by the calling transaction manager first.

Finding the updates applied to a BAT is relatively easy, because each BAT contains a delta structure. On commit these changes are written to the log file and the delta management is reset. Since each commit is written to the same log file, the beginning and end are marked by a log identifier.

A server restart should only (re)process blocks which are completely written to disk. A log replay therefore ends in a commit or abort on the changed bats. Once all logs have been read, the changes to the bats are made persistent, i.e. a bbp sub-commit is done.


MAPI Interface

MAPI Interface mk Tue, 04/28/2020 - 19:32

8.15 MAPI interface

The complete Mapi library is available to setup communication with another Mserver.

Clients may initialize a private listener to implement specific services. For example, in an OLTP environment it may make sense to have a listener for each transaction type, which simply parses a sequence of transaction parameters.

Authorization of access to the server is handled as part of the client record initialization phase.

This library internally uses pointer handles, which we replace with an index in a locally maintained table. It provides a handle to easily detect havoc clients.

A cleaner and simplier interface for distributed processing is available in the module remote.

module mapi;

command listen():int
address SERVERlisten_default comment "Start a Mapi server with the default settings.";
command listen(port:int):int
address SERVERlisten_port comment "Start a Mapi listener on the port given.";
command listen(port:int, maxusers:int):int
address SERVERlisten2 comment "Start a Mapi listener.";
command listen(port:int, maxusers:int, cmd:str):int
address SERVERlisten3 comment "Start the Mapi listener on <port> for <maxusers>. For a new client connection MAL procedure <cmd>(Stream s_in, Stream s_out) is called.If no <cmd> is specified a new client thread is forked.";
command stop():void
address SERVERstop comment "Terminate connection listeners.";
command suspend():void
address SERVERsuspend comment "Suspend accepting connections.";
command resume():void
address SERVERresume comment "Resume connection listeners.";
command malclient(in:streams, out:streams):void
address SERVERclient comment "Start a Mapi client for a particular stream pair.";
command trace(mid:int,flag:int):void
address SERVERtrace comment "Toggle the Mapi library debug tracer.";
pattern reconnect(host:str, port:int, usr:str, passwd:str,lang:str):int
address SERVERreconnectWithoutAlias comment "Re-establish connection with a remote mserver.";
pattern reconnect(host:str, port:int, db_alias:str, usr:str, passwd:str,lang:str):int
address SERVERreconnectAlias comment "Re-establish connection with a remote mserver.";
command reconnect(mid:int):void
address SERVERreconnect comment "Re-establish a connection.";
pattern connect(host:str, port:int, usr:str, passwd:str,lang:str):int
address SERVERconnect comment "Establish connection with a remote mserver.";
command disconnect(dbalias:str):int
address SERVERdisconnectWithAlias comment "Close connection with a remote Mserver.";
command disconnect():int
address SERVERdisconnectALL comment "Close connections with all remote Mserver.";
command setAlias(dbalias:str)
address SERVERsetAlias comment "Give the channel a logical name.";
command lookup(dbalias:str):int
address SERVERlookup comment "Retrieve the connection identifier.";
command disconnect(mid:int):void
address SERVERdisconnect comment "Terminate the session.";
command destroy(mid:int):void
address SERVERdestroy comment "Destroy the handle for an Mserver.";
command ping(mid:int):int
address SERVERping comment "Test availability of an Mserver.";
command query(mid:int, qry:str):int
address SERVERquery comment "Send the query for execution";
command query_handle(mid:int, qry:str):int
address SERVERquery_handle comment "Send the query for execution.";
pattern query_array(mid:int, qry:str, arg:str...):int
address SERVERquery_array comment "Send the query for execution replacing '?' by arguments.";
command prepare(mid:int, qry:str):int
address SERVERprepare comment "Prepare a query for execution.";
command finish(hdl:int):int
address SERVERfinish comment "Remove all remaining answers.";
command get_field_count(hdl:int):int
address SERVERget_field_count comment "Return number of fields.";
command get_row_count(hdl:int):lng
address SERVERget_row_count comment "Return number of rows.";
command rows_affected(hdl:int):lng
address SERVERrows_affected comment "Return number of affected rows.";
command fetch_row(hdl:int):int
address SERVERfetch_row comment "Retrieve the next row for analysis.";
command fetch_all_rows(hdl:int):lng
address SERVERfetch_all_rows comment "Retrieve all rows into the cache.";
command fetch_field(hdl:int,fnr:int):str
address SERVERfetch_field_str comment "Retrieve a single field.";
command fetch_field(hdl:int,fnr:int):int
address SERVERfetch_field_int comment "Retrieve a single int field.";
command fetch_field(hdl:int,fnr:int):lng
address SERVERfetch_field_lng comment "Retrieve a single lng field.";
command fetch_field(hdl:int,fnr:int):sht
address SERVERfetch_field_sht comment "Retrieve a single sht field.";
command fetch_field(hdl:int,fnr:int):void
address SERVERfetch_field_void comment "Retrieve a single void field.";
command fetch_field(hdl:int,fnr:int):oid
address SERVERfetch_field_oid comment "Retrieve a single void field.";
command fetch_field(hdl:int,fnr:int):chr
address SERVERfetch_field_chr comment "Retrieve a single chr field.";
command fetch_field_array(hdl:int):bat[:int,:str]
address SERVERfetch_field_bat comment "Retrieve all fields for a row.";
command fetch_line(hdl:int):str
address SERVERfetch_line comment "Retrieve a complete line.";
command fetch_reset(hdl:int):int
address SERVERfetch_reset comment "Reset the cache read line.";
command next_result(hdl:int):int
address SERVERnext_result comment "Go to next result set.";
command error(mid:int):int
address SERVERerror comment "Check for an error in the communication.";
command getError(mid:int):str
address SERVERgetError comment "Get error message.";
command explain(mid:int):str
address SERVERexplain comment "Turn the error seen into a string.";
pattern put(mid:int, nme:str, val:any_1):void
address SERVERput comment "Send a value to a remote site.";
pattern put(nme:str, val:any_1):str
address SERVERputLocal comment "Prepare sending a value to a remote site.";
pattern rpc(key:int,qry:str...):any
address SERVERmapi_rpc_single_row comment "Send a simple query for execution and fetch result.";
pattern rpc(key:int,qry:str):bat[:any_1,:any_2]
address SERVERmapi_rpc_bat;
command rpc(key:int,qry:str):void
address SERVERquery comment "Send a simple query for execution.";
pattern bind(key:int,rschema:str,rtable:str,rcolumn:str,i:int):bat[:any_1,:any_2]
address SERVERbindBAT comment "Bind a remote variable to a local one.";
pattern bind(key:int,rschema:str,rtable:str,i:int):bat[:any_1,:any_2]
address SERVERbindBAT comment "Bind a remote variable to a local one.";
pattern bind(key:int,remoteName:str):bat[:any_1,:any_2]
address SERVERbindBAT comment "Bind a remote variable to a local one.";



Manual mk Tue, 04/28/2020 - 19:33

8.14 Manual Inspection

This module introduces a series of commands that provide access to the help information stored in the runtime environment.

The manual bulk operations ease offline inspection of all function definitions. It includes an XML organized file, because we expect external tools to massage it further for presentation.

module manual;

pattern help(text:str)
address MANUALhelp comment "Produces a list of all <module>.<function> that match the text pattern. The wildcard '*' can be used for <module> and <function>. Using the '(' asks for signature information and using ')' asks for the complete help record.";
pattern search(text:str)
address MANUALsearch comment "Search the manual for command descriptions that match the regular expression 'text'";
pattern createXML(mod:str):void
address MANUALcreate1 comment "Generate a synopsis of a module";
pattern createXML():void
address MANUALcreate0 comment "Produces a XML-formatted manual over all modules loaded.";
pattern section(mod:str):void
address MANUALcreateSection comment "Generate a synopsis of a module for the reference manual";
pattern index():void
address MANUALcreateIndex comment "Produces an overview of all names grouped by module.";
pattern summary():void
address MANUALcreateSummary comment "Produces a manual with help lines grouped by module.";
pattern completion(pat:str):bat[:int,:str]
address MANUALcompletion comment "Produces the wordcompletion table.";

PCRE Library

PCRE Library mk Tue, 04/28/2020 - 19:34

8.19 PCRE library interface

The PCRE library is a set of functions that implement regular expression pattern matching using the same syntax and semantics as Perl, with just a few differences. The current implementation of PCRE (release 4.x) corresponds approximately with Perl 5.8, including support for UTF-8 encoded strings. However, this support has to be explicitly enabled; it is not the default.


Profiler mk Tue, 04/28/2020 - 19:35

8.18 Performance profiler

A key issue in developing fast programs using the Monet database back-end requires a keen eye on where performance is lost. Although performance tracking and measurements are highly application dependent, a simple to use tool makes life a lot easier (See stethoscope and tomograph).

Activation of the performance monitor has a global effect, i.e. all concurrent actions on the kernel are traced, but the events are only sent to the client who initiated the profiler thread.

The profiler event can be handled in several ways. The default strategy is to ship the event record immediately over a stream to a performance monitor. An alternative strategy is preparation of off-line performance analysis, where event information is retained in tables.

To reduce the interference of performance measurement with the experiments, the user can use an event cache, which is emptied explicitly upon need.

module profiler;
pattern activate(name:str...):void
pattern deactivate(name:str...):void
pattern openStream():void
address CMDopenProfilerStream comment "Send the events to output stream";
pattern openStream(fnme:str):void
command closeStream():void
address CMDcloseProfilerStream comment "Stop sending the event records";
pattern setAll():void
pattern setFilter(v:any):void
address CMDsetFilterVariable comment "Generate an event record for every instruction where v is used."
pattern clrFilter(mod:str,fcn:str):void
pattern setStartPoint(mod:str,fcn:str):void
pattern start():void
address CMDstartProfiler comment "Start performance tracing";
command noop():void
pattern stop():void
address CMDstopProfiler comment "Stop performance tracing";
command reset():void
address CMDclearTrace comment "Clear the profiler traces";
command dumpTrace():void
command getTrace(e:str):bat[:int,:any_1]
address CMDgetTrace comment "Get the trace details of a specific event";
pattern getEvent()(:lng,:lng,:lng)
command cleanup():void
address CMDcleanup comment "Remove the temporary tables for profiling";
command getDiskReads():lng
address CMDgetDiskReads comment "Obtain the number of physical reads";
command getDiskWrites():lng
address CMDgetDiskWrites comment "Obtain the number of physical reads";
command getUserTime():lng
address CMDgetUserTime comment "Obtain the user timing information.";
command getSystemTime():lng
address CMDgetSystemTime comment "Obtain the user timing information.";
pattern getFootprint():lng


Remote mk Tue, 04/28/2020 - 19:36

8.20 Remote querying functionality

Communication with other mservers at the MAL level is a delicate task. However, it is indispensable for any distributed functionality. This module provides an abstract way to store and retrieve objects on a remote site. Additionally, functions on a remote site can be executed using objects available in the remote session context. This yields in four primitive functions that form the basis for distribution methods: get, put, register and exec.

The get method simply retrieves a copy of a remote object. Objects can be simple values, strings or BATs. The same holds for the put method, but the other way around. A local object can be stored on a remote site. Upon a successful store, the put method returns the remote identifier for the stored object. With this identifier the object can be addressed, e.g. using the get method to retrieve the object that was stored using put.

The get and put methods are symmetric. Performing a get on an identifier that was returned by put, results in an object with the same value and type as the one that was put. The result of such an operation is equivalent to making an (expensive) copy of the original object.

The register function takes a local MAL function and makes it known at a remote site. It ensures that it does not overload an already known operation remotely, which could create a semantic conflict. Deregister a function is forbidden, because it would allow for taking over the remote site completely. C-implemented functions, such as io.print() cannot be remotely stored. It would require even more complicated (byte?) code shipping and remote compilation to make it work. Currently, the remote procedure may only returns a single value.

The choice to let exec only execute functions was made to avoid problems to decide what should be returned to the caller. With a function it is clear and simple to return that what the function signature prescribes. Any side effect (e.g. io.print calls) may cause havoc in the system, but are currently ignored.

This leads to the final contract of this module. The methods should be used correctly, by obeying their contract. Failing to do so will result in errors and possibly undefined behaviour.

The resolve() function can be used to query Merovingian. It returns one or more databases discovered in its vincinity matching the given pattern.

module remote;

# module loading and unloading funcs

command prelude():void
address RMTprelude comment "Initialise the remote module.";
command epilogue():void
address RMTepilogue comment "Release the resources held by the remote module.";

# global connection resolve function

command resolve(pattern:str):bat[:oid,:str]
address RMTresolve comment "resolve a pattern against Merovingian and return the URIs";

# session local connection instantiation functions

command connect(uri:str, user:str, passwd:str):str
address RMTconnect comment "returns a newly created connection for uri, using user name and password";
command connect(uri:str, user:str, passwd:str, scen:str):str
address RMTconnectScen comment "returns a newly created connection for uri, using user name, password and scenario";
command disconnect(conn:str):void
address RMTdisconnect comment "disconnects the connection pointed to by handle (received from a call to connect()";

# core transfer functions

pattern get(conn:str, ident:str):any
address RMTget comment "retrieves a copy of remote object ident";
pattern put(conn:str, object:any):str
address RMTput comment "copies object to the remote site and returns its identifier";
pattern register(conn:str, mod:str, fcn:str):void
address RMTregister comment "register <mod>.<fcn> at the remote site";
pattern exec(conn:str, mod:str, func:str):str
address RMTexec comment "remotely executes <mod>.<func> and returns the handle to its result";
pattern exec(conn:str, mod:str, func:str)(:str, :str)
address RMTexec comment "remotely executes <mod>.<func> and returns the handle to its result";
pattern exec(conn:str, mod:str, func:str, :str...):str
address RMTexec comment "remotely executes <mod>.<func> using the argument list of remote objects and returns the handle to its result";
pattern exec(conn:str, mod:str, func:str, :str...)(:str, :str)
address RMTexec comment "remotely executes <mod>.<func> using the argument list of remote objects and returns the handle to its result";


Transactions mk Tue, 04/28/2020 - 19:37

8.23 Transaction management

In the philosophy of Monet, transaction management overhead should only be paid when necessary. Transaction management is for this purpose implemented as a module. This code base is largely absolute and should be re-considered when serious OLTP is being supported. Note, however, the SQL front-end obeys transaction semantics.

module transaction;

command sync():bit
address TRNglobal_sync comment "Save all persistent BATs";
command commit():bit
address TRNglobal_commit comment "Global commit on all BATs";
command abort():bit
address TRNglobal_abort comment "Global abort on all BATs";
command subcommit(b:bat[:any_1,:str]):bit
address TRNsubcommit comment "commit only a set of BATnames, passed in the tail (to which you must have exclusive access!)";
pattern commit(c:any...)
address TRNtrans_commit comment "Commit changes in certain BATs.";
pattern abort(c:any...)
address TRNtrans_abort comment "Abort changes in certain BATs.";
pattern clean(c:any...)
address TRNtrans_clean comment "Declare a BAT clean without flushing to disk.";
command prev(b:bat[:any_1,:any_2]):bat[:any_1,:any_2]
address TRNtrans_prev comment "The previous stae of this BAT";
command alpha(b:bat[:any_1,:any_2]) :bat[:any_1,:any_2]
address TRNtrans_alpha comment "List insertions since last commit.";
command delta(b:bat[:any_1,:any_2]) :bat[:any_1,:any_2]
address TRNtrans_delta comment "List deletions since last commit.";