Skip to main content

Tachograph

The program tachograph monitors the event stream to determine the progress of a query execution. This tools is available as of Jul2015 release.

A synopsis of the calling conventions:

    tachograph [options]
      -d | --dbname=<database_name>
      -u | --user=<user>
      -p | --port=<portnr>
      -h | --host=<hostname>
      -b | --beat=<delay> in milliseconds (default 5000)
      -i | --interactive=<0 | 1> show trace on stdout (default 1)
      -o | --output=<webfile>
      -c | --cache=<query pool location> (default ‘cache’)
      -q | --queries=<query pool capacity>
      -w | --wait=<delay time> in milliseconds (default 5000)
      -? | --help

Assume a MonetDB server is running to serve the database “voc”. First, in terminal 1 start tachograph on the database as the following:

    $ tachograph -d voc
    -- opened UDP profile stream numa-db:50010 for localhost

Then, in terminal 2, start an mclient to execute a (long-running) query:

    $ mclient -d voc -s “select * from tables a, tables b, tables c, tables d, tables e;”

This triggers tachograph to respond with something like below in terminal 1:

    CACHE ID:0
    set time zone interval '+02:00' hour to minute

    [##################################################] 100%  00:00:00
    CACHE ID:1
    select * from tables a, tables b, tables c, tables d, tables e
    [##################################................>  68%  00:00:20      language.dataflow()

This indicates that tachograph has captured two queries. The first query “set time zone...” is executed automatically once at the start of each mclient to set the time zone for this client. For each captured query, tachograph shows on the command line:

  1. progress bar,
  2. percentage of completion,
  3. elapsed time, and
  4. MAL instruction currently being executed (if any).

The progress bar and percentage of completion are computed using the ETC (Expected Time to Completion) of the MAL instructions. To terminate tachograph at any time, press Ctrl-c. For each query, tachograph saves the execution information in four files (by default in the directory “cache”):

    $ ls cache
    tachograph_0.json      tachograph_0.trace    tachograph_1_stmt.csv
    tachograph_0_mal.csv   tachograph_1.json     tachograph_1.trace
    tachograph_0_stmt.csv  tachograph_1_mal.csv  tachograph.trace

The events streams are saved in the “.trace” files. The “.csv” files contain more readable information of the MAL statements extracted from the “.trace” files stored in CSV format. The “.csv” files can be easily loaded into a database for in-database analysis of the query execution, which are currently not used yet.

The most interesting output files here are the “.json” files, which contain the same information as the “.trace” files, but converted into JSON format. In addition, each “stmt” is accompanied by a “beautystmt”, in which the variables in the MAL statement are mapped back to tables and columns. This makes the event streams much more human readable. The table below shows an excerpt of the “.json” file generated for the query “select count(*) from tables;”:

    { "tachograph":0.1,
      "system":{ MonetDBversion:"11.20.0", release:"unreleased", host:"x86_64-unknown-linux-gnu", threads:"48", memory:"93.793 GB", oid:"64", packages:["huge"]}",   ],

      "qid":"user.s1_2{autoCommit=true}",
      "tag":"337",
      "query":"select count(*) from tables",
      "started": 85885322413,
      "duration":0,

      "instructions":34
    },
    {
    "qid":"user.s1_2{autoCommit=true}",
    "tag":337,
    "pc":1,
    "time": 1191,
    "status": "start",
    "estimate": 0,
    "stmt": "X_49=0@0:void := querylog.define(\"select count(*) from tables;\":str,\"default_pipe\":str,34:int)",
    "beautystmt": "querylog.define(\"select count(*) from tables;\",\"default_pipe\",34)",
    "prereq":[]
    },
    ...
    {
    "qid":"user.s1_2{autoCommit=true}",
    "tag":337,
    "pc":27,
    "time": 5139,
    "status": "start",
    "estimate": 0,
    "stmt": "X_38=0:wrd := aggr.count(X_37=<tmp_710>[0]:bat[:oid,:int])",
    "beautystmt": "count(tmp._tables.id[0])",
    "prereq":[, 26]
    },
    ...

Each “.json” file starts with a special block containing meta-information, e.g., hardware resources and software versions. Then it contains one block for each line of event in the “.trace” file.