Skip to main content

monetdbd man-page

NAME

monetdbd
− the MonetDB Database Server daemon

SYNOPSIS

monetdbd
command [ command_args ] dbfarm

DESCRIPTION

Monetdbd
is the MonetDB Database Server daemon. The program is mainly
meant to be used as daemon, but it also allows to setup and
change the configuration of a dbfarm. The use of
monetdbd is either as user-oriented way to configure,
start and stop a database farm, or to be started from a
startup script, such as from /etc/init.d/ on Linux
systems or smf(5) on Solaris systems, as part of a
system startup.

Monetdbd
is the system formerly known as merovingian. It was
renamed to monetdbd since the name merovingian
proved to be confusing to most regular end-users.
Internally, monetdbd uses the name merovingian
at many places for historical reasons.

A
monetdbd instance manages one local cluster based,
which is a directory in the system, referred to as the
dbfarm. Nowadays, the dbfarm location always has to
be given as argument to monetdbd.

Within its
local cluster monetdbd takes care of starting up
databases when necessary, and stopping them either upon
request via monetdb(1) or when being shut down.
Client database connections are made against monetdbd
initially which redirects or proxies the client to the
appropriate database process, started on the fly when
necessary.

When started,
monetdbd runs by default in the background, sending
log messages to merovingian.log, until being sent a
stop, terminate or interrupt signal, possibly using the stop
command of monetdbd.

monetdbd
uses a neighbor discovery scheme to detect other
monetdbd processes running in the local network.
Databases from those remote instances are made available to
a locally connecting client. Remote databases never override
local databases, and their availability is controlled by the
remote monetdbd process. See also the sharing
capabilities of monetdb(1) and the REMOTE
DATABASES
section below.

COMMANDS

The commands
for monetdbd are create, start,
stop, get, set, version, and
help. The commands facilitate initializing a dbfarm,
starting and stopping the MonetDB Database Server, and
retrieving or setting options.

create dbfarm

Initializes a new database
farm, such that a MonetDB Database Server can be started on
that location. All necessary directories are attempted to be
created, and an initial properties file is created in the
directory itself. dbfarm must be a location
addressable in the local filesystem hierarchy.

start [−n]
dbfarm

Starts monetdbd, the
MonetDB Database Server, on the given dbfarm. When the
−n flag is given, monetdbd will not fork
into the background, but instead remain attached to the
calling environment, until given a stop signal.

stop dbfarm

Sends a stop signal to the
monetdbd process responsible for the given
dbfarm.

get <all |
property[,property[,..]]> dbfarm

Prints the requested
properties, or all known properties, for the given dbfarm.
For each property, its value is printed. Some properties are
virtual, and given for information purposes only, they
cannot be modified using the set command.

set
property=value dbfarm

Sets property to value for the
given database. For a list of properties, run monetdbd
get all
. Some properties require a restart of the
MonetDB Database Server in order to take effect. The
set command, will however always write the property,
and tell the running monetdbd to reload the
properties file (if running). For an explanation of the
properties, see the CONFIGURATION section below.

CONFIGURATION

Monetdbd
reads its properties from the .merovingian_properties
file inside the dbfarm. This file is created by the
create command. This file is not meant to be edited
manually, instead it should be updated using the set
command. The following properties can be set:

logfile

This property points to the
file where all log messages are written to. It is relative
to the dbfarm directory, but can be absolute to point to
e.g. another medium. Changing this property takes effect
immediately at runtime.

pidfile

Monetdbd stores the
process ID of the background server in the file pointed to
by this property. The same rules apply as for the
logfile property.

sockdir

For faster access,
monetdbd uses UNIX domain sockets for its control
mechanism and regular database connections. The sockets are
placed as files in the filesystem hierarchy. The
sockdir property controls in which directory they are
placed. In general this setting should not be changed.

cellspacing="0" cellpadding="0">

port

This property specifies which TCP port monetdbd
should listen to for connection requests. Defaults to
50000.

control

For remote management of
monetdbd, the control property specifies
whether or not to enable remote management. Note that for
remote management, a passphrase is required, see below. It
defaults to false for security reasons. Changing this
property takes effect immediately at runtime.

passphrase

To control monetdbd from
a remote machine, a passphrase is necessary, to be given to
monetdb(1). The passphrase can be either given as
hashed value prefixed by the hash type in curly braces (e.g.
{SHA512}xxx...) or as plain text value which will be hashed
automatically. Note that the only hash accepted is the one
specified at configure time, which is SHA512. Changing this
property takes effect immediately at runtime.

discovery

Specifies whether neighbor
discovery is to be enabled using UDP broadcasts or not. The
broadcasts are done on the same portnumber as the port
setting.

discoveryttl

Monetdbd publishes
locally available databases to others periodically. The
interval used here, defined in seconds, depends on the
time-to-live of the databases before they need to get
refreshed. The default is 600 seconds (10 minutes), which
should keep traffic in your network fairly low. Additions
and removals are processed immediately regardless of this
timeout. If you are in a network environment where physical
network links disappear often, you may want to decrease this
value to more quickly remove no longer reachable
databases.

exittimeout

mservers that were started by
the MonetDB Database Server are shut down when
monetdbd is shut down. Setting the exittimeout
property to a positive non-zero value will shut down each
running mserver with the given time-out in seconds. If the
time-out expires, the mserver process is killed using the
SIGKILL signal. A time-out value of 0 means no mservers will
be shut down, and hence they will continue to run after
monetdbd has shut down. Note that this particular
configuration is extremely inconvenient. The default
time-out is 60 seconds. If your databases are rather large
and find your databases consistently being killed by
monetdbd upon shutdown, you may want to increase this
time-out. Changing this property takes effect immediately at
runtime.

forward

monetdbd has two ways in
which it can "attach" a connecting client to the
target database. The first method, redirect, uses a
redirect sent to the client with the responsible mserver
process. The second method, proxy, proxies the client
to the mserver over monetdbd. While redirect
is more efficient, it requires the connecting client to be
able to connect to the mserver. In many settings this may be
undesirable or even impossible, since a wide range of open
ports and routing are necessary for this. In such case the
proxy technique of monetdbd is a good
solution, which also allows a monetdbd instance on
the border of a network to serve requests to nodes in the
local (unreachable) network. Note that for local databases,
the proxy method uses a UNIX domain socket feature to
pass file-descriptors to the local mserver. This effectively
is as efficient as the redirect approach, but still
hides away the mservers properly behind monetdbd.
Hence, in practice it is only relevant for connections to
remote databases to use redirects instead of proxies.
Changing this property takes effect immediately at
runtime.

REMOTE DATABASES

The neighbor
discovery capabilities of monetdbd allow a user to
contact a remote database transparently, as if it were a
local database. By default, all local databases are
announced in the network, such that neighbors can pick them
up to make them available for their local users. This
feature can be disabled globally, or on database level. For
the latter, the monetdb(1) utility can be used to
change the share property of a database.

While neighbor
discovery in itself is sufficient to locate a database in a
cluster, it is limited in expressiveness. For instance,
database names are assumed to be unique throughout the
entire system. This means local databases overshadow remote
ones, and duplicate remote entries cannot be distinguished.
To compensate for this, monetdbd allows to adds a
tag to each database that is being shared. This tag
is sent in addition to the database name, and only
understood by other monetdbds.

Tags are
arbitrary ASCII-strings matching the pattern
[A−Za−z0−9./]+. There are no assumed
semantics in the tag, which allows for multiple approaches
when using the tag. The tag is always used in combination
with the database name. For this, the ‘/’
character is used as separator, which hence suggests the
user to use that character as separator for multilevel tags.
Monetdbd allows common path globbing using
‘*’ on tags, which allows for many use-cases.
Consider for instance the following three databases with
their tag:

dbX/master/tableQ


dbY/slave/tableQ

dbZ/slave/tableQ

A default match
has implicit ‘/*’ added to the search, making
more generic search strings match more specific ones. Hence,
a connect with database dbX is the same as
dbX/* and hence matches dbX/master/tableQ.
Similar, a database connect for */master matches the
same database as before. Note that the implicit
‘/*’ is not added if that would cause no
matches, such as for */master/tableQ which would
return all masters for tableQ, which in our
hypothetical example is only dbX. In contrast, a
database connect for */slave/tableQ matches with
either dbY or dbZ. Monetdbd returns the
two options to the client in a round-robin fashion, such
that subsequent connects for the same pattern result in a
load-balanced connect to either of both databases.

With tags in
use, one can possibly make distinction between databases, if
setup like that. The previous example could hence also be
setup like this:

tableQ/master


tableQ/slave

tableQ/slave

Connecting to
tableQ/slave would now return either of both
databases even though they are not unique (apart from the
host they are located on, which is not shown in the
example). While being confusing for humans, for
monetdbd it is the same situation as in the previous
example. However, because globbing allows to make things
easier to understand, tags for both slaves could be changed
to slaveX or slave/X and use the necessary
pattern to match them. It is up to the user to decide how to
use the tags.

MULTIPLEX-FUNNELS

Monetdbd
implements multiplex-funnel capabilities. As the name
suggests two techniques are combined, the multiplexer and
the funnel.

The
funnel capability limits the access to the database
to one client at a time. That is, if multiple clients
connect to the funnel, their queries will be serialized such
that they are executed one after the other. An effect of
this approach is that clients no longer have an exclusive
channel to the database, which means that individual queries
from one client may have been interleaved with queries from
others. This most notably makes SQL transaction blocks
unreliable with a funnel. The funnel, hence, is meant to
scale down a large amount of clients that perform
short-running (read-only) queries, as typically seen in
web-based query loads.

When a funnel
is defined to use multiple databases, the funnel adds a
multiplexer to its query channel. A multiplex-funnel
sends each query to all of the defined databases. This
behavior can be quite confusing at first, but proves to be
useful in typical sharding configurations, where in
particular simple selection queries have to be performed on
each of the shards. The multiplexer combines the answers
from all defined databases in one single answer that it
sends back to the client. However, this combining is without
any smart logic, that is, the multiplexer does not evaluate
the query it is running, but just combines all answers it
receives from the databases. This results in e.g. as many
return tuples for a SELECT COUNT(*) query, as there
are databases defined.

Due to the two
above mentioned characteristics, a multiplex-funnel has some
limitations. As mentioned before, transactions over multiple
queries are likely not to result in the desired behavior.
This is due to each query to the funnel is required to be
self-contained. Further, since for each query, the results
from multiple servers have to be combined into one, that
query must only return a single response, i.e.
multi-statement queries are most likely causing the funnel
to respond with an error, or return garbled results. Last,
the size of each query is limited to currently about 80K.
While this size should be sufficient for most queries, it is
likely not enough for e.g. COPY INTO statements. Apart from
the data transfer implications, such statements should not
be used with the funnel, as the results will be undefined
due to the limited query buffer. Applications using the
funnel should aim for short and single-statement queries
that require no transactions.

See the
create command in the monetdb(1) man-page for
details on how to setup a multiplex-funnel.

SIGNALS

Monetdbd
acts upon a number of signals as is common for a daemon.


SIGINT, SIGTERM, SIGQUIT

Any of these signals make
monetdbd enter the shutdown sequence. This sequence
involves cleanly shutting down listener sockets, shutting
down all started databases and finally terminating
itself.

cellspacing="0" cellpadding="0">

SIGHUP

When this signal is received by monetdbd it will
reopen the logfile as pointed to by the logfile
setting. Before it reopens the logfile, it will re-read the
properties file from the dbfarm, which might result in
opening a different file to continue logging.

RETURN VALUE

Monetdbd
returns exit code 0 if it was able to successfully
perform the requested action, e.g. start, stop, etc. When an
error occurs during the action, that prevents
monetdbd from successfully performing the action, the
exit code 1 is returned.

SEE ALSO

monetdb(1),
mserver5(1)