Installation

Installation zhang Mon, 02/17/2020 - 17:42

Although there are pre-built images for MonetDB, it is also possible to download and build it from source code.

In this section we are going to give a guide for binary installers, source code build, Windows and Mac OS setup.

Install from Source

Install from Source giulia Tue, 02/25/2020 - 16:36

Installing MonetDB from sources is often considered a tedious job.  Compilation of MonetDB on the Linux platforms is based on the bootstrap-configure-make pattern. It can be compiled in any location on your file system and it does not require special (root) permissions.

While the core task of compiling MonetDB itself is extremely simple (on UNIX based platforms), inexperienced users often run into problems caused by many effects of the build environment. The catch is the differences between the target platforms, because not all libraries needed may have been installed by your system administrator.  Typical missing components on a binary distribution such as Fedora or Ubuntu are the xxx-dev packages necessary for MonetDB to compile against the package itself.

MonetDB is a rather large C-based program, and it sometimes closely interacts with the host system it runs on to achieve the best end results for the user.  This makes the code not always as generic as we ultimately would like it to be.  Because we understand that these parts of the code are likely suspects for malfunctioning, we try to test our sources on a nightly basis on as many different platforms that we can access.  This mainly focuses on different architectures, kernels and libcs, but we also test a couple of Linux distributions, for example (See Nightly testing).

To allow doing the right thing on your system, we use autoconf, automake and libtool to build the software.  We aim at being very standard in this respect.   In a nutshell, building the software works with the usual ritual:

% configure

% make

% make install

This process often fails for users in the configure step, but the make step may also give problems.  We will discuss some of the frequent issues people find here in detail.

For Mac OS X instruction please check this page.

Configure

Calling the configure script will check the system for required files and packages, and determine the capabilities of the system, such that the right approach can be taken in certain parts of the code.

Before calling configure, it is wise to examine all available options this script has by using

% configure --help

In particular the --prefix argument will be necessary to be set in most cases.  If you don't know what value to use for the prefix argument, a location in your homedir will typically do, e.g. --prefix=$HOME/MonetDB.

If configure finishes, it gives a conclusion at the end of its output.  Be sure to read this carefully, if a component is disabled this means it will not be built and installed.  At the time of this writing, you will need at least the monetdb5 and sql components.  Unless if you know what you're doing, you should not continue as long as configure reports these components to be disabled.

When a component is disabled, it is usually due to a missing dependency.  A dependency is often a library that the component in question relies on for its functionality.  The following two dependencies are known trouble makers on most systems that aren't fairly recent: openssl, libxml2 and pcre or libpcre.  The former is usually available, but a not recent enough version.  The latter two often lack the development headers on systems.  You have to install missing dependencies either through the native package management system of your host system, or by compiling and installing them manually.  Since this in general differs from package to package, we cannot provide generalized help on this topic.  Make sure you install the xxx-dev versions on binary distributions of the necessary packages as well!

Make

After configure has successfully found all it is looking for, make usually churns away for some time, producing quite some output.  It can happen that make aborts:

make[3]: *** [something] Error 1

The real error is usually right above this message.  If you're looking for help, or want to report a bug, always copy at least about 20 lines above this message, as it gives the necessary context to investigate on the problem.

If you passed --enable-strict to configure, then try to configure with --disable-strict and try make again.  This can allow certain type of problems to be ignored, since they are typically non-critical.

Make can fail on a compilation or linking step.  The former usually manifests itself by a message from the compiler like

somefile.c:12: 'rl' may be used uninitialized in this function

the latter typically manifest itself by "undefined references".  For compilation issues Googling is usually the best action to get a clue on what's going on.  The undefined references during linking are often missing libraries, due to missing -L/path/to/dir flags in LDFLAGS.  If you have libraries in use that are in a non-standard location (for example because you installed them yourself), make sure you have the proper includes (-I/path/to/include/dir) in your CPPFLAGS environment variable, and the library search paths (-L/path/to/lib/dir and -rpath=/path/to/lib/dir on ELF-based systems e.g. Linux) in your LDFLAGS such that they can be properly found by configure, libtool, the compiler and the linker.

OS X Install

OS X Install dnedev Tue, 02/25/2020 - 16:39

Install Xcode

To be able to build (almost) any software package on OS X, one needs to have Apple Xcode installed, or at least the Xcode command-line tools. Xcode comes with a C/C++ compiler toolset, which (as of the more recent versions) is based on Clang/LLVM and not GCC.

If you want the most recent version of Xcode you can only obtain it from the Apple Developer Website (if you are a registered Apple developer) or directly from the Apple Mac App Store here.

Run the following command in Terminal in order to install the Xcode command-line tools:

xcode-select --install

Beware, OS X usually ships with a BSD toolset, unlike most Linux distribution that make use of the GNU toolset. This affects not only the compiler, but also tools (such as make, pkgconfig, aws, sed) used in the build scripts of MonetDB.

Installing required packages

There are two community supported package managers that one can use to install the dependencies for building MonetDB on OS X: Homebrew or MacPorts. At this point you would need to pick which package manager to use and install only one of them.

Using Homebrew

To install Homebrew please follow the up-to-date instructions for your OS X version. You can find these on their website found here.

Minimal modules build-set

After Homebrew is installed, you should install the required packages for building MonetDB absolute minimal set of modules. Run the command below to install the required packages (and their dependencies in the brackets): pkg-config; pcre

brew install pkg-config pcre openssl

Default modules build-set

Keep reading/installing if you want to build the other MonetDB modules included in the default build-set.

* JDBC & control
Install JDK 7, which you can get from the Oracle website. Note: You need the JDK, not the JRE.
After JDK in installed, set the JAVA_HOME variable and add the JDK bin directory to your shell's PATH. To do this, add the lines below to your .profile, .bashrc or .bash_profile (or the corresponding config file for your shell).

export JAVA_HOME=$(/usr/libexec/java_home) export PATH=${JAVA_HOME}/bin:$PATH

Install Apache Ant

brew install ant

* ODBC
Install unixODBC

brew install unixodbc

* GEOM
Install geos

brew install geos

Other

* sphinxclient
Install libsphinxclient

brew install libsphinxclient

* GSL
Install gsl

brew install gsl

* FITS
Install cfitsio

brew install cfitsio

Before building

You most likely need to install automake, autoconf, libtool, gettext and readline  from Homebrew as well. These will also be required to run the bootstrap script (see below).

brew install autoconf automake libtool gettext readline

Putting it all together

brew install autoconf automake libtool gettext readline pkg-config pcre openssl unixodbc geos gsl cfitsio

Using MacPorts

As an alternative to Homebrew, you can also use MacPorts to obtain the required packages. To install MacPorts please follow the up-to-date instructions for your OS X version. You can find these on their website here. Do not forget to restart your command prompt shell (since MacPorts will add new extensions to your PATH) and run:

sudo port -v selfupdate

Minimal modules build-set

After MacPorts is installed, you should install the required packages for building MonetDB absolute minimal set of modules. Run the command below to install the required packages (and their dependencies in the brackets): pkgconfig (libiconv); openssl (zlib); pcre (bzip2, ncurses, libedit); libxml2 (expat, gettext, xz).

sudo port install pkgconfig openssl pcre libxml2

Default modules build-set

Keep reading/installing if you want to build the other MonetDB modules included in the default build-set.

* JDBC & control
Install JDK 7, which you can get from the Oracle website. Note: You need the JDK, not the JRE.
After JDK in installed, set the JAVA_HOME variable and add the JDK bin directory to your shell's PATH. To do this, add the lines below to your .profile, .bashrc or .bash_profile (or the corresponding config file for your shell).

export JAVA_HOME=$(/usr/libexec/java_home) export PATH=${JAVA_HOME}/bin:$PATH

Install Apache Ant, which can be downloaded its website [https://ant.apache.org/bindownload.cgi]. Unpack the package in a directory in your home space, e.g. in /Users/<username>/tools/apache-ant. After that set the ANT_HOME variables and update the PATH like this:

export ANT_HOME=/Users/<username>/tools/apache-ant export PATH=${ANT_HOME}/bin:$PATH

* ODBC
Install unixODBC (libtool, readline)

sudo port install unixodbc

Add the line below when configuring the build to point the tool to the library (see Configuring below).

--with-unixodbc =/opt/local --with-readline=/opt/local

* GEOM
Install geos

sudo port install geos

Other

* curl
Install curl (curl-ca-bundle, libind)

sudo port install curl

* libmicrohttpd
Install libmicrohttpd (gmp, libtasn1, nettle, libffi, glib2, popt, desktop-file-utils, libxslt, p11-kit, gnutls, libgpg-error, libgcrypt)

sudo port install libmicrohttpd

* liburiparser
Install uriparser

sudo port install uriparser

* sphinxclient

Install libsphinxclient

sudo port install libsphinxclient

Add the line below when configuring the build to point the tool to the library (see Configuring below).

--with-sphinxclient=/opt/local

* GSL
Install gsl

sudo port install gsl

* FITS
Install cfitsio

sudo port install cfitsio

Before building

You will probably need to install automake (gdbm, perl5.16, perl5) and autoconf from MacPorts as well. These will also be required to run the bootstrap script (see below).

sudo port install automake autoconf

Putting it all together

sudo port install automake autoconf pkgconfig openssl pcre libxml2 unixodbc bison geos gsl cfitsio curl libmicrohttpd uriparser libsphinxclient

Obtaining sources and bootstrapping

Sources of the latest released and testing versions on MonetDB can be obtained from the repository: released or testing. Download the selected file and unpack it in a directory.

Cloning MonetDB Mercurial repository

Optionally, bleeding egde sources can be directly obtained from the MonetDB Mercurial repository. To do that make sure you have Mercurial installed on your system first - latest version available here. The clone the MonetDB repository. Note this can take some time and a lot of data will be transferred.

hg clone http://dev.monetdb.org/hg/MonetDB/

Bootstrapping

Sources check-out from Mercurial need to be bootstrapped first, before configure can be run. To do that, go in the directory you in check you cloned the repository and run:

./bootstrap

* If you are using Homebrew, you will also need to set M4DIRS to the directory where the Homebrew gettext M4 macros are in. The same holds for the OpenSSL libs and includes:

export M4DIRS=/usr/local/opt/gettext/share/aclocal
export LDFLAGS=-L/usr/local/opt/openssl/lib
export CPPFLAGS=-I/usr/local/opt/openssl/include

Configuring build

With Homebrew

Before starting the configure tool, go in the directory where the MonetDB sources are, and create a subdirectory BUILD. Go in the BUILD directory and run the following command, where prefix is the location where you would like MonetDB installed.

../configure --prefix=<full-path>

With MacPorts

Before starting the configure tool, go in the directory where the MonetDB sources are, and create a subdirectory BUILD. Go in the BUILD directory and run the following command, where prefix is the location where you would like MonetDB installed.

../configure --prefix=<full-path> --with-libiconv-prefix=/opt/local

* If you want to build the ODBC driver or enable sphinxclient support, do not forget to add the appropriate arguments, e.g.:

../configure --prefix=<full-path> --with-libiconv-prefix=/opt/local --with-sphinxclient=/opt/local --with-unixodbc =/opt/local --with-readline=/opt/local

Note: If you don't have root/admin permission for your system you will not be able to deploy the RubyGem. If you don't need the RubyGem simply disable with with the option --without-rubygem

Building

To build the configured modules, simply run from the <tt>BUILD</tt> directory, using the -j<number> argument for a parallel build. E.g.

make -j4

Once the build process finishes, install MonetDB in the prefix location set during the configure process.

make install

Note: If you have built the RubyGem, on OS X you will need to use sudo make install to deploy it.

Windows Install

Windows Install giulia Tue, 02/25/2020 - 16:37

On Windows the first step is to initialize a MonetDB server by simply opening: 'Start -> Programs -> MonetDB -> Start server'. 

Subsequently you can start the textual interface (mclient) by opening: 'Start -> Programs -> MonetDB -> SQL client'.  The commands entered there are identical to those found on other platforms. To stop the server, you can simply close the MonetDB SQL Server window. 

Note the server by default only accepts connections originating from the local host. If you need other machines to access your database, change the configuration file by setting mapi_open=yes.

Exploring the wealth of functionality offered by MonetDB/SQL is best started using a toy database. For this we use the VOC database which provides a peephole view into the administrative system of an early multi-national company, the VOC (or Dutch East India Company).

Download the VOC dataset voc_dump.zip (542K) gz (519K) bz2 (371K) which is a compressed file with SQL statements (SQL database dump). After the file has been extracted, load its contents into MonetDB using the mclient.exe file. From within mclient run:

sql> \< voc_dump.sql

Since the SQL interface is the same on all platforms, from here on you can follow the later part of the regular SQL tutorial.

Changing database location

If you need (or want) to change the location of the database then editing of the monetdb script is required. Change the mclient.bat and M5server.bat files in 'C:\Program Files\MonetDB\MonetDB5'. In the M5server.bat file you would need to change the setting of the MONETDBFARM variable, and in mclient.bat you would need to add a -d option to the call of mclient.exe.  Note: you upgrade such changed files will be overwritten.

About the VOC dataset

The data for the tutorial is published in the book J.R. Bruijn, F.S. Gaastra and I. Schaar Dutch-Asiatic Shipping in the 17th and 18th Centuries, which gives an account of the trips made to the East and ships returned safely (or wrecked on the way) by the VOC (or Dutch East India Company). A total of 8000 records are provided. They include information about ship name and type, captain, the arrival/departure of harbors along the route, personnel accounts, and anecdotal information. You can find more about the VOC on Wikipedia.