[MonetDB-users] I'm looking for a fast way to write data to the DB from a scripting language.

Michalis Polakis mpolakis at gmail.com
Sun Sep 7 10:37:05 CEST 2008


On Sun, Sep 7, 2008 at 9:02 AM, JJ Jakosky <jay.jakosky at gmail.com> wrote:
> Thank you for replying quickly. Ruby would be a good choice.
>
> I'm not clear about MAPI. I only know of Mail API. Does this mean Monet API?
Yes. For details on MAPI please see:
http://monetdb.cwi.nl/projects/monetdb/SQL/Documentation/The-Mapi-Library.html
>
> I see a reference to the MAPI include files. Can you please outline the
> first steps I can take to get this going in Ruby? Can I compile in Cygwin or
> do I need to do it on Linux?
>
Copying from MonetDB's adapter for Rails README :

1. Wrapping the Monet API to Ruby
---------------------------------
MonetDB's API, called MAPI provides the functions an application needs
to communicate with a MonetDB server. We need to wrap this API in
Ruby, so that the MonetDB adapter can utilize it to communicate with
the database and we are going to use the Simplified Wrapper and
Interface Generator (SWIG) to do it.

The necessary files for this task reside in the mapi_wrapper directory.
MapiLib.ruby.i is the input file for SWIG
extconf.rb is a ruby makefile generator script.

By now, you should have already installed MonetDB and its client
package that includes the MAPI library.

NOTES for MapiLib.ruby.i :
%module mapi denotes the name our ruby module will have.
There are two include statements. One is in %{ .. %}  and the other as
%include .. that both include the Mapi header file. MAKE SURE IT'S THE
CORRECT PATH. The default path is the default MonetDB installation
path, though.
Some functions cannot be wrapped in Ruby by SWIG and are therefore
ignored. It's ok though.

After you've set the parameters right, you execute :
> swig MapiLib.ruby.i
> ruby extconf.rb
> make
> make install (you should typically have superuser rights for this one)

You have now installed the wrapped in Ruby MonetDB API. You can use it
in your Ruby applications to connect and interact with MonetDB
databases.

Short example :
	require 'mapi'
	include Mapi
	mid = Mapi::mapi_connect("localhost", 50000, "monetdb", "monetdb",
"sql", "db_name_here")
-----------
The above procedure has been tried and tested on Ubuntu Linux.
It should be possible to do it with cygwin, but I'm not aware of any
(dirty) details.
For an alternative approach in installing MAPI for Ruby check the
clients package makefile etc.

Regards,
Michalis

> Thanks,
> Jay
>
>
> On Sat, Sep 6, 2008 at 11:20 PM, Michalis Polakis <mpolakis at gmail.com>
> wrote:
>>
>> Good morning Jay,
>>
>> you can use the Ruby MAPI Wrapper => MAPI calls through Ruby.
>> It's elegant and simple and was used to implement the Ruby on Rails
>> adapter for MonetDB.
>> Plus, from what I see in the SWIG input file, the insert and copy_from
>> functions should be
>> implemented in the Ruby MAPI Wrapper.
>> If you want to go the Ruby way, you may find the files in the MonetDB
>> clients package or
>> at http://rubyforge.org/projects/monetdb-ror/ as part of the Ruby on
>> Rails MonetDB adapter.
>>
>> Regards,
>> Michalis
>>
>>
>> -----------------------------------------------------------------------------------------------
>> Hi,
>>
>> I have 50 million rows and 15 columns that I'd like to put into one
>> table. My first attempt to transfer them was through PHP using the
>> monetdb_query method. This is causing a lot of disk thrashing and my
>> throughput is about 100 rows/sec.
>>
>> I get an "undefined function" error for both monetdb_insert and
>> monetdb_copy_from.
>>
>> I would like to find a fast method, but from inside a common scripting
>> language.
>>
>> Thanks,
>> Jay
>
>
>
> --
> JJ Jakosky
> 720.244.0137
>




More information about the users-list mailing list