August 2013 - users-list

Data Vault:SyntaxException
by integrity 06 Aug '13

06 Aug '13

hi all, What's the correct way to use FIT DATA VAULT in mal? I got the following errors and still havent try out the right syntax ...... [svomhpc@localhost /home/data2]$>/home/data1/opt/MonetDB.fits/bin/mclient -lmal -dmydb Welcome to mclient, the MonetDB interactive terminal (unreleased) Type \q to quit, \? for a list of available commands mal>?fits.attach() pattern fits.attach(fname:str):void address FITSattach; comment Open a FITS file and return catalog of the table HDUs mal>call fits.attach('/home/data3/gwacdata/20130111_177d550752_60d851283-530.fit'); MAPI = (monetdb) /tmp/.s.monetdb.50000 QUERY = call fits.attach('/home/data3/gwacdata/20130111_177d550752_60d851283-530.fit'); ERROR = !SyntaxException:parseError:call fits.attach('/home/data3/gwacdata/20130111_177d550752_60d851283-530.fit'); !SyntaxException:parseError: ^';' expected mal> mal> mal>call fitsattach('/home/data3/gwacdata/20130111_177d550752_60d851283-530.fit'); MAPI = (monetdb) /tmp/.s.monetdb.50000 QUERY = call fitsattach('/home/data3/gwacdata/20130111_177d550752_60d851283-530.fit'); ERROR = !SyntaxException:parseError:call fitsattach('/home/data3/gwacdata/20130111_177d550752_60d851283-530.fit'); !SyntaxException:parseError: ^';' expected mal> mal>call FITSattach('/home/data3/gwacdata/20130111_177d550752_60d851283-530.fit'); MAPI = (monetdb) /tmp/.s.monetdb.50000 QUERY = call FITSattach('/home/data3/gwacdata/20130111_177d550752_60d851283-530.fit'); ERROR = !SyntaxException:parseError:call FITSattach('/home/data3/gwacdata/20130111_177d550752_60d851283-530.fit'); !SyntaxException:parseError: ^';' expected Thanks! Meng

4 15

Re: How to use Data Vault with FITS files?
by integrity 05 Aug '13

05 Aug '13

Hi Ying, The thesis doesn't have instructions on how to use it....or it doesn't describe it very clearly maybe. Bests, Meng ------------------ Original ------------------ From: "Ying Zhang"<Y.Zhang(a)cwi.nl>; Date: Mon, Aug 5, 2013 04:18 PM To: "Communication channel for MonetDB users"<users-list(a)monetdb.org>; Subject: Re: How to use Data Vault with FITS files? Hai Meng, I already sent you the master thesis on FITS data vault. Didn't you receive it? Regards, Jennie On Mon, Aug 05, 2013 at 03:52:50PM +0800, integrity wrote: > Hi all, > > Could you give a tutorial of Data vault with FITS files, i searched the > moentdb.org and google, but didn't find the step-by-step tutorials. > > Thanks! > Meng > > _______________________________________________ > users-list mailing list > users-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/users-list _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list .

2 1

Re: How to use Data Vault with FITS files?
by integrity 05 Aug '13

05 Aug '13

Thanks very much Stefan! Does ' configure MonetDB with "--enable-fits" ' mean I have to re-install MonetDB? If I re-install it, can i still use my current db-farm? I have loaded a large catalog into it for a long time... Regards, Meng ------------------ Original ------------------ From: "Stefan Manegold"<Stefan.Manegold(a)cwi.nl>; Date: Mon, Aug 5, 2013 04:13 PM To: "Communication channel for MonetDB users"<users-list(a)monetdb.org>; Subject: Re: How to use Data Vault with FITS files? Hi Meng, to get the MonetDB FITS support, you need to have the cfitsio-dev[el] package installed and you need to configure MonetDB with "--enable-fits". However, while the respective code is more than 2 years old, I just realize that there is no (user-)documentation for it, and so far, support for fits files is only available in MAL, but not (yet?) in SQL. Since the original author of that code is no longer with out group, I currently cannot estimate, when we will have the resources to make this experimental code user-/production-ready. If you are familiar with MAL, please have a look at $ mclient -lmal Welcome to mclient, the MonetDB interactive terminal (unreleased) Type \q to quit, \? for a list of available commands mal>?fits.*() pattern fits.attach(fname:str):void address FITSattach; comment Open a FITS file and return catalog of the table HDUs pattern fits.export(tablename:str):void address FITSexportTable; comment Export a table to a FITS file command fits.fitstest(filename:str):int address FITStest; comment Returns the type of first extension in the FITS file filename pattern fits.load(tablename:str):void address FITSloadTable; comment Load a FITS table from an attached file pattern fits.listdirpattern(dirname:str,pattern:str):void address FITSdirpat; comment Attach all FITS file in the directory, giving a pattern pattern fits.listdir(dirname:str):void address FITSdir; comment Attach all FITS files in the directory mal> And if you're familiar with the MonetDB code base, you find the respective code in sql/backends/monet5/vaults/*fits* Best, Stefan ----- Original Message ----- > Hi all, > > > Could you give a tutorial of Data vault with FITS files, i searched the > moentdb.org and google, but didn't find the step-by-step tutorials. > > > Thanks! > Meng > _______________________________________________ > users-list mailing list > users-list(a)monetdb.org > http://mail.monetdb.org/mailman/listinfo/users-list > -- | Stefan.Manegold(a)CWI.nl | DB Architectures (DA) | | www.CWI.nl/~manegold/ | Science Park 123 (L321) | | +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) | _______________________________________________ users-list mailing list users-list(a)monetdb.org http://mail.monetdb.org/mailman/listinfo/users-list .

2 1

How to use Data Vault with FITS files?
by integrity 05 Aug '13

05 Aug '13

Hi all, Could you give a tutorial of Data vault with FITS files, i searched the moentdb.org and google, but didn't find the step-by-step tutorials. Thanks! Meng

4 3

How to use Data Vault with FITS files?
by integrity 04 Aug '13

04 Aug '13

Hi all, Could you give a tutorial of Data vault with FITS files, i searched the moentdb.org and google, but didn't find the step-by-step tutorials. Thanks! Meng

1 0

Scale Factor 2000 TPCH Database?
by Stephen P. Morgan 02 Aug '13

02 Aug '13

Has anyone successfully created a scale factor 2000 TPCH database with MonetDB? We're having some problems trying to do so. We don't have enough disk space to hold both the data (created by dbgen) and the MonetDB database, so we have to create the database in two steps. (Create the first half of the data as dbgen table files, load it, delete the dbgen files, create the second half of the data as dbgen table files, load it, then delete those files.) The problem shows up when loading the second half of the data: the mserver crashes and won't restart. This is what the dbfarm/merovingian.log says: 2013-08-02 18:42:19 MSG merovingian[39857]: database 'q1_s2000_n2' (39920) has crashed (dumped core) 2013-08-02 18:42:19 ERR merovingian[39857]: client error: database 'q1_s2000_n2' has crashed after starting, manual intervention needed, check monetdbd's logfile for details 2013-08-02 18:42:19 MSG merovingian[39857]: database 'q1_s2000_n2' has crashed after start on 2013-08-02 18:42:19, attempting restart, up min/avg/max: 0s/0s/0s, crash average: 1.00 0.50 0.17 (5-0=5) Other databases on the same system are still running. We're running Feb13 SP1 with minor local modifications that are not coming into play at this point. We're running on a large Linux server. Is there anything we can look at to help diagnose the problem? Thanks, Steve Morgan

3 3

Copy From SQL server
by soumik ghosh 02 Aug '13

02 Aug '13

Hello All, Could someone point me to an example of copying a table from SQL Server to MonetDB over a JDBC or ODBC bridge? Is this possible at all? Thanks Soumik

5 7

Background processes in MonetDB
by DrRock 01 Aug '13

01 Aug '13

Dear fellows, we installed MonetDB a few weeks ago on a Linux Machine (8 cores, 32GB RAM) and our overall experience is very good. So far, we are testing Monet with different ammounts of data, roughly peaking at 250GB. After loading plenty of data into the database or after interrupting a query, Monet feels somehow slow for a certain period of time. Another observation we made is that diskspace remains occupied after the deletion of all tables in the database for a certain time. Further, indices can be created, although this seems not to be really essential as MonetDB provides some kind of own intelligence, optimizing queries/ tables by themself. All of mentioned observations point at some kind of background processes and optimizing routines. Is there a way to modify the behaviour of MonetDB (Such as providing diskspace instantly after deletion of tables)? Is it possible to switch these processes off? Best regards, DrRock

1 0