June 2008 - users-list

Re: [MonetDB-users] [Monetdb-developers] Fastest way to calculate median?
by Martin Kersten 08 Jun '08

08 Jun '08

Rt Ibmer wrote: > Hi - I have an int column with several million rows of data. What is the fastest way to write a query to get the median value of the column? Apparently there is no median() function in SQL. A web search revealed lots of various talk about using stored procedures and fancy functions. None of this looked like it would be very efficient. I searched these archives but didn't see a single mention of median. Thank you! > True, the complexity of the median() in essence requires a partial sort of the complete list. It is a member of a group of statistical functions. A feature request has been added to the list. > > > > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > Monetdb-developers mailing list > Monetdb-developers(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-developers >

1 0

[MonetDB-users] Moving records from SQL Server to MonetDB
by Rt Ibmer 07 Jun '08

07 Jun '08

Hi All - I'm new to monetdb and it looks fantastic! I have about 2 millions rows of data that I'd like to export from SQL Server and import them into Monetdb so I can do some preliminary benchmarking. What is the best way to do this? The schemas are the same for SQL Server and MonetDB. So basically I'm looking for the best command to run on SQL Server to export the data in a format best suited for importing into MonetDB, and what is the command to then load the data into MonetDB? Thanks!

1 0

[MonetDB-users] [Fwd: RE: PROBLEM WITH MONETDB 5.0]
by Martin Kersten 07 Jun '08

07 Jun '08

-------- Original Message -------- Subject: RE: PROBLEM WITH MONETDB 5.0 Date: Sat, 7 Jun 2008 23:16:13 +0800 From: Tran Quoc Trung <tqtrung(a)comp.nus.edu.sg> To: 'Martin Kersten' <Martin.Kersten(a)cwi.nl> References: <38563.137.132.80.35.1212469981.squirrel(a)mysoc.nus.edu.sg> <4844E431.2090403(a)cwi.nl> <3363.137.132.80.35.1212475247.squirrel(a)mysoc.nus.edu.sg> <4844F364.70601(a)cwi.nl> Hi Mr. Martin, I have installed MonetDB both by using monetdb-install.h and following the document for expert in the website on Ubuntu 2.6 64-bits. I can use the MonetDB server and client. However, when writing a program (in C) to connect to MonetDB, I encounter the error that libMapi.so require libc-2.7.so (The libc in my system is libc-2.5.so). Are there any solution to handle this situation (b/c I restrict not to upgrade the libc due to using the common server with other project). Could you please help me to take a look at this. Thank you very much Trung. -----Original Message----- From: Martin Kersten [mailto:Martin.Kersten@cwi.nl] Sent: Tuesday, 3 June, 2008 3:32 PM To: tqtrung(a)comp.nus.edu.sg; Communication channel for MonetDB users Subject: Re: PROBLEM WITH MONETDB 5.0 tqtrung(a)comp.nus.edu.sg wrote: > Hi, > Thank you very much. I also guess the reason due to the size of the > operands. Currently, I am running on Linux Fedora Core 8, 32 bit. The > cardinality of the opearand s in the range 100,000 to 1,500,000 records. > the sizes themselves are not a problem, but the expected size of the join result could be. you could take a sample to see how it explodes (algebra.sample()) > Are there any other solutions if my machine could not play with those > queries. > Please help me to check it again. > > Thanks, > Trung > > > >> tqtrung(a)comp.nus.edu.sg wrote: >> >>> Dear all, >>> I am a PhD student at School of Computing, NUS, Singapore. Currently, I >>> used MonetDB as the DBMS to do some experiments. I issue a query with >>> about 10 joins, but the system responds with the following errors: >>> >>> >> Please inform us about the platform and distribution you are useing. >> >>> !MALException:algebra.join:GDKerror >>> !ERROR: GDKload: cannot mmap(): name=45/4542, ext=head.priv >>> !OS: Cannot allocate memory >>> >>> >> this typically is an error when the join algorithm can not allocated >> enough memory >> to hold the intermediate result. For example, on a 32-bit system this >> might cause >> a problem quickly. >> If the join result should fit in memory, then there might be a problem >> in the >> estimation of the intermediate size needed. For this we need more >> information, >> such as the cardinality of the operands and properties obtained by >> bat.info() >> >>> !ERROR: GDKload failed: name=45/4542, ext=head.priv >>> !ERROR: GDKload: cannot mmap(): name=45/4542, ext=tail.priv >>> !OS: Cannot allocate memory >>> !ERROR: GDKload failed: name=45/4542, ext=tail.priv >>> !ERROR: GDKload: cannot mmap(): name=45/4542, ext=head.priv >>> !OS: Cannot allocate memory >>> !ERROR: GDKload failed: name=45/4542, ext=head.priv >>> !ERROR: GDKload: cannot mmap(): name=45/4542, ext=head.priv >>> !OS: Cannot allocate memory >>> !ERROR: GDKload failed: name=45/4542, ext=head.priv >>> !ERROR: GDKload: cannot mmap(): name=44/4401, ext=tail.priv >>> !OS: Cannot allocate memory >>> !ERROR: GDKload failed: name=44/4401, ext=tail.priv >>> >>> I also try to change the parameter in monetdb5.conf about the memory: >>> gdk_maxsize and gdk_vm_minsize but it still does NOT work. >>> >>> >> please don't play with these settings. They only control a thread for >> flushing dirty pages to disk. >> >>> Could you please help me to figure out which parameters I could change >>> to >>> make it work. >>> >>> Thank you very much >>> Trung >>> >>> >>> >> > >

1 0

[MonetDB-users] [Fwd: Re: php extension]
by Martin Kersten 06 Jun '08

06 Jun '08

-------- Original Message -------- Subject: Re: php extension Date: Fri, 6 Jun 2008 11:20:39 +1200 From: Kees Varekamp <kees.varekamp(a)gmail.com> To: Sjoerd Mullender <sjoerd(a)acm.org> CC: Niels Nes <Niels.Nes(a)cwi.nl>, monet(a)cwi.nl, Sjoerd Mullender <Sjoerd.Mullender(a)cwi.nl> References: <d8981ee0806042218l13609c8an47065c5b9a8354e9(a)mail.gmail.com> <20080605054117.GA350(a)cwi.nl> <4847D4D4.8020608(a)acm.org> <d8981ee0806050543w19dd0028gbb9327c5378549b1(a)mail.gmail.com> <4847EEFD.8080102(a)acm.org> Tnx. Na een herstart doet ie het! Groeten, Kees 2008/6/6 Sjoerd Mullender <sjoerd(a)acm.org <mailto:sjoerd@acm.org>>: Kees Varekamp wrote: > Bedankt! Hij start nu, ik ben weer een stap verder. > > Krijg nu een *Warning*: monetdb_connect() [function.monetdb-connect > <http://localhost/monet/function.monetdb-connect>]: Unable to connect to > MonetDB server: gethostbyname failed > > Het voorbeeldje op de website > (http://monetdb.cwi.nl/projects/monetdb/SQL/Documentation/The-PHP-Library.ht…) > connect als volgt: > $db = monetdb_connect('sql', 'localhost', 50000, 'monetdb','monetdb') > > De synopsis eronder vermeldt echter: > monetdb_connect([string host [, string port [, string username [, string > password [, string language]]]]] > > Beide returnen bovenstaande foutmelding (zowel met localhost als met > 127.0.0.1 <http://127.0.0.1> <http://127.0.0.1>). Welke is juist? En klopt het dat ik in > principe de php_pgsql.dll functies kan vertalen naar Monet functies? > > Het is niet kritiek trouwens, if all else fails zet ik een Linux machine op. Het ziet ernaar uit dat de synopsis niet correct is en de volgorde van de argumenten inderdaad is zoals in dat voorbeeld. Ik heb dit ook net geprobeerd en het werkt. De hostnaam wordt met de C functie gethostbyname vertaald. Zowel de string 'localhost' als '127.0.0.1 <http://127.0.0.1>' zou moeten werken, maar het ligt aan je Windows installatie. Je kunt experimenteren met php -a (in een cmd window) en dan: php > print gethostbyname('localhost'); Dat zou 127.0.0.1 <http://127.0.0.1> moeten printen. Je kunt ook de file c:\windows\system32\drivers\etc\hosts bekijken om te zien of er een entry in staat voor localhost. -- Sjoerd Mullender

1 0

Re: [MonetDB-users] PROBLEM WITH MONETDB 5.0
by Martin Kersten 03 Jun '08

03 Jun '08

tqtrung(a)comp.nus.edu.sg wrote: > Hi, > Thank you very much. I also guess the reason due to the size of the > operands. Currently, I am running on Linux Fedora Core 8, 32 bit. The > cardinality of the opearand s in the range 100,000 to 1,500,000 records. > the sizes themselves are not a problem, but the expected size of the join result could be. you could take a sample to see how it explodes (algebra.sample()) > Are there any other solutions if my machine could not play with those > queries. > Please help me to check it again. > > Thanks, > Trung > > > >> tqtrung(a)comp.nus.edu.sg wrote: >> >>> Dear all, >>> I am a PhD student at School of Computing, NUS, Singapore. Currently, I >>> used MonetDB as the DBMS to do some experiments. I issue a query with >>> about 10 joins, but the system responds with the following errors: >>> >>> >> Please inform us about the platform and distribution you are useing. >> >>> !MALException:algebra.join:GDKerror >>> !ERROR: GDKload: cannot mmap(): name=45/4542, ext=head.priv >>> !OS: Cannot allocate memory >>> >>> >> this typically is an error when the join algorithm can not allocated >> enough memory >> to hold the intermediate result. For example, on a 32-bit system this >> might cause >> a problem quickly. >> If the join result should fit in memory, then there might be a problem >> in the >> estimation of the intermediate size needed. For this we need more >> information, >> such as the cardinality of the operands and properties obtained by >> bat.info() >> >>> !ERROR: GDKload failed: name=45/4542, ext=head.priv >>> !ERROR: GDKload: cannot mmap(): name=45/4542, ext=tail.priv >>> !OS: Cannot allocate memory >>> !ERROR: GDKload failed: name=45/4542, ext=tail.priv >>> !ERROR: GDKload: cannot mmap(): name=45/4542, ext=head.priv >>> !OS: Cannot allocate memory >>> !ERROR: GDKload failed: name=45/4542, ext=head.priv >>> !ERROR: GDKload: cannot mmap(): name=45/4542, ext=head.priv >>> !OS: Cannot allocate memory >>> !ERROR: GDKload failed: name=45/4542, ext=head.priv >>> !ERROR: GDKload: cannot mmap(): name=44/4401, ext=tail.priv >>> !OS: Cannot allocate memory >>> !ERROR: GDKload failed: name=44/4401, ext=tail.priv >>> >>> I also try to change the parameter in monetdb5.conf about the memory: >>> gdk_maxsize and gdk_vm_minsize but it still does NOT work. >>> >>> >> please don't play with these settings. They only control a thread for >> flushing dirty pages to disk. >> >>> Could you please help me to figure out which parameters I could change >>> to >>> make it work. >>> >>> Thank you very much >>> Trung >>> >>> >>> >> > >

1 0

Re: [MonetDB-users] PROBLEM WITH MONETDB 5.0
by Martin Kersten 03 Jun '08

03 Jun '08

tqtrung(a)comp.nus.edu.sg wrote: > Dear all, > I am a PhD student at School of Computing, NUS, Singapore. Currently, I > used MonetDB as the DBMS to do some experiments. I issue a query with > about 10 joins, but the system responds with the following errors: > Please inform us about the platform and distribution you are useing. > !MALException:algebra.join:GDKerror > !ERROR: GDKload: cannot mmap(): name=45/4542, ext=head.priv > !OS: Cannot allocate memory > this typically is an error when the join algorithm can not allocated enough memory to hold the intermediate result. For example, on a 32-bit system this might cause a problem quickly. If the join result should fit in memory, then there might be a problem in the estimation of the intermediate size needed. For this we need more information, such as the cardinality of the operands and properties obtained by bat.info() > !ERROR: GDKload failed: name=45/4542, ext=head.priv > !ERROR: GDKload: cannot mmap(): name=45/4542, ext=tail.priv > !OS: Cannot allocate memory > !ERROR: GDKload failed: name=45/4542, ext=tail.priv > !ERROR: GDKload: cannot mmap(): name=45/4542, ext=head.priv > !OS: Cannot allocate memory > !ERROR: GDKload failed: name=45/4542, ext=head.priv > !ERROR: GDKload: cannot mmap(): name=45/4542, ext=head.priv > !OS: Cannot allocate memory > !ERROR: GDKload failed: name=45/4542, ext=head.priv > !ERROR: GDKload: cannot mmap(): name=44/4401, ext=tail.priv > !OS: Cannot allocate memory > !ERROR: GDKload failed: name=44/4401, ext=tail.priv > > I also try to change the parameter in monetdb5.conf about the memory: > gdk_maxsize and gdk_vm_minsize but it still does NOT work. > please don't play with these settings. They only control a thread for flushing dirty pages to disk. > Could you please help me to figure out which parameters I could change to > make it work. > > Thank you very much > Trung > >

1 0