developers-list Digest, Vol 7, Issue 20

Meenesh sharma mrd.thinkbig at gmail.com
Sat Mar 30 14:18:02 CET 2013


sir/mam,
I unable to read the contents therefore neither get my query's answer nor
able to open it .... . . ..

do send way to open it .


On Fri, Mar 29, 2013 at 4:30 PM, <developers-list-request at monetdb.org>wrote:

> Send developers-list mailing list submissions to
>         developers-list at monetdb.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         http://mail.monetdb.org/mailman/listinfo/developers-list
> or, via email, send a message with subject or body 'help' to
>         developers-list-request at monetdb.org
>
> You can reach the person managing the list at
>         developers-list-owner at monetdb.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of developers-list digest..."
>
>
> Today's Topics:
>
>    1. Re: how to speed up the loading of data into a table (Niels Nes)
>    2. Re: how to speed up the loading of data into a table (Niels Nes)
>    3. Re: how to speed up the loading of data into a table
>       (Stefan Manegold)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 28 Mar 2013 20:17:08 +0100
> From: Niels Nes <Niels.Nes at cwi.nl>
> To: "Communication channel for developers of the MonetDB suite."
>         <developers-list at monetdb.org>
> Subject: Re: how to speed up the loading of data into a table
> Message-ID: <20130328191708.GB13470 at niels.cwi.nl>
> Content-Type: text/plain; charset="utf-8"
>
> On Thu, Mar 28, 2013 at 06:41:08PM +0000, Bouslah Ahmed wrote:
> > Good afternoon,
> >
> > I have 2 questions :
> > - 1st question
> > I have a csv file with 300000000 million lines.
> >
> > when I load it in a MonetDB table, it takes more than 16 hours, whereas
> > in Oracle it only takes 15 minutes. Where is the problem?
> > I use mclient as follows :?
> > mclient -u user1 -d demo -ims -s " COPY 300000009 RECORDS INTO
> > LINEORDER FROM STDIN USING DELIMITERS '|','\n'" - < D:\SSB\
> > LINEORDER.tbl
> >
> > ?for information I use :
> >
> > MonetDB5
> > Windows
> > Memory : 16 Go
> > There are two constraints : primary key and foreign key on the table
> > prior to loading data?
> >
> > 2nd question :
> > how inner joins are made in MongoDB?
>                          typo ?
>
>                          Niels
> >
> > cordialy.
>
> > _______________________________________________
> > developers-list mailing list
> > developers-list at monetdb.org
> > http://mail.monetdb.org/mailman/listinfo/developers-list
>
>
> --
> Niels Nes, Centrum Wiskunde & Informatica (CWI)
> Science Park 123, 1098 XG Amsterdam, The Netherlands
> room L3.14,  phone ++31 20 592-4098     sip:4098 at sip.cwi.nl
> url: http://www.cwi.nl/~niels   e-mail: Niels.Nes at cwi.nl
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: not available
> Type: application/pgp-signature
> Size: 198 bytes
> Desc: not available
> URL: <
> http://mail.monetdb.org/pipermail/developers-list/attachments/20130328/18cdf937/attachment-0001.sig
> >
>
> ------------------------------
>
> Message: 2
> Date: Thu, 28 Mar 2013 20:19:05 +0100
> From: Niels Nes <Niels.Nes at cwi.nl>
> To: "Communication channel for developers of the MonetDB suite."
>         <developers-list at monetdb.org>
> Subject: Re: how to speed up the loading of data into a table
> Message-ID: <20130328191905.GC13470 at niels.cwi.nl>
> Content-Type: text/plain; charset="utf-8"
>
> On Thu, Mar 28, 2013 at 06:41:08PM +0000, Bouslah Ahmed wrote:
> > Good afternoon,
> >
> > I have 2 questions :
> > - 1st question
> > I have a csv file with 300000000 million lines.
> >
> > when I load it in a MonetDB table, it takes more than 16 hours, whereas
> > in Oracle it only takes 15 minutes. Where is the problem?
> > I use mclient as follows :?
> > mclient -u user1 -d demo -ims -s " COPY 300000009 RECORDS INTO
> > LINEORDER FROM STDIN USING DELIMITERS '|','\n'" - < D:\SSB\
> > LINEORDER.tbl
> Did you try loading it directly on the server, ie not using the 'STDIN'
> and < D:\\etc
>
> Niels
> >
> > ?for information I use :
> >
> > MonetDB5
> > Windows
> > Memory : 16 Go
> > There are two constraints : primary key and foreign key on the table
> > prior to loading data?
> >
> > 2nd question :
> > how inner joins are made in MongoDB?
> >
> > cordialy.
>
> > _______________________________________________
> > developers-list mailing list
> > developers-list at monetdb.org
> > http://mail.monetdb.org/mailman/listinfo/developers-list
>
>
> --
> Niels Nes, Centrum Wiskunde & Informatica (CWI)
> Science Park 123, 1098 XG Amsterdam, The Netherlands
> room L3.14,  phone ++31 20 592-4098     sip:4098 at sip.cwi.nl
> url: http://www.cwi.nl/~niels   e-mail: Niels.Nes at cwi.nl
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: not available
> Type: application/pgp-signature
> Size: 198 bytes
> Desc: not available
> URL: <
> http://mail.monetdb.org/pipermail/developers-list/attachments/20130328/80c0f1d1/attachment-0001.sig
> >
>
> ------------------------------
>
> Message: 3
> Date: Thu, 28 Mar 2013 20:26:20 +0100
> From: Stefan Manegold <Stefan.Manegold at cwi.nl>
> To: developers-list at monetdb.org, Bouslah Ahmed <a.bouslah at yahoo.fr>
> Subject: Re: how to speed up the loading of data into a table
> Message-ID: <20130328192620.GA14024 at cwi.nl>
> Content-Type: text/plain; charset=iso-8859-1
>
> On Thu, Mar 28, 2013 at 06:41:08PM +0000, Bouslah Ahmed wrote:
> > Good afternoon,
> >
> > I have 2 questions :
> > - 1st question?
> > I have a csv file with 300000000 million lines.
>                          ^^^^^^^^^^^^^^^^^
> Is that   300000000         aka. 300 million aka. 3*10^8
> or indeed 300000000 million aka. 300 billion aka. 3*10^14 ?
>
> > when I load it in a MonetDB table, it takes more than 16 hours, whereas
> in Oracle it only takes 15 minutes. Where is the problem?
> > I use mclient as follows :?
> > mclient -u user1 -d demo -ims -s " COPY 300000009 RECORDS INTO LINEORDER
> FROM STDIN USING DELIMITERS '|','\n'" - < D:\SSB\LINEORDER.tbl
>
> Is the client running on the same machine as the server?
>
> Did/could you try my earlier suggestion of loading directly from file on
> the
> server side (requires to use default admin user "monetdb"), as opposed to
> from STDIN, i.e., passing all data through mclient?
>
> > ?for information I use :
> >
> > MonetDB5
>
> Which release? Feb2013-SP1?
>
> > Windows
> > Memory : 16 Go
>
> How many CPU cores / HW threads?
>
> > There are two constraints : primary key and foreign key on the table
> prior to loading data?
>
> Did/could you try creating the table without constraints, load the data,
> and
> then add the constraints via ALTER TABLE statements.
> Measure the time for both loading the data and adding the constraints
> separately.
>
> With MonetDB Feb2013-SP1, on an 8-core 16 GB Linux machine, we load the
> lineorder table
> of SSBM SF-100 (~600 million rows) in ~30 minutes, and create (and
> validate)
> the keys in ~60 minutes).
>
> > 2nd question :
> > how inner joins are made in MongoDB?
>                               ^^^^^^^
> I have no idea. Please ask the MongoDB people.
>
> In MonetDB, we usually use hash joins.
>
> > cordialy.
>
> Stefan
>
> --
> | Stefan.Manegold at CWI.nl | DB Architectures   (DA) |
> |  www.CWI.nl/~manegold <http://www.CWI.nl/%7Emanegold>  | Science Park
> 123 (L321) |
> |   +31 (0)20 592-4212   | 1098 XG Amsterdam  (NL) |
>
>
> ------------------------------
>
> _______________________________________________
> developers-list mailing list
> developers-list at monetdb.org
> http://mail.monetdb.org/mailman/listinfo/developers-list
>
>
> End of developers-list Digest, Vol 7, Issue 20
> **********************************************
>



-- 
"  HAVE A NICE DAY"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.monetdb.org/pipermail/developers-list/attachments/20130330/be5a6d31/attachment.html>


More information about the developers-list mailing list