[MonetDB-users] ODBC connection issues

Martin Kersten Martin.Kersten at cwi.nl
Wed Oct 19 02:37:21 CEST 2011


Dear Jeremy,

1) i guess they have used an older 32-bit windows system version.
See for information about the database sizes and platforms:
http://www.monetdb.org/Documentation/UserGuide/resources
In case of a crash, I hope they reported the problem on the
MonetDB user/developer list for handling it.

2) remarks on TPCH in isolation do not make much sense.
They should provide comparative figures on the same platform
and report against other open-source systems.
On the old website we had figures that clearly showed MySQL and Postgresql
to be way slower with > SF-2. MonetDB runs SF-100 (100GB) on my Linux desktop
without problems and it certainly does not have 100GB RAM ;)
Overall, on the 4TB Skyserver application, MonetDB and MS SQLserver,
were comparable, with some queries handled better by either one.

3) You might also have a look at http://homepages.cwi.nl/~mk/ontimeReport
for a comparison on a 120M row datawarehouse and comparative performance
against others.

Overall, performance evaluations have to be done with care. It requires
good understanding on many aspects of the systems under test. Most importantly
the impact of the resource parameters and versions.

regards, Martin

On 10/19/11 1:53 AM, Stefan de Konink wrote:
> On Tue, 18 Oct 2011, Jeremy OSullivan wrote:
>
>> till system crash when data volume exceeds total capacity of memory and
>> swap.
>
> Since MonetDB uses memory mapping any "data volume" is in essence 'swap',
> it is up to the operation system to load these parts in memory that it is
> processing on.
>
>
>> So the high performance of MMDB relies on sufficient memory supply. "
>>
>> Is this a fair assessment ?
>
> I don't think so; it is about selectivity. First of all MonetDB is a
> column store thus only the columns that are used in a query plan have to
> be in memory, either in paralel or in sequence. The total amount data is
> therefore not the bottle neck, the selectivity of the query is. Since meta
> data on columns exist data can be read from a specific offset.
>
> The memory mapping allows a specific region te be load into memory by the
> Operating System. Obviously this will not really help with a sequential
> scan on a column that is bigger than the available memory of the host
> system, but given that scenario the actual performance issue is in the
> bandwidth of storage to the memory, and not the amount of memory: the
> data is a supply by itself - swap space.
>
> The only thing regarding to 'system crash' I can currently think of: if
> the intermediate results exceed the amount available memory (in physical
> RAM and in swap)...
>
>
> Stefan
>
> ------------------------------------------------------------------------------
> All the data continuously generated in your IT infrastructure contains a
> definitive record of customers, application performance, security
> threats, fraudulent activity and more. Splunk takes this data and makes
> sense of it. Business sense. IT sense. Common sense.
> http://p.sf.net/sfu/splunk-d2d-oct
> _______________________________________________
> MonetDB-users mailing list
> MonetDB-users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/monetdb-users





More information about the users-list mailing list