Hi,
No need to resubmit your question. It was received, but I'm sorry that we've been too busy to answer this question.
On 15-09-2006 21:25:15 +0200, Wouter Scherphof wrote:
Hi,
The TPC-H benchmark (http://www.tpc.org/tpch/default.asp) is part of the MonetDB SQL source and the benchmark is run during weekends, as I can see on the TestWeb (http://monetdb.cwi.nl/testing/projects/monetdb/Current/sql/.mTests03/times.h...). How should I interpret the listed results? According to TPC, results should be stated as a "QphH@Size" value, where "QphH" is the number of queries per hour for the H benchmark, and "Size" is a "scaling factor". On the TestWeb, the results are stated as seconds of execution time.
The testweb tpc-h is very very very small, to keep both testweb small in diskspace, and to keep the test time reasonable on our slower testing machines.
Is it correct to compute the QphH value as follows: Queries 01-22 took 1.985 seconds, so QphH is (3600 / 1.985) * 22 = 39.899 ?
Well, the testweb framework does way more than just executing the test, so the performance time of the test might not be accurate. Also, you probably want to test with a higher workload, to get a more accurate result.
What scaling factor is used in the benchmark runs; is this 1? Is it possible to let the TestWeb run with scaling factor 100? Because that's the smallest Size for which results are published on the TPC web site.
It's very small. I don't recall how small, but very very small.
Did anyone yet try to determine the Price of the test platforms, following the TPC Pricing Spec? TPC-H results normally include a Price/Performance metric in the form of $/QphH@Size, and I guess MonetDB could score particularly well on this metric.
We never really did this. But, feel free to try and setup such test, we are very interested in the outcomes, of course.
I hope this answers your questions.
Regards