Although still unfinished, I can informally share some results from benchmarking MonetDB as follows (pseudo-bash).
for RELEASE in Nov2019 Jun2020
for CC in gcc clang
for OPT in "-O2" "-O3"
for ARCH in "" "-march=skylake -mtune=skylake" # "" = default, "skylake"=the architecture I tested on
# compile ...
for WORKLOAD in tpc-h text spinque
for COMPILATION in (those above)
for OPT_PIPE in default_pipe no_mitosis_pipe sequential_pipe
for RUN in cold hot
# test ...
This was on a 16GB 4-core i7 with SSD disk.
As for workloads, tpc-h was sf 10, "text" is a benchmark on string-oriented functionalities (tokenization, string similarity, etc.), and "spinque" (still to be expanded) is a small collection of real queries that we run on Spinque - very long and nested generated SQL queries, rich of aggregations and string operations, typically on skewed data distributions.
I wanted to write a nice blog about these experiments, but if things stay as they are it would be a very boring blog.
This is my main finding so far: while the choice of optimization pipeline can have a big impact on some queries (for some queries mitosis helps a lot, for others is a real pain),
I have found NO significant difference amongst all the compilation options, in no workload.
I hope it helps.
Roberto