[Monetdb-developers] Performance of COPY INTO: data source & constraint checking

Stefan Manegold Stefan.Manegold at cwi.nl
Mon May 18 13:03:24 CEST 2009

On Mon, May 18, 2009 at 11:57:57AM +0200, Stefan de Konink wrote:
> On Mon, 18 May 2009, Stefan Manegold wrote:
> > In case you came to your conclusion from experimental/emperical study, I
> > would be very interested to know about the experiemntes performed and the
> > results measured.
> > In that case, I would be verythankful if you could share the informative
> > details with us, i.e. detailed descriptions of
> > - experimental setup (HW, SW, datasets, workload/queries)
> > - measured results (what was measured how, and what are the results?)
> > - (your) discussion/analysis/interpretation of the results
> I thought the MADAM meeting was for this.

Ok. fine.

... we should then revive the ideas of keeping note of MADAMs (e.g., on a
Wiki or alike) as reference for those who cannot be present ...

> > Could you please elaborate on whether you consider you setup/schema
> > "representaive" and/or whether/to which extend you think that your results
> > for one specific case could be generalized to other cases?
> Technically the only thing I have to prove to get my code reverted is that
> there are cases where late contraints hurt performance.

IMHO, our task should be the following:

- analyze whether there is indeed a significat performance difference
  between early and late constraint checking;
  (5% do not justify any further time to be spent on this)
- and if so, analyze under which circumstances which altertative performs
  (significatly) better or worse than the other;
- and (only) then try to find out where the (significat) performance
  difference comes from
- and (only) then 
  + either try to eliminate the difference;
  + or 
    * make dump/restore use the most suitable alternative (as far as
      possible with reasonable effort);
    * and document our experiences to give adivce for users


> Stefan

| Dr. Stefan Manegold | mailto:Stefan.Manegold at cwi.nl |
| CWI,  P.O.Box 94079 | http://www.cwi.nl/~manegold/  |
| 1090 GB Amsterdam   | Tel.: +31 (20) 592-4212       |
| The Netherlands     | Fax : +31 (20) 592-4312       |

More information about the developers-list mailing list