Re: Help with performance issue

25 Jan 2018


      try to reclaim a.s.a.p., this reduces resource competitions.
martin
Sent from my iPad
...
On 25 Jan 2018, at 19:22, Roberto Cornacchia roberto.cornacchia@gmail.com wrote:
One obvious optimization was to avoid the explicit BATproject and use the candidate inut in BATappend, when possible.
This improved things and now the clean up takes 3/4 of the total time.
Still...
loop(...) {
   BAT *gn, *en, *hn;
   BAT *idxn, *pn;
    // tokenize, group, transform, append
    // this takes 1/4 of the total loop time
    if (BATutf8_tokenize(&tokens, s, delims, min_tok_len) != GDK_SUCCEED) goto fail;

if (BATgroup(&gn, &en, &hn, tokens, NULL, NULL, NULL, NULL) != GDK_SUCCEED) goto fail;
   idxn = BATconstant(0, TYPE_int, &idx, BATcount(en), TRANSIENT);
   pn = BATconvert(hn, NULL, TYPE_dbl, TRUE);
   if (BATappend(br1,idxn,NULL,FALSE) != GDK_SUCCEED) goto fail;
   if (BATappend(br2,tokens,en,FALSE) != GDK_SUCCEED) goto fail;
   if (BATappend(br3,pn,NULL,FALSE) != GDK_SUCCEED) goto fail;
    // this takes 3/4 of the total loop time
    BBPreclaim(idxn);

BBPreclaim(pn);
   BBPreclaim(hn);
   BBPreclaim(gn);
   BBPreclaim(en);
   BBPreclaim(tokens);
}
...
On Thu, 25 Jan 2018 at 18:26 Roberto Cornacchia roberto.cornacchia@gmail.com wrote:
Hi there,
Would someone like to pick up this wonderful opportunity to tell me that I'm doing something silly? Please do.
In this loop, the cleanup take 4/5 of the total time. Is there a better way of doing this?
loop(...) {
   BAT *gn, *en, *hn;
   BAT *idxn, *tn, *pn;
            // tokenize, group, transform, append
            // this takes 1/5 of the total loop time
            if  (BATutf8_tokenize(&tokens, s, delims, min_tok_len) != GDK_SUCCEED) goto fail;

if (BATgroup(&gn, &en, &hn, tokens, NULL, NULL, NULL, NULL) != GDK_SUCCEED) goto fail;
   idxn = BATconstant(0, TYPE_int, &idx, BATcount(en), TRANSIENT);
   tn = BATproject(en,tokens);
   pn = BATconvert(hn, NULL, TYPE_dbl, TRUE);
   if (BATappend(br1,idxn,NULL,FALSE) != GDK_SUCCEED) goto fail;
   if (BATappend(br2,tn,NULL,FALSE) != GDK_SUCCEED) goto fail;
   if (BATappend(br3,pn,NULL,FALSE) != GDK_SUCCEED) goto fail;
            // this takes 4/5 of the total loop time
    BBPreclaim(idxn);

BBPreclaim(tn);
   BBPreclaim(pn);
   BBPreclaim(hn);
   BBPreclaim(gn);
   BBPreclaim(en);
   BBPreclaim(tokens);
}

users-list mailing list
users-list@monetdb.org
https://www.monetdb.org/mailman/listinfo/users-list

Re: Help with performance issue

Martin Kersten