Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk_bbp.mx, , 1.251, 1.252 gdk_utils.mx, , 1.244, 1.245
Peter Boncz wrote:
Update of /cvsroot/monetdb/MonetDB/src/gdk In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv27450
Modified Files: gdk_bbp.mx gdk_utils.mx Log Message: gdk_utils.mx:
- performance fix: do not use malloc() on Linux either its malloc() implementation appears to give sizable private arenas to each new thread that performs significant malloc work; this leads to huge memory fragmentation (especially when used with mitosis)
gdk_bbp.mx:
- BBPtrim() should not be allowed to trim logical views this may lead to BBPdecref errors!!
Is this a bug fix that should also be applied to the stable branch?
U gdk_bbp.mx Index: gdk_bbp.mx =================================================================== RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_bbp.mx,v retrieving revision 1.251 retrieving revision 1.252 diff -u -d -r1.251 -r1.252 --- gdk_bbp.mx 31 Mar 2009 13:42:54 -0000 1.251 +++ gdk_bbp.mx 9 Apr 2009 15:29:18 -0000 1.252 @@ -1600,7 +1600,7 @@ @{ @h #define BBPswappable(b) ((b) && BBP_refs((b)->batCacheid) == 0) -#define BBPtrimmable(b) (BBPswappable(b) && !isVIEW(b) && (BBP_status((b)->batCacheid)&BBPWAITING) == 0) +#define BBPtrimmable(b) (BBPswappable(b) && b->P->lview == 0 && isVIEW(b) == 0 && (BBP_status((b)->batCacheid)&BBPWAITING) == 0) @- The @%BBP_ref@ contains the amount of live references to a BAT. These might be in recursive BATs, C or MIL variables. The count is
U gdk_utils.mx Index: gdk_utils.mx =================================================================== RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_utils.mx,v retrieving revision 1.244 retrieving revision 1.245 diff -u -d -r1.244 -r1.245 --- gdk_utils.mx 7 Apr 2009 14:22:55 -0000 1.244 +++ gdk_utils.mx 9 Apr 2009 15:29:18 -0000 1.245 @@ -1053,7 +1053,6 @@ #endif } size = (size + 7) & ~7; /* round up to a multiple of eight */ -#ifndef HAVE_POSIX_FADVISE if (size > GDK_mem_bigsize) { size_t newsize = size + sizeof(size_t) + sizeof(size_t); size_t newmax = MAX(*maxsize, newsize); @@ -1068,7 +1067,6 @@ *maxsize = newmax - (sizeof(size_t) + sizeof(size_t)); return (void *) s; } -#endif CHKMEM(size, 0); GDKmalloc_prefixsize(s, size); if (s == NULL) { @@ -1205,7 +1203,6 @@ } } #ifndef GDK_MEM_MISALIGN -#ifndef HAVE_POSIX_FADVISE else if (size <= GDK_mem_bigsize) { void *oldblk = blk; size_t newsize = size + 8; @@ -1239,7 +1236,6 @@ return blk; } } -#endif /* alloc© due to failed realloc (may be very big heap that needs vm) */ #else /* alloc© because we cannot trust realloc due to misalignment */
This SF.net email is sponsored by: High Quality Requirements in a Collaborative Environment. Download a free trial of Rational Requirements Composer Now! http://p.sf.net/sfu/www-ibm-com _______________________________________________ Monetdb-checkins mailing list Monetdb-checkins@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
Yes, it actually should.
Sent from my iPhone
On Apr 9, 2009, at 5:42 PM, "Sjoerd Mullender" sjoerd@acm.org wrote:
Peter Boncz wrote:
Update of /cvsroot/monetdb/MonetDB/src/gdk In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv27450
Modified Files: gdk_bbp.mx gdk_utils.mx Log Message: gdk_utils.mx:
- performance fix: do not use malloc() on Linux either
its malloc() implementation appears to give sizable private arenas to each new thread that performs significant malloc work; this leads to huge memory fragmentation (especially when used with mitosis)
gdk_bbp.mx:
- BBPtrim() should not be allowed to trim logical views
this may lead to BBPdecref errors!!
Is this a bug fix that should also be applied to the stable branch?
U gdk_bbp.mx Index: gdk_bbp.mx =================================================================== RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_bbp.mx,v retrieving revision 1.251 retrieving revision 1.252 diff -u -d -r1.251 -r1.252 --- gdk_bbp.mx 31 Mar 2009 13:42:54 -0000 1.251 +++ gdk_bbp.mx 9 Apr 2009 15:29:18 -0000 1.252 @@ -1600,7 +1600,7 @@ @{ @h #define BBPswappable(b) ((b) && BBP_refs((b)->batCacheid) == 0) -#define BBPtrimmable(b) (BBPswappable(b) && !isVIEW(b) && (BBP_status((b)->batCacheid)&BBPWAITING) == 0) +#define BBPtrimmable(b) (BBPswappable(b) && b->P->lview == 0 && isVIEW(b) == 0 && (BBP_status((b)->batCacheid)&BBPWAITING) == 0) @- The @%BBP_ref@ contains the amount of live references to a BAT. These might be in recursive BATs, C or MIL variables. The count is
U gdk_utils.mx Index: gdk_utils.mx =================================================================== RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_utils.mx,v retrieving revision 1.244 retrieving revision 1.245 diff -u -d -r1.244 -r1.245 --- gdk_utils.mx 7 Apr 2009 14:22:55 -0000 1.244 +++ gdk_utils.mx 9 Apr 2009 15:29:18 -0000 1.245 @@ -1053,7 +1053,6 @@ #endif } size = (size + 7) & ~7; /* round up to a multiple of eight */ -#ifndef HAVE_POSIX_FADVISE if (size > GDK_mem_bigsize) { size_t newsize = size + sizeof(size_t) + sizeof(size_t); size_t newmax = MAX(*maxsize, newsize); @@ -1068,7 +1067,6 @@ *maxsize = newmax - (sizeof(size_t) + sizeof(size_t)); return (void *) s; } -#endif CHKMEM(size, 0); GDKmalloc_prefixsize(s, size); if (s == NULL) { @@ -1205,7 +1203,6 @@ } } #ifndef GDK_MEM_MISALIGN -#ifndef HAVE_POSIX_FADVISE else if (size <= GDK_mem_bigsize) { void *oldblk = blk; size_t newsize = size + 8; @@ -1239,7 +1236,6 @@ return blk; } } -#endif /* alloc© due to failed realloc (may be very big heap that needs vm) */ #else /* alloc© because we cannot trust realloc due to misalignment */
This SF.net email is sponsored by: High Quality Requirements in a Collaborative Environment. Download a free trial of Rational Requirements Composer Now! http://p.sf.net/sfu/www-ibm-com _______________________________________________ Monetdb-checkins mailing list Monetdb-checkins@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
-- Sjoerd Mullender
No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.0.238 / Virus Database: 270.11.39/2038 - Release Date: 04/08/09 19 :02:00
<signature.asc>
This SF.net email is sponsored by: High Quality Requirements in a Collaborative Environment. Download a free trial of Rational Requirements Composer Now! http://p.sf.net/sfu/www-ibm-com _______________________________________________ Monetdb-checkins mailing list Monetdb-checkins@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
I have backported the gdk_bbp change.
On 2009-04-09 19:31, Peter Boncz wrote:
Yes, it actually should.
Sent from my iPhone
On Apr 9, 2009, at 5:42 PM, "Sjoerd Mullender" sjoerd@acm.org wrote:
Peter Boncz wrote:
Update of /cvsroot/monetdb/MonetDB/src/gdk In directory 23jxhf1.ch3.sourceforge.com:/tmp/cvs-serv27450
Modified Files: gdk_bbp.mx gdk_utils.mx Log Message: gdk_utils.mx:
- performance fix: do not use malloc() on Linux either
its malloc() implementation appears to give sizable private arenas to each new thread that performs significant malloc work; this leads to huge memory fragmentation (especially when used with mitosis)
gdk_bbp.mx:
- BBPtrim() should not be allowed to trim logical views
this may lead to BBPdecref errors!!
Is this a bug fix that should also be applied to the stable branch?
U gdk_bbp.mx Index: gdk_bbp.mx =================================================================== RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_bbp.mx,v retrieving revision 1.251 retrieving revision 1.252 diff -u -d -r1.251 -r1.252 --- gdk_bbp.mx 31 Mar 2009 13:42:54 -0000 1.251 +++ gdk_bbp.mx 9 Apr 2009 15:29:18 -0000 1.252 @@ -1600,7 +1600,7 @@ @{ @h #define BBPswappable(b) ((b) && BBP_refs((b)->batCacheid) == 0) -#define BBPtrimmable(b) (BBPswappable(b) && !isVIEW(b) && (BBP_status((b)->batCacheid)&BBPWAITING) == 0) +#define BBPtrimmable(b) (BBPswappable(b) && b->P->lview == 0 && isVIEW(b) == 0 && (BBP_status((b)->batCacheid)&BBPWAITING) == 0) @- The @%BBP_ref@ contains the amount of live references to a BAT. These might be in recursive BATs, C or MIL variables. The count is
U gdk_utils.mx Index: gdk_utils.mx =================================================================== RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_utils.mx,v retrieving revision 1.244 retrieving revision 1.245 diff -u -d -r1.244 -r1.245 --- gdk_utils.mx 7 Apr 2009 14:22:55 -0000 1.244 +++ gdk_utils.mx 9 Apr 2009 15:29:18 -0000 1.245 @@ -1053,7 +1053,6 @@ #endif } size = (size + 7) & ~7; /* round up to a multiple of eight */ -#ifndef HAVE_POSIX_FADVISE if (size > GDK_mem_bigsize) { size_t newsize = size + sizeof(size_t) + sizeof(size_t); size_t newmax = MAX(*maxsize, newsize); @@ -1068,7 +1067,6 @@ *maxsize = newmax - (sizeof(size_t) + sizeof(size_t)); return (void *) s; } -#endif CHKMEM(size, 0); GDKmalloc_prefixsize(s, size); if (s == NULL) { @@ -1205,7 +1203,6 @@ } } #ifndef GDK_MEM_MISALIGN -#ifndef HAVE_POSIX_FADVISE else if (size <= GDK_mem_bigsize) { void *oldblk = blk; size_t newsize = size + 8; @@ -1239,7 +1236,6 @@ return blk; } } -#endif /* alloc© due to failed realloc (may be very big heap that needs vm) */ #else /* alloc© because we cannot trust realloc due to misalignment */
This SF.net email is sponsored by: High Quality Requirements in a Collaborative Environment. Download a free trial of Rational Requirements Composer Now! http://p.sf.net/sfu/www-ibm-com _______________________________________________ Monetdb-checkins mailing list Monetdb-checkins@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
-- Sjoerd Mullender
No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.0.238 / Virus Database: 270.11.39/2038 - Release Date: 04/08/09 19 :02:00
<signature.asc>
This SF.net email is sponsored by: High Quality Requirements in a Collaborative Environment. Download a free trial of Rational Requirements Composer Now! http://p.sf.net/sfu/www-ibm-com _______________________________________________ Monetdb-checkins mailing list Monetdb-checkins@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-checkins
This SF.net email is sponsored by: High Quality Requirements in a Collaborative Environment. Download a free trial of Rational Requirements Composer Now! http://p.sf.net/sfu/www-ibm-com _______________________________________________ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
participants (2)
-
Peter Boncz -
Sjoerd Mullender