[Monetdb-developers] windows VM fragmentation

p.a.boncz p.a.boncz at chello.nl
Fri Sep 8 17:30:08 CEST 2006


Hi,

Windows is indeed more vulnerable to VM fragmentation, but we still may take
a look at the current behavior on 32-bits systems. Maybe things are worse
than they need to be.

Erwin, maybe you can try on your windows machine to type in the Mserver
console:

mem_printmap();

and report on the results?

Windows gets some more breathing space when running on an Enterprise Edition
OS, it then can use 3GB instead of 2GB. However, binaries that want to be
able to use 3GB should be flagged to do so. This can be done using the
EDITBIN.EXE utility from Microsoft (part of the VC++ dirstribution):

EDITBIN /LARGEADDRESSAWARE 
        /STACK:1048576,65536 
        /HEAP:1677216,65536 
        Mserver.exe

(Type this all on one command line. It will modify some fields in the
Mserver.exe binary)

The above EDITBIN command also reduces the stack space to 1MB per thread and
creates a heap of 16MB. The default stack space on windows is a huge 32MB
and with all the threads that MonetDB/XQuery creates, you are easily 500MB
poorer in terms of VM space available. 

[ Sjoerd: we should do this by default on our Win32 builds ]

Note that the Enterprise Editions of Windows are very expensive. I do think
though, that win32 programs on Windows-64bits edition should be able to use
3GB, and even 4GB, when running 32-bits compiled programs. So, if you posses
an Athlon64 or modern Pentium4, you could switch to the 64-bits Windows XP
Professional, which is not very expensive. 

The high VM pressure generated by MonetDB is a feature/design-error that at
this stage is impossible to correct. When compiled in 64-bits (on a 64-bits
OS, of course), though, it is a non-issue. There we can happily play with
terabytes of VM. For the moment, however, as a MonetDB/XQuery user that
wants to run 64-bits, your only option is Linux, as we do not provide a
64-bits Windows build yet. 

The problem is that Microsoft Visual C++ compiler cannot compile
MonetDB/XQuery, as we use the full ANSI C-99 standard (which Microsoft does
not support). The current MonetDB/XQuery windows binary is therefore
compiled with the GNU gcc compiler (MingW), which is only available in
32-bit. 

We are trying to address this issue (well, Sjoerd is), and hope to provide a
64-bits Windows binary in the next major MonetDB/XQuery release.

Best,

Peter Boncz





More information about the developers-list mailing list