[Monetdb-developers] windows VM fragmentation
Hi,
Windows is indeed more vulnerable to VM fragmentation, but we still may take a look at the current behavior on 32-bits systems. Maybe things are worse than they need to be.
Erwin, maybe you can try on your windows machine to type in the Mserver console:
mem_printmap();
and report on the results?
Windows gets some more breathing space when running on an Enterprise Edition OS, it then can use 3GB instead of 2GB. However, binaries that want to be able to use 3GB should be flagged to do so. This can be done using the EDITBIN.EXE utility from Microsoft (part of the VC++ dirstribution):
EDITBIN /LARGEADDRESSAWARE /STACK:1048576,65536 /HEAP:1677216,65536 Mserver.exe
(Type this all on one command line. It will modify some fields in the Mserver.exe binary)
The above EDITBIN command also reduces the stack space to 1MB per thread and creates a heap of 16MB. The default stack space on windows is a huge 32MB and with all the threads that MonetDB/XQuery creates, you are easily 500MB poorer in terms of VM space available.
[ Sjoerd: we should do this by default on our Win32 builds ]
Note that the Enterprise Editions of Windows are very expensive. I do think though, that win32 programs on Windows-64bits edition should be able to use 3GB, and even 4GB, when running 32-bits compiled programs. So, if you posses an Athlon64 or modern Pentium4, you could switch to the 64-bits Windows XP Professional, which is not very expensive.
The high VM pressure generated by MonetDB is a feature/design-error that at this stage is impossible to correct. When compiled in 64-bits (on a 64-bits OS, of course), though, it is a non-issue. There we can happily play with terabytes of VM. For the moment, however, as a MonetDB/XQuery user that wants to run 64-bits, your only option is Linux, as we do not provide a 64-bits Windows build yet.
The problem is that Microsoft Visual C++ compiler cannot compile MonetDB/XQuery, as we use the full ANSI C-99 standard (which Microsoft does not support). The current MonetDB/XQuery windows binary is therefore compiled with the GNU gcc compiler (MingW), which is only available in 32-bit.
We are trying to address this issue (well, Sjoerd is), and hope to provide a 64-bits Windows binary in the next major MonetDB/XQuery release.
Best,
Peter Boncz
There is a /3GB switch on normal XP versions as well:
http://www.microsoft.com/whdc/system/platform/server/PAE/PAEmem.mspx
2cts
Hi,
Windows is indeed more vulnerable to VM fragmentation, but we still may take a look at the current behavior on 32-bits systems. Maybe things are worse than they need to be.
Erwin, maybe you can try on your windows machine to type in the Mserver console:
mem_printmap();
and report on the results?
Windows gets some more breathing space when running on an Enterprise Edition OS, it then can use 3GB instead of 2GB. However, binaries that want to be able to use 3GB should be flagged to do so. This can be done using the EDITBIN.EXE utility from Microsoft (part of the VC++ dirstribution):
EDITBIN /LARGEADDRESSAWARE /STACK:1048576,65536 /HEAP:1677216,65536 Mserver.exe
(Type this all on one command line. It will modify some fields in the Mserver.exe binary)
The above EDITBIN command also reduces the stack space to 1MB per thread and creates a heap of 16MB. The default stack space on windows is a huge 32MB and with all the threads that MonetDB/XQuery creates, you are easily 500MB poorer in terms of VM space available.
[ Sjoerd: we should do this by default on our Win32 builds ]
Note that the Enterprise Editions of Windows are very expensive. I do think though, that win32 programs on Windows-64bits edition should be able to use 3GB, and even 4GB, when running 32-bits compiled programs. So, if you posses an Athlon64 or modern Pentium4, you could switch to the 64-bits Windows XP Professional, which is not very expensive.
The high VM pressure generated by MonetDB is a feature/design-error that at this stage is impossible to correct. When compiled in 64-bits (on a 64-bits OS, of course), though, it is a non-issue. There we can happily play with terabytes of VM. For the moment, however, as a MonetDB/XQuery user that wants to run 64-bits, your only option is Linux, as we do not provide a 64-bits Windows build yet.
The problem is that Microsoft Visual C++ compiler cannot compile MonetDB/XQuery, as we use the full ANSI C-99 standard (which Microsoft does not support). The current MonetDB/XQuery windows binary is therefore compiled with the GNU gcc compiler (MingW), which is only available in 32-bit.
We are trying to address this issue (well, Sjoerd is), and hope to provide a 64-bits Windows binary in the next major MonetDB/XQuery release.
Best,
Peter Boncz
Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&da... _______________________________________________ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
Erwin, maybe you can try on your windows machine to type in the Mserver console:
mem_printmap();
and report on the results?
Hi,
I run using XQuery Server (Start -> Programs -> MonetDB XQuery -> XQuery Server) and type mem_printmap(); but nothing happens.
FYI, I use windows XP Professional version 2002 service pack 2.
Regards,
Klarinda
participants (3)
-
kla gw -
p.a.boncz -
W.Alink＠cwi.nl