Big Slowdown (15-20X ) on import functions in Monetdb October 2014 Release
All,
With October,2014 release we are stucked in to very strange performance issue while importing data on virtualised environments.
On Virtual machines in our local VMware servers and amazon machine we are seeing import performance has downgraded to close to 15 to 20 while importing ~ 2000 files as compared to 1 hour time being taken with old release. This we are seeing consistently with CentOS, SLES vm and configuration is 4 Core 33 GB machine. We tried on quite a few amazon vm just to make sure its not an issue with a particular instance setup.
On a physical machine of 4 core 24 GB we are able to match speed of import with old monetdb. We are issuing copy into command through JDBC connection.
Can you guys help me with pointers on this issue, I will be happy to share any debug info.
Regards, Ashish Kumar Singh Mobile: +91-9972045095 Skype: toashishk@gmail.com
Is any body else seeing this problem with October,2014 release?
I wish developer team can provide some insights in this area.
Regards, Ashish
From: Ashish Singh <ashishkumar.singh@altair.commailto:ashishkumar.singh@altair.com> Date: Thursday, 13 November 2014 11:22 pm To: "developers-list@monetdb.orgmailto:developers-list@monetdb.org" <developers-list@monetdb.orgmailto:developers-list@monetdb.org> Cc: "users-list@monetdb.orgmailto:users-list@monetdb.org" <users-list@monetdb.orgmailto:users-list@monetdb.org> Subject: Big Slowdown (15-20X ) on import functions in Monetdb October 2014 Release
All,
With October,2014 release we are stucked in to very strange performance issue while importing data on virtualised environments.
On Virtual machines in our local VMware servers and amazon machine we are seeing import performance has downgraded to close to 15 to 20 while importing ~ 2000 files as compared to 1 hour time being taken with old release. This we are seeing consistently with CentOS, SLES vm and configuration is 4 Core 33 GB machine. We tried on quite a few amazon vm just to make sure its not an issue with a particular instance setup.
On a physical machine of 4 core 24 GB we are able to match speed of import with old monetdb. We are issuing copy into command through JDBC connection.
Can you guys help me with pointers on this issue, I will be happy to share any debug info.
Regards, Ashish Kumar Singh Mobile: +91-9972045095 Skype: toashishk@gmail.commailto:toashishk@gmail.com
Additional information here,
If I merge all these files in to one file and then import to monetdb then time taken is 3-5 minutes as compared to 10-15 hours with October 2014 branch.
Hopefully some one from moentdb team will look in to this. I am glad to see that inspite of this slowness ( due to which this build is unusable) the segfaults are gone!
Regards, Ashish
From: Ashish Singh <ashishkumar.singh@altair.commailto:ashishkumar.singh@altair.com> Date: Friday, 14 November 2014 7:41 pm To: "developers-list@monetdb.orgmailto:developers-list@monetdb.org" <developers-list@monetdb.orgmailto:developers-list@monetdb.org> Cc: "users-list@monetdb.orgmailto:users-list@monetdb.org" <users-list@monetdb.orgmailto:users-list@monetdb.org> Subject: Re: Big Slowdown (15-20X ) on import functions in Monetdb October 2014 Release
Is any body else seeing this problem with October,2014 release?
I wish developer team can provide some insights in this area.
Regards, Ashish
From: Ashish Singh <ashishkumar.singh@altair.commailto:ashishkumar.singh@altair.com> Date: Thursday, 13 November 2014 11:22 pm To: "developers-list@monetdb.orgmailto:developers-list@monetdb.org" <developers-list@monetdb.orgmailto:developers-list@monetdb.org> Cc: "users-list@monetdb.orgmailto:users-list@monetdb.org" <users-list@monetdb.orgmailto:users-list@monetdb.org> Subject: Big Slowdown (15-20X ) on import functions in Monetdb October 2014 Release
All,
With October,2014 release we are stucked in to very strange performance issue while importing data on virtualised environments.
On Virtual machines in our local VMware servers and amazon machine we are seeing import performance has downgraded to close to 15 to 20 while importing ~ 2000 files as compared to 1 hour time being taken with old release. This we are seeing consistently with CentOS, SLES vm and configuration is 4 Core 33 GB machine. We tried on quite a few amazon vm just to make sure its not an issue with a particular instance setup.
On a physical machine of 4 core 24 GB we are able to match speed of import with old monetdb. We are issuing copy into command through JDBC connection.
Can you guys help me with pointers on this issue, I will be happy to share any debug info.
Regards, Ashish Kumar Singh Mobile: +91-9972045095 Skype: toashishk@gmail.commailto:toashishk@gmail.com
participants (1)
-
Ashish Kumar Singh