On Thu, Apr 22, 2010 at 6:53 AM, dariuszs <dariuszs@svnp.com> wrote:

Hi Cheng,
Try this
Change the limit of open files:
ulimit -n1024000
Start your server
Start your client
This worked for me. Dariusz

Hering Cheng wrote:

Using Feb2010 release resulted in a different error. This time, mserver5 did not crash:

MAPI = chenher@rdcuxsrv220:50000
QUERY = COPY 34274958 RECORDS INTO app.t_compute_all_closeprice FROM '/home/chenher/gaal/tmp/tar/t_compute_all_closeprice.txt' DELIMITERS '|', '\n';
ERROR = !MALException:!ERROR: HEAPextend: failed to extend to 274199664 for 17/1745tail

Here is the configuration from log file:

2010-04-21 13:11:51 MSG tar[537]: # MonetDB server v5.18.1, based on kernel v1.36.1
2010-04-21 13:11:51 MSG tar[537]: # Serving database 'tar', using 16 threads
2010-04-21 13:11:51 MSG tar[537]: # Compiled for sparc-sun-solaris2.10/64bit with 64bit OIDs dynamically linked
2010-04-21 13:11:51 MSG tar[537]: # Found 32.000 GiB available main-memory.
2010-04-21 13:11:51 MSG tar[537]: # Copyright (c) 1993-July 2008 CWI.
2010-04-21 13:11:51 MSG tar[537]: # Copyright (c) August 2008-2010 MonetDB B.V., all rights reserved
2010-04-21 13:11:51 MSG tar[537]: # Visit http://monetdb.cwi.nl/ for further information
2010-04-21 13:11:51 MSG tar[537]: # Listening for connection requests on mapi:monetdb://127.0.0.1:50001/ <http://127.0.0.1:50001/>

2010-04-21 13:11:52 MSG tar[537]: # MonetDB/SQL module v2.36.1 loaded

Any suggestion?

On Wed, Apr 21, 2010 at 1:16 PM, Hering Cheng <hering.cheng@computer.org <mailto:hering.cheng@computer.org>> wrote:

The statement I gave is the only statement that I passed to
mclient, using --statement argument.

On Wed, Apr 21, 2010 at 11:32 AM, Niels Nes <Niels.Nes@cwi.nl
<mailto:Niels.Nes@cwi.nl>> wrote:

On Wed, Apr 21, 2010 at 11:08:46AM -0700, Hering Cheng wrote:
> Here is the single COPY statement I was using:
>
> COPY 34274958 RECORDS INTO app.t_compute_all_closeprice FROM
'/home/
> chenher/gaal/tmp/tar/t_compute_all_closeprice.txt'
DELIMITERS '|', '\
> n';

I was wondering because the crash you have seems to indicate its
in the commit, ie your load finished but then the commit (auto
or not)
crashes. Probably because of some out of memory problem.

niels
>
> On Wed, Apr 21, 2010 at 11:02 AM, Niels Nes
<Niels.Nes@cwi.nl <mailto:Niels.Nes@cwi.nl>> wrote:
>
> On Wed, Apr 21, 2010 at 10:48:48AM -0700, Hering Cheng
wrote:
> > Hi,
> >
> > I am trying to load 77 GB of data (or 34,274,958
records) into
> MonetDB,
> > using the Feb2010 SP1 source code compiled on Solaris
SPARC with
> Sun
> > Studio 12.1 compiler with debugging enabled, and the
database
> crashes 5
> > hours into the loading. Here is some information about my
> > configuration:
> >
> > I start up with Merovingian like this:
> >
> > $ ( ulimit -d $[32*1024*1024]; export
LD_PRELOAD_64=/usr/lib/64/
> > libumem.so:${LD_PRELOAD_64}; export LD_PRELOAD=/usr/lib/
> libumem.so:$
> > {LD_PRELOAD}; export MONETDB5CONF=/GAAL/chenher/
> rdcuxsrv220-local-disk/
> > chenher/monetdb/feb2010sp1/etc/monetdb5.conf;
/GAAL/chenher/share
> /
> > monetdb/distro-sparc-feb2010-sp1-64bit/bin/merovingian; )
> >
> > Here are the relevent log sections:
> >
> > 2010-04-16 13:25:28 MSG merovingian[4302]: starting
database
> 'tar', up
> > min/avg/max: 4m/4m/4m, crash average: 0.00 0.00 0.00
(1-1=0)
> > 2010-04-16 13:25:29 MSG tar[15594]: arguments:
/GAAL/chenher/
> share/
> > monetdb/distro-sparc-feb2010-sp1-64bit/bin/mserver5
--config=/
> GAAL/
> >
chenher/rdcuxsrv220-local-disk/chenher/monetdb/feb2010sp1/etc/
> > monetdb5.conf --dbname=tar --dbinit=include sql; --set
> merovingian_uri=
> > mapi:monetdb://rdcuxsrv220:50000/tar --set
monet_daemon=yes --set
> > mapi_open=false --set mapi_autosense=true --set
mapi_port=50001
> --set
> >
monet_vault_key=/GAAL/chenher/rdcuxsrv220-local-disk/chenher/
> monetdb/
> > feb2010sp1/var/MonetDB5/dbfarm/tar/.vaultkey --set
sql_optimizer=
> > default_pipe
> > 2010-04-16 13:25:34 MSG tar[15594]: # MonetDB server
v5.18.3,
> based on
> > kernel v1.36.3
> > 2010-04-16 13:25:34 MSG tar[15594]: # Serving database
'tar',
> using 16
> > threads
> > 2010-04-16 13:25:34 MSG tar[15594]: # Compiled for
> > sparc-sun-solaris2.10/64bit with 64bit OIDs
dynamically linked
> > 2010-04-16 13:25:34 MSG tar[15594]: # Found 32.000 GiB
available
> > main-memory.
> > 2010-04-16 13:25:34 MSG tar[15594]: # Copyright (c)
1993-July
> 2008 CWI.
> > 2010-04-16 13:25:34 MSG tar[15594]: # Copyright (c) August
> 2008-2010
> > MonetDB B.V., all rights reserved
> > 2010-04-16 13:25:34 MSG tar[15594]: # Visit
http://monetdb.cwi.nl
> / for
> > further information
> > 2010-04-16 13:25:34 MSG tar[15594]: # Listening for
connection
> requests
> > on mapi:monetdb://127.0.0.1:50001/
<http://127.0.0.1:50001/>

> > 2010-04-16 13:25:35 MSG tar[15594]: # MonetDB/SQL
module v2.36.3
> loaded
> > 2010-04-16 13:25:35 MSG control[4302]:
rdcuxsrv220:58146: started
> > database 'tar'
> > ...
> > 2010-04-21 10:23:37 MSG merovingian[4302]: database
'tar' (15594)
> has
> > crashed (dumped core)
> >
> > Using Sun's DBX tool to examine the crash:
> >
> > ...
> > t@212 (l@212) terminated by signal SEGV (no mapping at
the fault
> > address)
> > Current function is temp_create
> > 87 temp_dup(b->batCacheid);
> > (dbx) where
> > current thread: t@212
> > =>[1] temp_create(b = (nil)), line 87 in "bat_utils.c"
> > [2] ebat2real(b = 5714, ibase = 0), line 162 in
"bat_utils.c"
> > [3] delta_append_bat(bat = 0x104923c98, i =
0x10143ec98), line
> 234 in
> > "bat_storage.c"
> > [4] append_col(tr = 0x104fe1e98, c = 0x1045c7af8, i =
> 0x10143ec98,
> > tpe = 5), line 267 in "bat_storage.c"
> > [5] mvc_append_wrap(cntxt = 0xffffffff7f352c00, mb =
> 0x1050d5d98, stk
> > = 0x101816018, pci = 0x1056a7c18), line 1120 in "sql.c"
> > [6] runMALsequence(cntxt = 0xffffffff7f352c00, mb =
> 0x1050d5d98,
> > startpc = 1, stoppc = 0, stk = 0x101816018, env = (nil),
> pcicaller =
> > (nil)), line 2908 in "mal_interpreter.c"
> > [7] callMAL(cntxt = 0xffffffff7f352c00, mb =
0x1050d5d98, env =
> > 0xfffffffddabff908, argv = 0xfffffffddabff968, debug =
'\0'),
> line 402
> > in "mal_interpreter.c"
> > [8] SQLexecutePrepared(c = 0xffffffff7f352c00, be =
> 0x1050f0ec8, q =
> > 0x104fe1b18), line 1196 in "sql_scenario.c"
> > [9] SQLengineIntern(c = 0xffffffff7f352c00, be =
0x1050f0ec8),
> line
> > 1249 in "sql_scenario.c"
> > [10] SQLengine(c = 0xffffffff7f352c00), line 1349 in
> "sql_scenario.c"
> > [11] runPhase(c = 0xffffffff7f352c00, phase = 4),
line 363 in
> > "mal_scenario.c"
> > [12] runScenarioBody(c = 0xffffffff7f352c00), line
412 in
> > "mal_scenario.c"
> > [13] runScenario(c = 0xffffffff7f352c00), line 438 in
> > "mal_scenario.c"
> > [14] MSserveClient(dummy = 0xffffffff7f352c00), line
368 in
> > "mal_session.c"
> > (dbx) threads
> > t@1 a l@1 ?() LWP suspended in __pollsys()
> > t@2 a l@2 SERVERlistenThread() LWP
suspended in > __pollsys
> > ()
> > t@3 a l@3 mvc_logmanager() LWP suspended in > __pollsys()
> > t@4 b l@4 umem_update_thread() sleep on
(unknown) in > > __lwp_park()
> > t@13 a l@13 runDFLOWworker() sleep on
0x101ea9a48 in > > __lwp_park()
> > t@14 a l@14 runDFLOWworker() sleep on
0x101ea9a48 in > > __lwp_park()
> > t@15 a l@15 runDFLOWworker() sleep on
0x101ea9a48 in > > __lwp_park()
> > t@16 a l@16 runDFLOWworker() sleep on
0x101ea9a48 in > > __lwp_park()
> > t@17 a l@17 runDFLOWworker() sleep on
0x101ea9a48 in > > __lwp_park()
> > t@18 a l@18 runDFLOWworker() sleep on
0x101ea9a48 in > > __lwp_park()
> > t@19 a l@19 runDFLOWworker() sleep on
0x101ea9a48 in > > __lwp_park()
> > t@20 a l@20 runDFLOWworker() sleep on
0x101ea9a48 in > > __lwp_park()
> > t@21 a l@21 runDFLOWworker() sleep on
0x101ea9a48 in > > __lwp_park()
> > t@22 a l@22 runDFLOWworker() sleep on
0x101ea9a48 in > > __lwp_park()
> > t@23 a l@23 runDFLOWworker() sleep on
0x101ea9a48 in > > __lwp_park()
> > t@24 a l@24 runDFLOWworker() sleep on
0x101ea9a48 in > > __lwp_park()
> > t@25 a l@25 runDFLOWworker() sleep on
0x101ea9a48 in > > __lwp_park()
> > t@26 a l@26 runDFLOWworker() sleep on
0x101ea9a48 in > > __lwp_park()
> > t@27 a l@27 runDFLOWworker() sleep on
0x101ea9a48 in > > __lwp_park()
> > t@28 a l@28 runDFLOWworker() sleep on
0x101ea9a48 in > > __lwp_park()
> > o> t@212 a l@212 ?() signal SIGSEGV in temp_create()
> >
> > With Feb2010 release, I managed to load a much larger
database
> without
> > a crash. I will try loading this smaller data set
again with the
> > Feb2010 release and report what happens. In the
meantime, can
> you give
> > me some suggestions?
> >
> > Thanks.
> > Hering
> >
>
> Are you loading with a single COPY into or with several
insert
> into?
>
> Niels
>
> >
> ------------------------------------------------------------------------------
>
> > _______________________________________________
> > MonetDB-users mailing list
> > MonetDB-users@lists.sourceforge.net
<mailto:MonetDB-users@lists.sourceforge.net>

> > https://lists.sourceforge.net/lists/listinfo/monetdb-users
>
>
> --
> Niels Nes, Centrum Wiskunde & Informatica (CWI)
> Science Park 123, 1098 XG Amsterdam, The Netherlands
> room L3.14, phone ++31 20 592-4098 sip:4098@sip.cwi.nl <mailto:sip%3A4098@sip.cwi.nl>

> url: http://www.cwi.nl/~niels
<http://www.cwi.nl/%7Eniels> e-mail: Niels.Nes@cwi.nl
<mailto:Niels.Nes@cwi.nl>

>
>

--
Niels Nes, Centrum Wiskunde & Informatica (CWI)
Science Park 123, 1098 XG Amsterdam, The Netherlands
room L3.14, phone ++31 20 592-4098 sip:4098@sip.cwi.nl
<mailto:sip%3A4098@sip.cwi.nl>
url: http://www.cwi.nl/~niels <http://www.cwi.nl/%7Eniels> e-mail: Niels.Nes@cwi.nl <mailto:Niels.Nes@cwi.nl>

------------------------------------------------------------------------

------------------------------------------------------------------------------

------------------------------------------------------------------------

_______________________________________________
MonetDB-users mailing list
MonetDB-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/monetdb-users