How to find all running processes and their statuses in MonetDB??
I found this...
https://www.monetdb.org/Documentation/Cookbooks/SQLrecipes/monitor
But I couldn't get this work... I ran a query (which took some 20 seconds
to complete) in a window and connected the same database in another
terminal and ran "select * from sys.queue" (several times during those 20
seconds) but I could get only this...
sql>select * from sys.queue;
+-------+---------+----------------------------+----------------------------+----------+---------+------------+--------------------------+
| qtag | user | started | estimate
| progress | status | tag | query |
+=======+=========+============================+============================+==========+=========+============+==========================+
| 27339 | monetdb | 2015-08-13 05:35:40.000000 | null
| null | running | 12532134@0 | select * from sys.queue; |
+-------+---------+----------------------------+----------------------------+----------+---------+------------+--------------------------+
I am using Oct-2014 version of monetdb on CentOS...
Is there a way to find all running processes and their statuses in
MonetDB??
Thanks & Regards,
Vijayakrishna.P.
Mobile : (+91) 9500402305.
Hi all,
We found a strange case this week when testing some insert.
We tried to insert -1.7976931348623157e308 in a double value and we get a
null in the column.
Data to reproduce the case :
create table test.testdouble
(
id int,
testdouble double,
primary key (id)
)
insert into test.testdouble (id,testdouble) values (1,
-1.7976931348623157e308)
select * from test.testdouble
Regards
Mathieu
Hi all -
Sorry for the multiple posts. Earlier in the fall I downloaded monetdb Jul2015 version on my MAC and could successfully load a file with 60M rows, 37 columns of mixed int/real/text/boolean data. Later I updated to the Jul2015-SP1 pack and was unable to load this same file. After trying to debug, I removed monetdb, reinstalled Jul2015-SP1, still had the problem. Then un-installed, reinstalled the Jul2015 version and it worked again.
So the problem appears to be with the Jul2015-SP1 version.
Details: After creating a schema, I load a .sql file that creates this table:
sql>\d annosites
CREATE TABLE "testjeff"."annosites" (
"chr" INTEGER,
"pos" INTEGER,
"hapmap31_total_depth" INTEGER,
"hapmap31_num_taxa" SMALLINT,
"hapmap31_num_alleles" SMALLINT,
"hapmap31_minor_allele_avg_depth" REAL,
"hapmap31_minor_allele_avg_phred" REAL,
"hapmap31_num_hets" SMALLINT,
"hapmap31_ed_factor" REAL,
"hapmap31_seg_test_p_value" REAL,
"hapmap31_ibd_one_allele" BOOLEAN,
"hapmap31_in_local_ld" BOOLEAN,
"hapmap31_maf" REAL,
"hapmap31_near_indel" BOOLEAN,
"hapmap31_first_alt_allele_is_ins_or_del" BOOLEAN,
"snpeff40e_effect_hapmap31" CHARACTER LARGE OBJECT,
"snpeff40e_effectimpact_hapmap31" CHARACTER LARGE OBJECT,
"snpeff40e_functionalclass_hapmap31" CHARACTER LARGE OBJECT,
"gerp_neutral_tree_length" REAL,
"gerp_score" REAL,
"gerp_conserved" BOOLEAN,
"mnase_low_minus_high_rpm_shoots" REAL,
"mnase_bayes_factor_shoots" REAL,
"mnase_hotspot_shoots" BOOLEAN,
"mnase_low_minus_high_rpm_roots" REAL,
"mnase_bayes_factor_roots" REAL,
"mnase_hotspot_roots" BOOLEAN,
"within_gene" BOOLEAN,
"within_transcript" BOOLEAN,
"within_exon" BOOLEAN,
"within_cds" BOOLEAN,
"within_cds_from_gff3" BOOLEAN,
"within_five_prime_utr" BOOLEAN,
"within_three_prime_utr" BOOLEAN,
"codon_position" SMALLINT,
"go_term_accession" CHARACTER LARGE OBJECT,
"go_term_name" CHARACTER LARGE OBJECT
);
sql>
Then I use a COPY INTO command (sometimes specifying number of records, sometimes not) to load my data file containing 60M lines: (this is the command when it WORKS on the Jul2015 load):
sql>COPY INTO annoSites FROM '/Users/lcj34/notes_files/machineLearningDB/annoDB_related/siteAnnoNoHdrsCol35Fixed_20151011.txt' USING DELIMITERS '\t','\n';
60362853 affected rows (3m 5s)
sql>
When running with the Jul2015-SP1 installed on the MAC (OS 10.9.5) . the COPY INTO command dies (i.e., I get back an sql prompt with no message). The merovingian.log from the MAC shows this:
2015-11-23 13:08:21 MSG test1[1531]: # MonetDB/SQL module loaded
2015-11-23 13:08:21 MSG merovingian[1528]: proxying client 0.0.0.0:0 for database 'test1' to mapi:monetdb:///users/lcj34/development/mydbfarm/test1/.mapi.sock?database=test1
2015-11-23 13:08:21 MSG merovingian[1528]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying
2015-11-23 13:08:21 MSG merovingian[1528]: proxying client 0.0.0.0:0 for database 'test1' to mapi:monetdb:///users/lcj34/development/mydbfarm/test1/.mapi.sock?database=test1
2015-11-23 13:08:21 MSG merovingian[1528]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying
2015-11-23 13:13:01 ERR test1[1531]: mserver5(1531,0x38967f000) malloc: *** error for object 0x7fc17393b208: incorrect checksum for freed object - object was probably modified after being freed.
2015-11-23 13:13:01 ERR test1[1531]: *** set a breakpoint in malloc_error_break to debug
2015-11-23 13:13:03 MSG merovingian[1528]: database 'test1' (1531) was killed by signal SIGABRT
My MAC is just a test bed, our real server is RedHat Release which I am trying to get setup with monetdb. On this machine I loaded the Jul2015-sp1 for REdHat/CentOS following these instructions:
http://rogerhosto.com/installing-monetdb-on-centosredhat/
The errors are a bit different. On Redhat, the command also aborts, sometimes with no message, sometimes with the message below. I show running both with and without specifying the number of records as the error message is slightly different. (Note on the MAC I did not receive a command line message):
sql>COPY 61000000 records INTO annosites FROM '/home/lcj34/monetdbFiles/sites10000Jeff.txt' USING DELIMITERS '\t','\n';
Failed to import table Leftover data 'component of nuclear inner membrane;molecular_function;biological_process;endoplasmic reticulum'
sql>
sql>COPY INTO annosites FROM '/home/lcj34/monetdbFiles/sites10000Jeff.txt' USING DELIMITERS '\t','\n';
Failed to import table Leftover data 'binding'
sql>
There are multiple days/attempts (yesterday and today). The last attempt to COPY the file has these messages:
2015-11-24 12:39:45 MSG merovingian[15627]: proxying client (local) for database 'jeffTest' to mapi:monetdb:///opt/dbfarm/jeffTest/.mapi.sock?database=jeffTest
2015-11-24 12:39:45 MSG merovingian[15627]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying
2015-11-24 12:43:38 ERR jeffTest[15693]: *** Error in `/usr/bin/mserver5': free(): invalid next size (normal): 0x00007f5b44004ac0 ***
2015-11-24 12:43:38 ERR jeffTest[15693]: ======= Backtrace: =========
2015-11-24 12:43:38 ERR jeffTest[15693]: /lib64/libc.so.6(+0x7d1fd)[0x7f5b628101fd]
2015-11-24 12:43:38 ERR jeffTest[15693]: /lib64/libbat.so.12(GDKfree+0x13)[0x7f5b64f6f093]2015-11-24 12:43:38 ERR jeffTest[15693]: /lib64/libmonetdb5.so.19(+0x135e11)[0x7f5b655e1e11]
2015-11-24 12:43:38 ERR jeffTest[15693]: /lib64/libmonetdb5.so.19(+0x136534)[0x7f5b655e2534]2015-11-24 12:43:38 ERR jeffTest[15693]: /lib64/libpthread.so.0(+0x7df5)[0x7f5b62b5bdf5]
2015-11-24 12:43:38 ERR jeffTest[15693]: /lib64/libc.so.6(clone+0x6d)[0x7f5b628891ad]2015-11-24 12:43:38 ERR jeffTest[15693]: ======= Memory map: ========
2015-11-24 12:43:38 ERR jeffTest[15693]: 00400000-00405000 r-xp 00000000 fd:01 1076941767 /usr/bin/mserver5
2015-11-24 12:43:38 ERR jeffTest[15693]: 00604000-00605000 r--p 00004000 fd:01 1076941767 /usr/bin/mserver5
2015-11-24 12:43:38 ERR jeffTest[15693]: 00605000-00606000 rw-p 00005000 fd:01 1076941767 /usr/bin/mserver5
2015-11-24 12:43:38 ERR jeffTest[15693]: 00606000-00608000 rw-p 00000000 00:00 02015-11-24 12:43:38 ERR jeffTest[15693]: 012af000-02860000 rw-p 00000000 00:00 0 [heap]
I can forward a log file if requested.
I'm thinking perhaps there is some problem in the file with a floating point/real number? Though importing this file works with the Jul2015 release. Were there changes related to this in the service pack?
Final question: We need to be testing on the RedHat machine. Is the Jul2015 EPEL for Redhat/CentOS still available?
Thanks - Lynn
Hello. I am trying to create a new table as the union of two existing tables via an R DBI connection, but am receiving an error message that I cannot seem to fix. Any advice to help correct the syntax would be greatly appreciated.
> dbGetQuery(mdb,
+ "CREATE TABLE nis_0411 AS
+ Select * FROM nis_0407
+ UNION ALL
+ Select * FROM is_0811")
Error in .local(conn, statement, ...) :
Unable to execute statement 'CREATE TABLE nis_0411 AS
Select * FROM nis_0407
UNION ALL
Select * FROM nis_0811'.
Server says 'syntax error, unexpected SCOLON, expecting WITH in: "create table nis_0411 as
select * from nis_0407
union all
select * from nis_"
' [#42000].
Cheers
Charles
Charles DiMaggio, PhD, MPH
Director of Injury Research
Department of Surgery
New York University School of Medicine
462 First Avenue, NBV 15
New York, NY 10016-9196
Charles.Dimaggio(a)nyumc.org
Direct Line: 516.308.6426 (preferred)
Office: 212.263.3202
Hi Guys,
I have been noticing quite a few issues on the mailing list regarding
existing features of monetdb.
I think myself testing the new Remote / Merge technology of which bugs are
to be expected to be happening at the moment.
This seems unusual with existing features and reminds me of data corruption
along while in 11.15 I think it was.
Is there anything we can do as the user base / community to help document
as well as compile with debug and run such cases under gdb ?
Thoughts everyone who believes in the amazing MonetDB.
Regards,
Brian Hood
Hi Guys,
I have 1 master and 4 mserver nodes running.
I have 4 ippacket and 4 tcppacket schemas identical per table schemas
across all nodes.
i have 4 remote references on the master to ippacket remote nodes and the
same for tcppacket.
However when i try and join 2 MERGE tables each containing 4 schemas i get
the following.
sql>select count(*) from ippacket a, tcppacket b where a.guid = b.guid;
(mapi:monetdb://monetdb@172.17.0.5/threatmonitor) GDK reported error.
ERROR: BATproject: does not match always
Any pointers please ?
Regards,
Brian Hood
Hi, Martin, great to hear it, thank you very much!
Do you have a schedule, when to release this feature? We're looking forward to running MonetDB in the production environment!
----- 原始邮件 -----
发件人：Martin Kersten <martin.kersten(a)cwi.nl>
收件人：Communication channel for MonetDB users <users-list(a)monetdb.org>
主题：Re: 回复：Range query in MonetDB
日期：2015-11-26 04:02:19
On 25/11/15 17:08, chenkaijiang(a)bigknow.com.cn wrote:
> Sometimes the traditional DB is 10 to 100 times faster with the help of the index.
Yes. It is on the roadmap.
>
> ----- 原始邮件 -----
> 发件人：<chenkaijiang(a)bigknow.com.cn>
> 收件人：users-list<users-list(a)monetdb.org>,developers-list<developers-list(a)monetdb.org>,
> 主题：Range query in MonetDB
> 日期：2015-11-25 23:11:07
>
> Since MonetDB actually ignore SQL statement CREATE INDEX, the performance sometimes is quite poor comparing to traditional DB. Such as: https://www.monetdb.org/pipermail/users-list/2008-April/001839.html,
> http://stackoverflow.com/questions/22430284/monetdb-slow-query-with-index
>
> In reality, queries with ranges (such as date ranges) are very common and thus index is critical. Sometimes, it gives 100 times performance boost, especially when the queries are relatively simple.
>
> We debug into MonetDB source codes and found that CREATE INDEX only creates an entry in the system catalog.
>
> Is MonetDB going to support index in the coming future, or in some roadmap?
>
>
>
>
>
>
>
>
> _______________________________________________
> users-list mailing list
> users-list(a)monetdb.org
> https://www.monetdb.org/mailman/listinfo/users-list
>
_______________________________________________
users-list mailing list
users-list(a)monetdb.org
https://www.monetdb.org/mailman/listinfo/users-list
Sometimes the traditional DB is 10 to 100 times faster with the help of the index.
----- 原始邮件 -----
发件人：<chenkaijiang(a)bigknow.com.cn>
收件人：users-list<users-list(a)monetdb.org>,developers-list<developers-list(a)monetdb.org>,
主题：Range query in MonetDB
日期：2015-11-25 23:11:07
Since MonetDB actually ignore SQL statement CREATE INDEX, the performance sometimes is quite poor comparing to traditional DB. Such as: https://www.monetdb.org/pipermail/users-list/2008-April/001839.html, http://stackoverflow.com/questions/22430284/monetdb-slow-query-with-index
In reality, queries with ranges (such as date ranges) are very common and thus index is critical. Sometimes, it gives 100 times performance boost, especially when the queries are relatively simple.
We debug into MonetDB source codes and found that CREATE INDEX only creates an entry in the system catalog.
Is MonetDB going to support index in the coming future, or in some roadmap?
Driver JDBC 2.8 dont work in JUL2015SP1?
I use 2.8 for ETL (run work with decimal, 2.18 dont work), and 2.18
for dashboards access.
Att,
--
Luciano Sasso Vieira
Data Scientist & Solutions Architect
luciano(a)gsgroup.com.br <http://www.gsgroup.com.br> | tel: 17 3353-0833
| cel: 17 99706-9335
www.gsgroup.com.br <http://www.gsgroup.com.br>
---
Este email foi escaneado pelo Avast antivírus.
https://www.avast.com/antivirus
Since MonetDB actually ignore SQL statement CREATE INDEX, the performance sometimes is quite poor comparing to traditional DB. Such as: https://www.monetdb.org/pipermail/users-list/2008-April/001839.htm…
In reality, queries with ranges (such as date ranges) are very common and thus index is critical. Sometimes, it gives 100 times performance boost, especially when the queries are relatively simple.
We debug into MonetDB source codes and found that CREATE INDEX only creates an entry in the system catalog.
Is MonetDB going to support index in the coming future, or in some roadmap?
