users-list June 2012

users-list@monetdb.org

18 participants
29 discussions

[MonetDB-users] Different results with same query

by Matthieu Guamis

Hello, Running the same query on the same data leeds to different results between postgresql and monetdb. I have not been able to get a simple example to reproduce the problem. So here is what I observe: the query: select temps_mois.rfoperdmo as c1, sum((case when dwhinv.dwhinv___rfodomide = 'RH' and dwhinv.dwhinv___rfoindide = 'tech_rh_effectif_rem_reel' then dwhinv.dwhinvqte else 0 end)) as m0 from rfoper_temps_mois as temps_mois, dwhinv as dwhinv, rfovsn as rfovsn_0, rrhamv as rrhamv_1, rrhcov as rrhcov_2, rfoadv as rfoadv_3 where temps_mois.rfoper___rforefide = 'HPLUS' and dwhinv.dwhinv___rforefide = 'HPLUS' and dwhinv.dwhinv___rfodomide = 'RH' and dwhinv.dwhinv___rfoindide = 'tech_rh_effectif_rem_reel' and dwhinv.dwhinvdtd = temps_mois.rfoperdtd and temps_mois.rfoperyea = '2011' and rfovsn___rforefide = 'HPLUS' and dwhinv.dwhinv___rfovsnide = rfovsn_0.rfovsnide and rfovsn_0.rfovsnide = '201111_reel' and rrhamv_1.rrhamv___rrhvemide='GRACOR' AND rrhamv_1.rrhamvrvs=1 AND rrhamv_1.rrhamv___rforefide= 'HPLUS' and dwhinv.dwhinv___rrhempide = rrhamv_1.rrhamvinf and rrhamv_1.rrhamvsup = 'CEMP' and rrhcov_2.rrhcov___rrhvcoide='CONTRATS' AND rrhcov_2.rrhcovrvs=1 AND rrhcov_2.rrhcov___rforefide= 'HPLUS' and dwhinv.dwhinv___rrhcntide = rrhcov_2.rrhcovinf and rrhcov_2.rrhcovsup = 'CONTRATS' and rfoadv_3.rfoadv___rfovdeide='STRC' AND rfoadv_3.rfoadvrvs=1 AND rfoadv_3.rfoadv___rforefide= 'HPLUS' and dwhinv.dwhinv_p2rfodstide = rfoadv_3.rfoadvinf and rfoadv_3.rfoadvsup = 'HPLUS' group by c1 order by c1 Postgresql result: c1 | m0 ----+----------------- 01 | 7111.5376967750 02 | 7100.9108821426 03 | 7150.2597967742 04 | 7151.4283666667 05 | 7109.1641451610 06 | 6976.2108421239 07 | 6329.7404193564 08 | 6302.9823032247 09 | 6426.9459633351 10 | 6519.6889580648 11 | 6549.5235033402 12 | 6492.5477161292 (12 lignes) Monetdb result: +------+----------------------+ | c1 | m0 | +======+======================+ | 12 | 1.000000 | | 11 | 1.000000 | | 10 | 2.000000 | | 09 | 0.366667 | | 08 | 1.000000 | | 07 | 1.000000 | | 06 | 1.000000 | | 05 | 3.000000 | | 04 | 4.000000 | | 03 | 15.000000 | | 02 | 9.964286 | | 01 | 1.000000 | +------+----------------------+ 12 tuples (212.346ms) As you see, all "m0" values are very different. I have no idea of what is causing this but if I modify the query in order to simplify the CASE...WHEN...ELSE part of the select, it produce the expected result: replacing sum((case when dwhinv.dwhinv___rfodomide = 'RH' and dwhinv.dwhinv___rfoindide = 'tech_rh_effectif_rem_reel' then dwhinv.dwhinvqte else 0 end)) as m0 by sum((case when dwhinv.dwhinv___rfoindide = 'tech_rh_effectif_rem_reel' then dwhinv.dwhinvqte else 0 end)) as m0 Its not exactly the same query but it must leeds to the same result due to my test datas (its the case in postgres) I join to this email two files with the TRACES, respectively for the request with bad/correct result (ko.txt/ok.txt) Config: Ubuntu Server 11.04 x64 MonetDB v11.7.9 (Dec2011-SP2), MonetDB Database Server v1.6 (Dec2011-SP2), MonetDB Database Server Toolkit v1.0 (Dec2011-SP2) Thank you very much for your help!! -- *Matthieu Guamis* *Logo Axège <http://www.axege.com/>* /Axège// 23,rue Saint-Simon 63000 Clermont-Ferrand/ Tél: +33 (0)4 63 05 95 40 Fax: +33 (0)4.73.70.65.29 Email: matthieu.guamis(a)axege.com <mailto:matthieu.guamis@axege.com>

9 years, 8 months

[MonetDB-users] COPY INTO: insufficient privileges: binary COPY INTO requires administrator rights

by 万萌

Hi guys, i got the following error when COPY INTO TABLE test: sql>COPY INTO test from ('/home/svom/log3'); COPY INTO: insufficient privileges: binary COPY INTO requires administrator rights the file "log3" is in mode 777, so i dont know why this happend, thanks for help! Wan meng SVOM, NAOC

10 years

[MonetDB-users] MonetDB Python client

by Alexey Mints

Hello! I had a problem with Python client hanging up if MonetDB connection was closed unexpectedly. This happened for example, when I have passed a request that caused "ERROR = !Connection terminated" in mclient. Such things should not occur in a ideal life, but they do =( The problem was that socket.recv cannot handle this situation properly. The solution is to use select.select statement. I attach the diff to mapi2.py file that solves the problem. Regards, Alexey

10 years

[MonetDB-users] importing data from csv

by Flavio Coelho

I am trying to import data from a CSV file separated by ';' and with string values surrounded by double quotes: " but I keep getting the following error messages: missing separator ';' line 225387 expecting 8 got 7 fields missing separator ';' line 75129 expecting 8 got 7 fields missing separator ';' line 0 expecting 8 got 7 fields missing separator ';' line 450774 expecting 8 got 7 fields missing separator ';' line 300516 expecting 8 got 7 fields missing separator ';' line 150258 expecting 8 got 7 fields missing separator ';' line 525903 expecting 8 got 7 fields missing separator ';' line 375645 expecting 8 got 7 fields failed to import table I am using the following command: copy 300000000 offset 2 records into mobilidade.ligacoes from '/Users/fccoelho/Documents/dados_mobilidade/bilhetagem.txt' using delimiters ';','\n','"' ; after creating the table like this: sql dump of my table: CREATE TABLE "mobilidade"."ligacoes" ( "id" INTEGER NOT NULL DEFAULT next value for "mobilidade"."seq_5840", "DT_INI_ATIVIDADE_REDE" VARCHAR(10), "HR_INI_ATIVIDADE_REDE" VARCHAR(7), "NU_SETOR_CELULA_ORIG" INTEGER, "NU_AREA_CELULA_ORIG" INTEGER, "NU_DDD_ORIG" INTEGER, "NU_DDD_DESTINO" INTEGER, "CD_TELEFONE_ORIG_ALTER" INTEGER DEFAULT NULL, "CD_TELEFONE_DESTINO_ALTER" VARCHAR(16) DEFAULT NULL, CONSTRAINT "ligacoes_id_pkey" PRIMARY KEY ("id") ); Naturally, I checked line 0 and it does have 8 fields. What am I doing wrong? -- Flávio Codeço Coelho

10 years

[MonetDB-users] COPY INTO

by DADOU

COPY INTO test from '/tmp/bla'; COPY INTO: insufficient privileges: COPY INTO from file(s) requires administrator rights, use 'COPY INTO "test" FROM STDIN' instead can you help me -- View this message in context: http://old.nabble.com/COPY-INTO-tp34088907p34088907.html Sent from the monetdb-users mailing list archive at Nabble.com.

10 years

[MonetDB-users] COPY INTO

by DADOU

I have a Problem with copy into copy into "LINEORDER" from ('E:\TEST.tbl'); COPY INTO: insufficient privileges: binary COPY INTO requires administrator rights TEST.tbl is a file with 6 000 000 rows. these a format for two rows in file TEST.tbl 1|1|7381|155190|4137|19960102|TRUCK| 1|2|7381|67310|815|19960102|MAIL| can help me please -- View this message in context: http://old.nabble.com/COPY-INTO-tp34088847p34088847.html Sent from the monetdb-users mailing list archive at Nabble.com.

10 years, 1 month

[MonetDB-users] Basic sum aggregation returning incorrect value

by Tapomay Dey

I performed the following operations sequentially. Please look at the result when I use SUM. Sum values are incorrect. They should be 60/600/6000/60000.AVG returns correct values though. What could be going wrong? Is it something I can fix? sql> sql>create table table1 (id int, value int); operation successful (64.427ms) sql>create table table2 (id int, value int); operation successful (78.679ms) sql>insert into table1 values(1, 10); 1 affected row (25.389ms) sql>insert into table1 values(1, 20); 1 affected row (28.254ms) sql>insert into table1 values(1, 30); 1 affected row (27.512ms) sql>insert into table1 values(2, 300); 1 affected row (28.255ms) sql>insert into table1 values(2, 200); 1 affected row (28.658ms) sql>insert into table1 values(2, 100); 1 affected row (29.606ms) sql>insert into table2 values(1, 1000); 1 affected row (25.144ms) sql>insert into table2 values(1, 2000); 1 affected row (26.130ms) sql>insert into table2 values(1, 3000); 1 affected row (27.490ms) sql>insert into table2 values(2, 30000); 1 affected row (18.623ms) sql>insert into table2 values(2, 20000); 1 affected row (40.441ms) sql>insert into table2 values(2, 20000); 1 affected row (29.716ms) sql>insert into table2 values(2, 10000); 1 affected row (25.907ms) sql>select t1.id, sum(t1.value) as v1, sum(t2.value) as v2 from table1 t1, table2 t2 where t1.id=t2.id group by t1.id; +------+------+--------+ | id | v1 | v2 | +======+======+========+ | 1 | 180 | 18000 | | 2 | 2400 | 240000 | +------+------+--------+ 2 tuples (1.769ms) sql>select t1.id, avg(t1.value) as v1, avg(t2.value) as v2 from table1 t1, table2 t2 where t1.id=t2.id group by t1.id; +------+--------------------------+--------------------------+ | id | v1 | v2 | +======+==========================+==========================+ | 1 | 20 | 2000 | | 2 | 200 | 20000 | +------+--------------------------+--------------------------+ 2 tuples (2.275ms) sql> Regards, Tapomay.

10 years, 1 month

[MonetDB-users] last inserted id

by Tomas Mikula

Hi, how can I get the ID (value of an AUTO_INCREMENT column) of the last inserted row, from within an SQL script? I am able to get it through the Python API, so I suppose there is a way to get it inside an SQL script, too. Thanks, Tomas

10 years, 1 month

[MonetDB-users] strange decrease in performance after some deletes

by Majid Azimi

Hi guys, I have loaded 80 million records in database. a very complex aggregate query(multiple COUNT, SUM, AVG, GROUP BY combined) runs deadly fast in about 30 seconds. But after deleting 700 records performance has been decreased alot. the same query runs in 8 minutes now. Is this usual? Thank you

10 years, 1 month

[MonetDB-users] Changing SysV parameteres

by Majid Azimi

Hi guys, Does changing SysV parameters such as SHMMAX, SHMMIN, SHMSEG , ... affects MonetDB performance? How they affect? I mean the vanilla installation of MonetDB on a very powerful hardware needs massive optimization like what we see on other relational DBMS products?

10 years, 1 month

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

users-list June 2012