users-list November 2019

users-list@monetdb.org

9 participants
4 discussions

by Kalaiarasi Gowtham

Hi, While trying to use Monet/Python Loader function,we are getting the following error *pymonetdb.exceptions.OperationalError: 'pyapi.eval_loader' undefined in: pyapi.eval_loader(0x7f34f01f2c60:ptr, "{_emit.emit( { 'event_date': '2019-10-10', 'status': 1})};":str);* *Followed the below steps for installation* *Python installation steps:* We have installed python 3.6 1)yum install autoconf 2)yum install automake 3)yum install libtool 4)yum install openssl 5)yum install openssl-devel 6)yum install python-devel 7)Configured environment path settings for python export PYTHONPATH=$PATH:/usr/local/lib/python3.6/site-packages *Monet installation steps:* 1)git clone https://github.com/MonetDB/MonetDB.git 2)cd /MonetDB 3)./bootstrap 4)cd.. 5)mkdir testdir 6)cd testdir 7)../MonetDB/configure --enable-pyintegration=yes While configuring we get the following *py3integration is enabled* 8)make 9)make install 10)monetdbd create /path/to/mydbfarm 11)monetdbd start /path/to/mydbfarm 12)monet stop -a 13)monet set embedpy=true 14)monet start -a 13)Enabled numpy(pip install numpy) *Created a loader by using the below python script* import pymonetdb import sys import os connection = pymonetdb.connect(username="xyz", password="xyz123", hostname="ipaddress", database="testdb") cursor = connection.cursor() cursor.execute("CREATE LOADER myloader() LANGUAGE PYTHON {_emit.emit( { 'event_date': '2019-10-10', 'status': 1})};") #create loader cursor.execute("COPY LOADER INTO store FROM myloader();") #append the row from loader to table connection.commit() *While running the above script getting the following error* pymonetdb.exceptions.OperationalError: 'pyapi.eval_loader' undefined in: pyapi.eval_loader(0x7f34f01f2c60:ptr, "{_emit.emit( { 'event_date': '2019-10-10', 'status':1})};":str); Please, help us to fix this issue. Thanks & Regards Kalaiarasi A

2 years, 8 months

Announcement: New Nov2019 Feature release of MonetDB suite

by Sjoerd Mullender

The MonetDB team at CWI/MonetDB BV is pleased to announce the Nov2019 feature release of the MonetDB suite of programs. More information about MonetDB can be found on our website at <https://www.monetdb.org/>. For details on this release, please see the release notes at <https://www.monetdb.org/Downloads/ReleaseNotes>. As usual, the download location is <https://dev.monetdb.org/downloads/>. Nov 2019 feature release (11.35.3) MonetDB5 Server * The server "console" has been removed, as has the --daemon option. The server now doesn't read from standard input anymore. The way to stop a server is by sending it a TERM signal (on Linux/Unix) or by sending it an interrupt signal (usually control-C -- on all systems). * Implemented a function bat.diffcand to calculate difference of two candidate lists. * The mtime module was completely rewritten, the atom types date, daytime, and timestamp were changed. Upgrade code for BATs containing these types has been implemented. The old daytime type used a 32 bit integer to record milliseconds since the start of the day. The new daytime type uses a 64 bit integer to record microseconds since the start of the day. The old date type recorded days since or before the year 1. The new daytime type records the day of the month and the number of months since the year -4712 separately in a single 32 bit integer of which only 26 bits are used. Dates now use the proleptic Gregorian calendar, meaning the normal Gregorian callendar has been extended backward, and before the year 1, we have the year 0. Both the old and new timestamp types are a combination of a daytime and a date value, but since those types have changed, the timestamp type has also changed. The new date type has a smaller range than the old. The new date range is from the year -4712 to the year 170049. During conversion of date and timestamp columns, the dates are clamped to this range. * The tzone and rule atom types have been removed. They were not used by any code, and they were defined in a non-portable way. * Added "mapi_ipv6" property to monet_options to force ipv6 address binding only. This property is inherited while forking from monetdbd if it is also set there. * Removed (bat)calc.between_symmetric and changed (bat)calc.between by adding a number of extra arguments, all of type :bit: symmetric, low inclusive, high inclusive, nils false. Merovingian * Added "vmmaxsize" and "memmaxsize" mserver5 options to the daemon in order to set mserver5's maximum virtual and committed memory respectively. * Added ipv6 property to monetdbd properties to force IPv6 addresses binding only. By default this property is false to allow IPv4 addresses as well. MonetDB Common * BATcalcbetween and all its variants now have an extra bool parameter "anti" to invert the test. * All forms of BATcalcbetween and VARcalcbetween have extra bool arguments for low inclusive, high inclusive and nils false. The latter causes the result to be false instead of nil if the value being checked is nil. SQL Frontend * Removed functions json.text(string) returns string and json.text(int) returns string. Their MAL implementation didn't exist, so they were meaningless. * There are new aggregate functions sys.median_avg and sys.quantile_avg that return the interpolated value if the median/quantile doesn't fall exactly on a particular row. These functions always return a value of type DOUBLE and only work for numeric types (various width integers and floating point). * Added sys.deltas ("schema" string, "table" string, "column" string) returns table ("values" bigint) system function which returns a single column with 6 values: a flag indicating if the column's upper table is cleared or not, the count of the RDONLY, RD_INS and RD_UPD_ID deltas of the column itself, the number of deleted values of the column's table, as well as the level of the current transaction in the transaction level tree. It should be used for debugging purposes only. * Added "VALUES row_list" statement as a top SQL projection statement. * The implementation of in-expression now follows a join-based approach instead of using iterative union/selects. This greatly improves performance for large in-value-lists. Furthermore the old approach has large in-value-lists generate large MAL plans. This is now no longer the case. * Strings are now limited to 1GB, double-quoted tokens are limited to 2kB. These sizes are bytes of (UTF-8 encoded) input data. Bug Fixes * 3533: SQL aggregate functions avg(), sum() and median() return an error when used on a column with datatype interval second * 6134: Query produces error: HEAPalloc: Insufficient space for HEAP of 1168033427456 bytes. * 6613: LATERAL crash /.../rel_bin.c:1473: rel2bin_table: Assertion `0' failed. * 6683: Bug in subselect * 6686: Bug in subselect (count function) * 6688: Bug in subselect (or condition) * 6689: Trying to improve the performance of SQL queries with a large list of members in IN clause. * 6695: timestamp transformation * 6700: Monetdb Bugs in Subselect statements: * 6722: window functions issues * 6740: while upgrading the database from verison (MonetDB-11.27.13) to (MonetDB-11.33.3) we are unable to bring up the database * 6754: in mclient a strang msg is reported after issueing command: set schema sys; * 6755: Assertion failure in rel_bin.c * 6756: Error in optimizer garbageCollector on merge tables select * 6757: Double free or corruption (out) * 6758: SIGSEGV in __strcmp_sse2_unaligned() * 6759: Python JSON loader creates invalid data type for strings * 6761: Error: Program contains errors.:(NONE).multiplex * 6762: mserver5 crashes on (re-)start * 6764: mserver5 crashes with corruption, double free, invalid size or invalid pointer * 6766: Missing bulk implementation for get_value and next_value calls * 6769: ProfilerStart is not threadsafe * 6771: R-devel * 6772: TRACE includes information from all active sessions * 6773: json.filter returns corrupted string when selecting JSON null value * 6774: PROD aggregation gives wrong result * 6775: NOT IN with an AND containing an OR gives wrong result * 6776: Creating a table with a full outer join query gives type with wrong digits on the joined key. * 6779: Using Windows Messages translation for errno error codes. * 6780: Wrong value of the rank function * 6781: Insert after index creation crash * 6783: AVG changes scale of its results * 6784: function sys.isauuid(string) should return false if string value cannot be converted to a UUID

2 years, 8 months

UDF bulk implementation with some arguments as scalars

by Roberto Cornacchia

Hi, When defining a UDF, e.g. CREATE FUNCTION myfunc(a int, b int) RETURNS int we can have: - a scalar implementation command myfunc(a:int, b:int) :int - a bulk implementation command myfunc(a:bat[:int], b:bat[:int]) :bat[:int] - a mixed implementation (e.g. a is a column, b is scalar): command myfunc(a:bat[:int], b:int) :bat[:int] I'm trying to understand what the restrictions are, if any, for the third case. Is it expected to be allowed in (a combination of) the following cases? - Interleaved bulk/scalar arguments command myfunc(a:bat[:int], b:int, c:bat[:int]) :bat[:int] - Table-returning function command myfunc(a:bat[:int], b:int) (:bat[:int], :bat[:int]) - Table-returning function, used with a sub-select in input SELECT * FROM myfunc( (SELECT a, 10 FROM mytable) ); In particular I can't get the last one to work. Even though the sub-select has a constant as the second column, this constant seems to be first replicated into a full column and then given to myfunc() as a column. When the constant parameters are a few, that becomes a waste of time/space. Plus the annoyance of selecting the first value of a constant column for the actual internal implementation, which of course expects a scalar for those arguments. Am I missing something? Is there a way to make that work as I would expect? Thanks, Roberto

2 years, 8 months

Report a bug

by lhy1149

Hello, I don't know if it's appropriate, but because I can't register monetdb's bugzilla, I'm going to report a bug here. This problem causes the insert statement to hang and mserver5 takes up 100% of the CPU time. Here is the code that caused the problem: Create table a(a int, b int, id bigserial); -- comment any of the next 2 lines will resolve the problem Create ordered index a_pk on a(id); Create index a_idx1 on a(a); Insert into a(a) values(1); -- this line will hang Enviroment: macOS Catalina 10.15 MonetDB v11.33.11 I hope the development team can check if there is this problem. Thank you

2 years, 8 months

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

users-list November 2019