Hi All
I installed pymonetdb v 1.2.1 and trying to connect to remote server MonetDB v11.35.3 (Nov2019) listening on IPV6 only. I have got next error in python client:
>>> mapi_connection.connect(hostname="<server_name>", port=xxxxx, username="monetdb", password="monetdb", database="demo", language="sql", unix_socket=None, connect_timeout=-1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.7/site-packages/pymonetdb/mapi.py", line 143, in connect
self.socket.connect((hostname, port))
socket.gaierror: [Errno -2] Name or service not known
>>>
My pc client is also using ipv6 to connect to MonetDB server. I also have jdbc connection that works fine.
Question:
Does it mean pymonetdb does not support ipv6 only environment?
What would be the reason if it has nothing to do with protocol?
Thanks
Alex
Sometimes the MonetDB server just shuts down. I want to find at least some information about what happens before the shutdown. Please tell me how you can turn on logs on Windows or in some other way see what happens before shutting down.
MonetDB 5 server 11.35.9 (Nov2019-SP1) (64-bit)
Copyright (c) 1993 - July 2008 CWI
Copyright (c) August 2008 - 2019 MonetDB B.V., all rights reserved
Visit https://www.monetdb.org/ for further information
Found 32.0GiB available memory, 8 available cpu cores
Libraries:
libpcre: 8.41 2017-07-05
openssl: OpenSSL 1.1.1d 10 Sep 2019
libxml2: 2.9.9
Compiled by: monet@WIN2K12R2 (x86_64-pc-winnt)
Compilation: cl -GF -W3 -WX -MD -nologo -Ox -Zi -Oi
Linking : cl -GF -W3 -WX -MD -nologo -Ox -Zi -Oi
Thank you Ying and Jenin. I am going to upgrade the MonetDB as you suggested and then go from there. Can you please let me know where can I find the upgrade procedure.
Thank you for your help in advance.
Suresh Devarakonda
84.51°
TDO - Database Support Team
D: 513.632.3692
M: 859.512.1839
A: 100 W. 5th Street, Cincinnati, Ohio 45202
Cincinnati | Chicago | New York | Portland
Facebook LinkedIn Instagram Twitter
Please consider the environment before printing this email.
-----Original Message-----
From: users-list <users-list-bounces+suresh.devarakonda=8451.com(a)monetdb.org> On Behalf Of users-list-request(a)monetdb.org
Sent: Friday, January 17, 2020 6:00 AM
To: users-list(a)monetdb.org
Subject: users-list Digest, Vol 89, Issue 6
Send users-list mailing list submissions to
users-list(a)monetdb.org
To subscribe or unsubscribe via the World Wide Web, visit
https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.monet…
or, via email, send a message with subject or body 'help' to
users-list-request(a)monetdb.org
You can reach the person managing the list at
users-list-owner(a)monetdb.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of users-list digest..."
Today's Topics:
1. Re: MonetDB crashing due to out of memory - monetdbd[2247]
1.7 (Aug2018-SP2) (Ying Zhang)
----------------------------------------------------------------------
Message: 1
Date: Thu, 16 Jan 2020 12:01:27 +0100
From: Ying Zhang <Y.Zhang(a)cwi.nl>
To: Communication channel for MonetDB users <users-list(a)monetdb.org>
Subject: Re: MonetDB crashing due to out of memory - monetdbd[2247]
1.7 (Aug2018-SP2)
Message-ID: <E6A3FCCA-8023-4EBD-AF7A-CF89F615B2D8(a)cwi.nl>
Content-Type: text/plain; charset=utf-8
Hai Suresh,
First of all, I?d recommend you to upgrade your MonetDB installation to the most recent one, i.e. Nov2019-SP1. There are many improvements related to your problem since Aug2018-SP2. Also, the version you?re using is no longer supported by the open-source team.
Please let us know if upgrade helps. But below, a bit more info. inlined.
> On 15 Jan 2020, at 23:44, Suresh Devarakonda <Suresh.Devarakonda(a)8451.com> wrote:
>
> Hi,
>
> The Moentdb is crashing every couple of weeks due to out of memory in our environment.
Did it really crash, or did MonetDB give error messages and stop executing the query?
Crash always indicates a problem in MonetDB, but if it?s the second case, it?s expected.
> What I have noticed that MonetDB is eating memory and not releasing it at all once we brought up the MonetDB server. I am not sure if this is how MonetDB works or is it a bug.
MonetDB is supposed to free up the memory when it no longer use it, which often happens towards the end of a query execution.
For further investigation, we need more information after you have upgraded to Nov2019-SP1. In general, we?ll need information with which we can reproduce the problem, e.g. data + queries.
> This is running on monetdbd[2247] 1.7 (Aug2018-SP2) version. Also wondering is there way to limit the memory usage of MonetDB.
Please beware that depending on your data sizes and (concurrent) queries, MonetDB might run faster into out-of-memory problem, if you limit the amount of memory it may use,
How about putting MonetDB in a VM?
There are also some undocumented kernel configurations for this, which we don?t recommend to our users lightly?
>
> Can you guys help me to resolve this issue.
If you need professional support, please contact the supporting company MonetDB Solutions.
Regards,
Jennie
>
> [root@xxx ~]# monetdb status
> name state health remarks
> INVPRDB R 6h 67% 3d mapi:monetdb://xxxxx:50000/INVPRDB
> FSPPRDB R 6h 50% 6w mapi:monetdb://xxxxx:50000/FSPPRDB
> [root@xxxxx ~]# free -g --- Current utilization
> total used free shared buff/cache available
> Mem: 31 0 22 0 7 30
> Swap: 3 0 3
>
>
> <image003.png>
>
>
>
>
>
> <image002.jpg>
> Suresh Devarakonda
> 84.51?
> TDO - Database Support Team
> D: 513.632.3692
> M: 859.512.1839
> A: 100 W. 5th Street, Cincinnati, Ohio 45202
>
> Cincinnati | Chicago | New York | Portland
> Facebook LinkedIn Instagram Twitter
> Please consider the environment before printing this email.
>
>
> _______________________________________________
> users-list mailing list
> users-list(a)monetdb.org
> https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.
> monetdb.org%2Fmailman%2Flistinfo%2Fusers-list&data=02%7C01%7Csures
> h.devarakonda%408451.com%7Cae946815e409491cb83d08d79b3c9c2c%7C5f9dc6bd
> f38a454a864cc803691193c5%7C0%7C0%7C637148556952386018&sdata=3JmWyH
> MNaeMNrBp8vX%2FAYES%2By%2FDqNHqGmE%2FKX%2F%2F4m%2Bs%3D&reserved=0
------------------------------
Subject: Digest Footer
_______________________________________________
users-list mailing list
users-list(a)monetdb.org
https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.monet…
------------------------------
End of users-list Digest, Vol 89, Issue 6
*****************************************
That was it. I was missing python packages
Thank you Panos
------ Original Message ------
From: Panagiotis Koutsourakis
To: users-list(a)monetdb.org
Sent: January 21, 2020 at 7:54 AM
Subject: Re: embedded python in latest release
Hello Alex,
make sure that you have the MonetDB-python or MonetDB-python3 packages installed:
dnf install MonetDB-python
or
dnf install MonetDB-python3
Also you will need to notify the MonetDB server that you want to start with embedded python support. If you are starting the server using the MonetDB daemon, you need to set the relevant property for your database:
monetdb set embedpy=true <database>
or
monetdb set embedpy3=true <database>
If you are starting the server by hand, then
mserver5 --dbpath=/path/to/database --set embedded_py=2
or
mserver5 --dbpath=/path/to/database --set embedded_py=3
Best regards,
Panos.
On 1/21/20 5:11 AM, ALEX OSSIPOV wrote:
Roberto
I have all mentioned packages installed.
I do not have any issues with R though. Centos based instance also works fine.
Thanks
Alex
On January 20, 2020 at 3:49 AM Roberto Cornacchia <roberto.cornacchia(a)gmail.com> wrote:
Alex,
Do you have packages python3-devel python3-numpy (python2-devel python2-numpy for python 2) installed in your Fedora system?
Yes, you can import what you like in the body of your function, as long as the package is installed in your system.
Roberto
On Sun, 19 Jan 2020 at 00:12, ALEX OSSIPOV < alexo15(a)comcast.net mailto:alexo15@comcast.net > wrote:
I am having problem enabling embedded python in latest realease, MonetDB 5 server v11.35.9 (Nov2019-SP1)
My 64bit Fedora environment has multiple versions of both python 2 and 3 wih 32 and 64 bit libs.
MonetDB was installed following https://www.monetdb.org/downloads/Fedora/
with additional SELinux.
I tried random_integers function from documentation with both embedpy=true and embedpy3=true separately but getting the same type of error:
SQL Error [22000]: TypeException:user.s10_0[9]:'pyapi3.eval' undefined in: X_10:bat[:int] := pyapi3.eval(0x559cb0440270:ptr, "{ \n return numpy.random.randint(low, high, size=(amount,))\n}\n;":str, X_7:int, X_8:int, X_9:int);
Both py2 and py3 versions work fine on my computer.
How do I properly point to a specific python version to be used by MonetDB?
Are there any specific env variables that should be set prior to installation/db creation? I tried to define PYTHONPATH but it did not help.
Aside question: Can additional modules be imported in function body for PYTHON type functions or does it only work with numpy?
Thanks Alex
_______________________________________________
users-list mailing list
users-list(a)monetdb.org mailto:users-list@monetdb.org
https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________
users-list mailing list
users-list(a)monetdb.org
https://www.monetdb.org/mailman/listinfo/users-list
_______________________________________________
users-list mailing list
users-list(a)monetdb.org
https://www.monetdb.org/mailman/listinfo/users-list
I am having problem enabling embedded python in latest realease, MonetDB 5 server v11.35.9 (Nov2019-SP1)
My 64bit Fedora environment has multiple versions of both python 2 and 3 wih 32 and 64 bit libs.
MonetDB was installed following https://www.monetdb.org/downloads/Fedora/
with additional SELinux.
I tried random_integers function from documentation with both embedpy=true and embedpy3=true separately but getting the same type of error:
SQL Error [22000]: TypeException:user.s10_0[9]:'pyapi3.eval' undefined in: X_10:bat[:int] := pyapi3.eval(0x559cb0440270:ptr, "{ \n return numpy.random.randint(low, high, size=(amount,))\n}\n;":str, X_7:int, X_8:int, X_9:int);
Both py2 and py3 versions work fine on my computer.
How do I properly point to a specific python version to be used by MonetDB?
Are there any specific env variables that should be set prior to installation/db creation? I tried to define PYTHONPATH but it did not help.
Aside question: Can additional modules be imported in function body for PYTHON type functions or does it only work with numpy?
Thanks Alex
Hi,
The Moentdb is crashing every couple of weeks due to out of memory in our environment. What I have noticed that MonetDB is eating memory and not releasing it at all once we brought up the MonetDB server. I am not sure if this is how MonetDB works or is it a bug. This is running on monetdbd[2247] 1.7 (Aug2018-SP2) version. Also wondering is there way to limit the memory usage of MonetDB.
Can you guys help me to resolve this issue.
[root@xxx ~]# monetdb status
name state health remarks
INVPRDB R 6h 67% 3d mapi:monetdb://xxxxx:50000/INVPRDB
FSPPRDB R 6h 50% 6w mapi:monetdb://xxxxx:50000/FSPPRDB
[root@xxxxx ~]# free -g --- Current utilization
total used free shared buff/cache available
Mem: 31 0 22 0 7 30
Swap: 3 0 3
[cid:image003.png@01D5CBCA.6268DF10]
[cid:image002.jpg@01D3E7B3.57185110]
Suresh Devarakonda
84.51<http://www.8451.com/>°
TDO - Database Support Team
D: 513.632.3692
M: 859.512.1839
A: 100 W. 5th Street, Cincinnati, Ohio 45202
Cincinnati | Chicago | New York | Portland
Facebook<https://www.facebook.com/8451group/?ref=aymt_homepage_panel> LinkedIn<https://www.linkedin.com/company/84-51/?trk=biz-companies-cym> Instagram<https://www.instagram.com/84.51/> Twitter<https://twitter.com/8451group>
Please consider the environment before printing this email.
Hi,
Currently, we are trying to start the database created in monetdb using the
following command
monetdb start <dbname>
starting database 'dbname'... FAILED
start: starting '<dbname>' failed: database '<dbname>' has crashed after
starting, manual intervention needed, check monetdbd's logfile
(merovingian.log) for details
Getting logs as follows
[root@localhost mydbfarm]# tail -f merovingian.log
2020-01-14 14:56:20 MSG merovingian[19527]: database '<dbname>' (19757) has
exited with exit status 1
2020-01-14 14:56:20 ERR control[19527]: !monetdbd: an internal error has
occurred 'database '<dbname>' has crashed after starting, manual
intervention needed, check monetdbd's logfile (merovingian.log) for details'
2020-01-14 14:56:25 ERR merovingian[19527]: client error: database
'<dbname>' has crashed after starting, manual intervention needed, check
monetdbd's logfile (merovingian.log) for details
Please,help us to solve this issue.
Thanks
Kalai
Hi Jenni,
Please, find the below steps I have followed to install monetdb
*Installation of MonetDB in CentOS 7*
*Installing dependencies*
# yum update
# yum install python autoconf automake libtool bison openssl openssl-devel
gettext-devel git
# git clone https://github.com/MonetDB/MonetDB.git
# cd /MonetDB
# ./bootstrap
# cd ..
# mkdir testdir
# cd testdir
*To enable py3integration python version 3 should be installed*
# yum install centos-release-scl
# yum install rh-python36
# scl enable rh-python36 bash
*check python version*
# python -V
then
# ../MonetDB/configure –enable-py3integration=yes
# make
# make install
Thanks and regards,
Kalai
On Wed, 8 Jan 2020 at 16:31, <users-list-request(a)monetdb.org> wrote:
> Send users-list mailing list submissions to
> users-list(a)monetdb.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://www.monetdb.org/mailman/listinfo/users-list
> or, via email, send a message with subject or body 'help' to
> users-list-request(a)monetdb.org
>
> You can reach the person managing the list at
> users-list-owner(a)monetdb.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of users-list digest..."
>
>
> Today's Topics:
>
> 1. Re: pymonetdb issue (ferreira)
> 2. Re: Monetdb (Ying Zhang)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 7 Jan 2020 17:22:59 +0000
> From: ferreira <pedro.ferreira(a)monetdbsolutions.com>
> To: Communication channel for MonetDB users <users-list(a)monetdb.org>,
> ALEX OSSIPOV <alexo15(a)comcast.net>
> Subject: Re: pymonetdb issue
> Message-ID:
> <0e74a920-b2ad-2993-b1dc-fcd7a49105bb(a)monetdbsolutions.com>
> Content-Type: text/plain; charset="utf-8"; Format="flowed"
>
> Hello Alex,
>
> I just looked at pymonetdb source code and indeed it does not support
> IPv6 yet. I'm going to create an issue to request for it. The reason is
> because in the past, the MAPI protocol (which MonetDB uses for socket
> connections) didn't support IPv6, so the clients (Python, Ruby, JDBC,
> etc.) never had to implemented them. However they can now add this option.
>
> Best regards,
>
> Pedro
>
> On 12/22/19 5:30 AM, ALEX OSSIPOV wrote:
> > Hi All
> > I installed pymonetdb v 1.2.1 and trying to connect to remote
> > server?MonetDB v11.35.3 (Nov2019) listening on IPV6 only. I have got
> > next error in python client:
> > >>> mapi_connection.connect(hostname="<server_name>", port=xxxxx,
> > username="monetdb", password="monetdb", database="demo",
> > language="sql", unix_socket=None, connect_timeout=-1)
> > Traceback (most recent call last):
> > File "<stdin>", line 1, in <module>
> > File "/usr/local/lib/python3.7/site-packages/pymonetdb/mapi.py", line
> > 143, in connect
> > self.socket.connect((hostname, port))
> > socket.gaierror: [Errno -2] Name or service not known
> > >>>
> > My pc client is also using ipv6 to connect to MonetDB server. I also
> > have jdbc connection that works fine.
> > Question:
> > Does it mean pymonetdb does not support ipv6 only environment?
> > What would be the reason if it has nothing to do with protocol?
> >
> > Thanks
> > Alex
> >
> >
> > _______________________________________________
> > users-list mailing list
> > users-list(a)monetdb.org
> > https://www.monetdb.org/mailman/listinfo/users-list
>