users-list June 2021

users-list@monetdb.org

7 participants
7 discussions

by Arjen P. de Vries

Hi all, Not really a bug report because I did not manage to figure out the cause. However, after upgrading from FC31 to FC32 I could not login any more, due to SELinux problems. Auto-relabeling did not work, nothing really... ... until I did dnf uninstall MonetDB-selinux. I came to this point because trying to give systemd services the correct labels with restorecon failed with an error referencing a monetdb specific file. I do not have the details unfortunately, but if you get problems, beware that MonetDB SELinux package and systemd may interfere in some way beyond my knowledge of these services. Best regards, Arjen PS: Some output from logs: sudo ausearch -c monetdb -m AVC,SELINUX_ERR [..] ---- time->Sat May 2 20:57:01 2020 type=AVC msg=audit(1588445821.693:203): avc: denied { open } for pid=1232 comm="monetdbd" path="/etc/resolv.conf" dev="dm-0" ino=3409775 scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:default_t:s0 tclass=file permissive=1 ---- time->Sat May 2 21:12:56 2020 type=AVC msg=audit(1588446776.043:1194): avc: denied { execute } for pid=2861 comm="(monetdbd)" name="monetdbd" dev="dm-0" ino=2147256 scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file permissive=1 trawcon="unconfined_u:object_r:monetdbd_exec_t:s0" ---- time->Sat May 2 21:12:56 2020 type=AVC msg=audit(1588446776.043:1195): avc: denied { execute_no_trans } for pid=2861 comm="(monetdbd)" path="/usr/bin/monetdbd" dev="dm-0" ino=2147256 scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file permissive=1 trawcon="unconfined_u:object_r:monetdbd_exec_t:s0" ---- time->Sat May 2 21:12:56 2020 type=AVC msg=audit(1588446776.044:1196): avc: denied { map } for pid=2861 comm="monetdbd" path="/usr/bin/monetdbd" dev="dm-0" ino=2147256 scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file permissive=1 trawcon="unconfined_u:object_r:monetdbd_exec_t:s0" ---- time->Sat May 2 21:12:56 2020 type=AVC msg=audit(1588446776.714:1197): avc: denied { remove_name } for pid=1232 comm="monetdbd" name="merovingian.pid" dev="tmpfs" ino=34369 scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=dir permissive=1 trawcon="system_u:object_r:monetdbd_var_run_t:s0" ---- time->Sat May 2 21:12:56 2020 type=AVC msg=audit(1588446776.714:1198): avc: denied { unlink } for pid=1232 comm="monetdbd" name="merovingian.pid" dev="tmpfs" ino=34369 scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file permissive=1 ---- time->Sat May 2 21:12:56 2020 type=AVC msg=audit(1588446776.714:1199): avc: denied { write } for pid=1232 comm="monetdbd" name=".merovingian_lock" dev="dm-0" ino=5899443 scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file permissive=1 trawcon="system_u:object_r:monetdbd_lock_t:s0" ---- time->Sat May 2 21:13:15 2020 type=AVC msg=audit(1588446795.214:1209): avc: denied { read } for pid=2925 comm="(monetdbd)" name="passwd" dev="dm-0" ino=524514 scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:var_t:s0 tclass=file permissive=1 ---- time->Sat May 2 21:13:15 2020 type=AVC msg=audit(1588446795.214:1210): avc: denied { open } for pid=2925 comm="(monetdbd)" path="/var/lib/sss/mc/passwd" dev="dm-0" ino=524514 scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:var_t:s0 tclass=file permissive=1 ---- time->Sat May 2 21:13:15 2020 type=AVC msg=audit(1588446795.214:1211): avc: denied { map } for pid=2925 comm="(monetdbd)" path="/var/lib/sss/mc/passwd" dev="dm-0" ino=524514 scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:var_t:s0 tclass=file permissive=1 ---- time->Sat May 2 21:14:24 2020 type=AVC msg=audit(1588446864.487:1281): avc: denied { read } for pid=3072 comm="(monetdbd)" name="passwd" dev="dm-0" ino=524514 scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:var_t:s0 tclass=file permissive=1 ---- time->Sat May 2 21:14:24 2020 type=AVC msg=audit(1588446864.487:1282): avc: denied { open } for pid=3072 comm="(monetdbd)" path="/var/lib/sss/mc/passwd" dev="dm-0" ino=524514 scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:var_t:s0 tclass=file permissive=1 ---- time->Sat May 2 21:14:24 2020 type=AVC msg=audit(1588446864.487:1283): avc: denied { map } for pid=3072 comm="(monetdbd)" path="/var/lib/sss/mc/passwd" dev="dm-0" ino=524514 scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:var_t:s0 tclass=file permissive=1 -- ==================================================================== ICIS, office M1.00.05 Radboud University Mercator 1 Faculty of Science Toernooiveld 212 arjen(a)cs.ru.nl NL-6525 EC Nijmegen, The Netherlands +31-(0)24-365 2354 ===================== http://www.informagus.nl/ ==================== -- ==================================================================== ICIS, office M1.00.05 Radboud University Mercator 1 Faculty of Science Toernooiveld 212 arjen(a)cs.ru.nl NL-6525 EC Nijmegen, The Netherlands +31-(0)24-365 2354 ===================== http://www.informagus.nl/ ====================

7 months

Unable to connect to Monetdb after Upgrade : Challenge string is not valid, it is empty

by AUBERT Simon

Hello all I had an old monetdb installation on my Windows laptop -Did the upgrade in Oct2020-SP5 Bugfix Release (11.39.17) It really looks like I can start the server C:\Program Files\MonetDB\MonetDB5\bin>mserver5 # MonetDB 5 server v11.39.17 (Oct2020-SP5) # Serving database 'demo', using 4 threads # Compiled for amd64-pc-windows-msvc/64bit # Found 15.862 GiB available main-memory of which we use 12.928 GiB # Copyright (c) 1993 - July 2008 CWI. # Copyright (c) August 2008 - 2021 MonetDB B.V., all rights reserved # Visit https://www.monetdb.org/ for further information # Listening for connection requests on mapi:monetdb://localhost:50000/ # MonetDB/GIS module loaded # MonetDB/SQL module loaded # MonetDB server is started. To stop server press Ctrl-C. But when I want to use the mclient, it asks the user and password and then... nothing and after a few minutes I have this message Challenge string is not valid, it is empty Same thing with an ODBC connection : after a long time, it fails. I have reinstalled Monetdb but nothing changed. I have no idea of what's wrong. Best regards, Simon Simon AUBERT [cid:image001.jpg@01D76D97.13BF6A80] Bourse Maritime - 1, place Lainé - 33 000 Bordeaux Mob : +33(0)6 63 83 76 29

1 year

Re: Re: RSS memory

by Vojtech Kurka

Hello Roberto, Sjoerd, I think I am hitting the same problem described in this thread. I am running MonetDB in Docker container on a server with 30GB of memory, allocating 21GB of it to MonetDB container. Sometimes, under heavier load I get OOM - process mserver5 uses more memory than is the memory.limit_in_bytes for a cgroup of this container. It looks like this: [Mon Jun 21 12:56:36 2021] memory: usage 20971484kB, limit 20971520kB, failcnt 48706810 [Mon Jun 21 12:56:36 2021] memory+swap: usage 0kB, limit 9007199254740988kB, failcnt 0 [Mon Jun 21 12:56:36 2021] kmem: usage 143696kB, limit 9007199254740988kB, failcnt 0 .. [Mon Jun 21 12:56:36 2021] Tasks state (memory values in pages): [Mon Jun 21 12:56:36 2021] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [Mon Jun 21 12:56:36 2021] [ 26927] 0 26927 1371 151 53248 0 0 run.sh [Mon Jun 21 12:56:36 2021] [ 27054] 0 27054 134763 326 139264 0 0 monetdbd [Mon Jun 21 12:56:36 2021] [ 27068] 0 27068 40646510 5209845 64159744 0 0 mserver5 [Mon Jun 21 12:56:36 2021] [ 27289] 0 27289 1019 69 45056 0 0 tail [Mon Jun 21 12:56:36 2021] [ 29503] 0 29503 6425 547 86016 0 0 mclient [Mon Jun 21 12:56:36 2021] oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=d37591a31d4d010a7b4e34ebb19ba11220a84a264733f70a00ee92899c7918fc,mems_allowed=0,oom_memcg=/docker/d37591a31d4d010a7b4e34ebb19ba11220a84a264733f70a00ee92899c7918fc,task_memcg=/docker/d37591a31d4d010a7b4e34ebb19ba11220a84a264733f70a00ee92899c7918fc,task=mserver5,pid=27068,uid=0 [Mon Jun 21 12:56:36 2021] Memory cgroup out of memory: Killed process 27068 (mserver5) total-vm:162586040kB, anon-rss:20823812kB, file-rss:15568kB, shmem-rss:0kB, UID:0 pgtables:62656kB oom_score_adj:0 Have you made any progress in resolving the OOMs? Things I am trying to test are (I am using docker-compose v2 for running the container, so settings are from there): mem_limit: 21G # this sets memory.limit_in_bytes for the cgroup of the container mem_reservation: 14G # this sets memory.soft_limit_in_bytes oom_score_adj: -1000 # to lower the chance OOM will kill this process Exploring "select * from env();" in MonetDB tells me that the gdk_mem_maxsize is 12251394211, which corresponds to the 81.5% of the mem_reservation, or memory.soft_limit_in_bytes. I am monitoring memory usage of the container and it still is spiking at 19G, which is uncomfortably high, very close to the hard limit of the cgroup. Is this normal? FYI, originally meant only for PostgreSQL, but it became our standard - we are using vm.overcommit = 2 and vm.overcommit_ration = 100, if it changes anything. I would be thankful for any clue on how to run MonetDB in the container reliably as right now it is causing me quite a headache. Running it on a host solely for the MonetDB would be a last resort for me. Thank you and have a great day!

1 year, 1 month

New MonetDB JDBC driver released

by Sjoerd Mullender

The MonetDB team at MonetDB BV is pleased to announce a bugfix release of the MonetDB Java jars and JDBC driver. We recommend all users of MonetDB JDBC driver to upgrade to version: monetdb-jdbc-3.1.jre8.jar The download location is as usual: <https://www.monetdb.org/downloads/Java/>. For a list of changes please read: <https://www.monetdb.org/downloads/Java/ChangeLog>. For information and tips on using the MonetDB JDBC driver, see: <https://www.monetdb.org/downloads/Java/release.txt>. Note: as of release monetdb-jdbc-3.0.jre8.jar the MonetDB JDBC Driver class name has been changed to: org.monetdb.jdbc.MonetDriver It used to be nl.cwi.monetdb.jdbc.MonetDriver which is now marked as deprecated. Please use the new driver class name asap in your configuration files/scripts and/or Java code. The old driver class name is still included in the jar file to ease the transition for existing deployments. However it will be removed in a future release of the MonetDB JDBC driver. -- Sjoerd Mullender

1 year, 1 month

DBSeeder Version 2.9.1

by Walter Weinmann

The latest version of DBSeeder is now available on GitHub: https://github.com/KonnexionsGmbH/db_seeder. This also includes from MonetDB the latest available versions on Docker Hub (Oct2020-SP5) and https://www.monetdb.org/downloads/Java (JDBC monet-jdbc-3.0.jre8). MonetDB now shows a huge performance improvement after a manual commit (instead of autocommit) is executed in DBSeeder after each batch of insert operations. [image: Company_9.9.9_win10.png]

1 year, 1 month

Linux kernel vm settings

by Roberto Cornacchia

Hi there, I looked around but couldn't find any recommendation about kernel vm settings in Linux for MonetDB. In particular: - vm.overcommit_memory: 0 (default) : a heuristics decides whether overcommitting is allowed 1: no check, overcommit is always allowed 2: overcommitting is regulated by vm.overcommit_ratio (default = 50%) Do I understand correctly that using vm.overcommit_memory=1 will only make the OOM kill mserver5 when the total VM available is exhausted? If that is true, should it be reasonably safe to use on a server that is mainly intended for MonetDB, as long as sufficient disk space is available? - vm.swappiness Generic recommendations are usually 60 for a desktop and 30 for a server. Oracle recommends 10. Redis recommends 1. Are there studies / recommendations for MonetDB?

1 year, 1 month

DBSeeder Version 2.9.0

by Walter Weinmann

We are happy to announce that version 2.9.0 of DBSeeder ( https://github.com/KonnexionsGmbH/db_seeder) is now available for general use. DBSeeder also supports the latest version of MonetDB available on DockerHub (Oct2020-SP5).

1 year, 1 month

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

users-list June 2021