Hi Sjoerd,

>LD_LIBRARY_PATH is for run time, -L is for compile time
thanks for clarifying, now it makes sense

Actually it look like libpy_LIBS (python2.7) doesn't get properly propagated into sql/backends/monet5/UDF/pyapi/Makefile (when generated by ./MonetDB-11.27.5/configure from configure.ac). 
And libpy3_LIBS (python3.6) does get properly propagated into sql/backends/monet5/UDF/pyapi3/Makefile.

for python2 (sql/backends/monet5/UDF/pyapi/Makefile):
libpy_LIBS = -lpython2.7 -lpthread -ldl -lutil -lm -Xlinker -export-dynamic

for python3 (sql/backends/monet5/UDF/pyapi3/Makefile):
libpy3_LIBS =  -L/home/SECDEV.LOCAL/akravchenko/anaconda3/lib -lpython3.6m -lpthread -ldl  -lutil -lrt -lm  -Xlinker -export-dynamic

Thanks,
Anton

On Mon, Sep 18, 2017 at 11:29 AM, Sjoerd Mullender <sjoerd@acm.org> wrote:
LD_LIBRARY_PATH and the -L option are two totally different things.
LD_LIBRARY_PATH is for run time, -L is for compile time.  At run time,
the system needs to find the shared library you linked against, but it
can be at a different location then when you compiled it, for instance
because you installed the library after compilation, therefore you need
to be able to specify locations for both.
You can also specify the (default) run time location at compile time.
That is done using either -R or -rpath (depending on the compiler/linker
you're using).

On 09/18/2017 06:33 PM, Anton Kravchenko wrote:
> Hi there,
>
> Here are some updates.
>
> 1) I tried to set LD_LIBRARY_PATH 
> export LD_LIBRARY_PATH=/home/SECDEV.LOCAL/akravchenko/anaconda2/lib
> but it did not help.
>
> 2) I tried to link "by hand" with added
> "-L/home/SECDEV.LOCAL/akravchenko/anaconda2/lib" option:
> [akravchenko@hnode1 pyapi]$ pwd
> /home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/sql/backends/monet5/UDF/pyapi
> [akravchenko@hnode1 pyapi]$ /bin/sh ../../../../../libtool  --tag=CC 
>  --mode=link gcc -DLIBPYAPI  -g -O2   -D_REENTRANT -module
> -avoid-version  -o lib_pyapi.la <http://lib_pyapi.la> -rpath
> /home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/lib/monetdb5
> lib_pyapi_la-pyapi.lo lib_pyapi_la-unicode.lo lib_pyapi_la-pytypes.lo
> lib_pyapi_la-type_conversion.lo lib_pyapi_la-formatinput.lo
> lib_pyapi_la-connection.lo lib_pyapi_la-pyloader.lo lib_pyapi_la-emit.lo
> lib_pyapi_la-conversion.lo  ../../../../../monetdb5/tools/libmonetdb5.la
> <http://libmonetdb5.la> ../../../../../gdk/libbat.la <http://libbat.la
> *-L/home/SECDEV.LOCAL/akravchenko/anaconda2/lib* -lpython2.7 -lpthread
> -ldl -lutil -lm -Xlinker -export-dynamic
> libtool: link: rm -fr  .libs/lib_pyapi.la <http://lib_pyapi.la>
> .libs/lib_pyapi.lai .libs/lib_pyapi.so .libs/lib_pyapi.soT
> libtool: link: gcc -shared  -fPIC -DPIC  .libs/lib_pyapi_la-pyapi.o
> .libs/lib_pyapi_la-unicode.o .libs/lib_pyapi_la-pytypes.o
> .libs/lib_pyapi_la-type_conversion.o .libs/lib_pyapi_la-formatinput.o
> .libs/lib_pyapi_la-connection.o .libs/lib_pyapi_la-pyloader.o
> .libs/lib_pyapi_la-emit.o .libs/lib_pyapi_la-conversion.o   -Wl,-rpath
> -Wl,/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/monetdb5/tools/.libs
> -Wl,-rpath
> -Wl,/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/gdk/.libs
> -Wl,-rpath
> -Wl,/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/clients/mapilib/.libs
> -Wl,-rpath
> -Wl,/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/common/stream/.libs
> -Wl,-rpath -Wl,/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/lib
> -L/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/gdk/.libs
> -L/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/clients/mapilib/.libs
> -L/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/common/stream/.libs
> ../../../../../monetdb5/tools/.libs/libmonetdb5.so
> /home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/gdk/.libs/libbat.so
> /home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/clients/mapilib/.libs/libmapi.so
> -lpcre -lxml2 ../../../../../gdk/.libs/libbat.so
> /home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/common/stream/.libs/libstream.so
> -lsnappy -lssl -lcrypto -lz
> -L/home/SECDEV.LOCAL/akravchenko/anaconda2/lib -lpython2.7 -lpthread
> -ldl -lutil -lm  -g -O2 -Wl,-export-dynamic   -pthread -Wl,-soname
> -Wl,lib_pyapi.so -o .libs/lib_pyapi.so
> libtool: link: ( cd ".libs" && rm -f "lib_pyapi.la
> <http://lib_pyapi.la>" && ln -s "../lib_pyapi.la <http://lib_pyapi.la>"
> "lib_pyapi.la <http://lib_pyapi.la>" )
>
> And it worked.
>
> 3) I added  -L/home/SECDEV.LOCAL/akravchenko/anaconda2/lib into Makefile
> at /home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/sql/backends/monet5/UDF/pyapi
> libpy_LIBS = -L/home/SECDEV.LOCAL/akravchenko/anaconda2/lib -lpython2.7
> -lpthread -ldl -lutil -lm -Xlinker -export-dynamic #AK added -L
>
> and make -j4 from the BUILD dir
> (/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD).
>
> This one worked too.
>
> 4) I tested newly installed m5server:
> [akravchenko@hnode1 ~]$ mserver5 --dbpath=/tmp/test --set embedded_py=2
> # MonetDB 5 server v11.27.5 "Jul2017-SP1"
> # Serving database 'test', using 4 threads
> # Compiled for x86_64-unknown-linux-gnu/64bit with 128bit integers
> # Found 31.485 GiB available main-memory.
> # Copyright (c) 1993-July 2008 CWI.
> # Copyright (c) August 2008-2017 MonetDB B.V., all rights reserved
> # Visit https://www.monetdb.org/ for further information
> # Listening for connection requests on mapi:monetdb://127.0.0.1:50000/
> <http://127.0.0.1:50000/>
> # MonetDB/SQL module loaded
> # MonetDB/Python2 module loaded
>
> [akravchenko@hnode1 ~]$ mclient
> Welcome to mclient, the MonetDB/SQL interactive terminal (Oct2014-SP2)
> Database: MonetDB v11.27.5 (Jul2017-SP1), 'test'
> Type \q to quit, \? for a list of available commands
> auto commit mode: on
> sql>create or replace function f1() returns table(v1 INT)
> LANGUAGE PYTHON {
>          return [1,2,3]
> };
> operation successful (2.355ms)
> sql>select * from f1();
> | v1   |
> +======+
> |    1 |
> |    2 |
> |    3 |
> +------+
> 3 tuples (2.390ms)
>
> p.s. I still don't understand why 
> export LD_LIBRARY_PATH=/home/SECDEV.LOCAL/akravchenko/anaconda2/lib
> did not work and had to pass
> -L/home/SECDEV.LOCAL/akravchenko/anaconda2/lib "by hand" into libtool linker
> to satisfy -lpython2.7.
>
> Thank you,
> Anton
>
> On Fri, Sep 15, 2017 at 4:17 PM, Anton Kravchenko
> <kravchenko.anton86@gmail.com <mailto:kravchenko.anton86@gmail.com>> wrote:
>
>     Hi there,
>
>     I am able to build and install MonetDB v11.21.19 "Jul2015-SP4" by
>     following and example
>     from https://www.monetdb.org/wiki/MonetDB:Building_from_sources
>     <https://www.monetdb.org/wiki/MonetDB:Building_from_sources>
>     But when I try build MonetDB v11.27.5 on same machine I get this
>     error (for a full log info
>     see https://drive.google.com/drive/folders/0B7dMycp0Ku-NTlRsZW95R0xCNmM?usp=sharing
>     <https://drive.google.com/drive/folders/0B7dMycp0Ku-NTlRsZW95R0xCNmM?usp=sharing>):
>
>     /bin/sh ../../../../../libtool  --tag=CC   --mode=link gcc
>     -DLIBPYAPI  -g -O2   -D_REENTRANT -module -avoid-version 
>     -o lib_pyapi.la <http://lib_pyapi.la/> -rpath
>     /home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/lib/monetdb5
>     lib_pyapi_la-pyapi.lo lib_pyapi_la-unicode.lo
>     lib_pyapi_la-pytypes.lo lib_pyapi_la-type_conversion.lo
>     lib_pyapi_la-formatinput.lo lib_pyapi_la-connection.lo
>     lib_pyapi_la-pyloader.lo lib_pyapi_la-emit.lo
>     lib_pyapi_la-conversion.lo 
>     ../../../../../monetdb5/tools/libmonetdb5.la
>     <http://libmonetdb5.la/> ../../../../../gdk/libbat.la
>     <http://libbat.la/>  -lpython2.7 -lpthread -ldl -lutil -lm -Xlinker
>     -export-dynamic
>     libtool: link: gcc -shared  -fPIC -DPIC  .libs/lib_pyapi_la-pyapi.o
>     .libs/lib_pyapi_la-unicode.o .libs/lib_pyapi_la-pytypes.o
>     .libs/lib_pyapi_la-type_conversion.o
>     .libs/lib_pyapi_la-formatinput.o .libs/lib_pyapi_la-connection.o
>     .libs/lib_pyapi_la-pyloader.o .libs/lib_pyapi_la-emit.o
>     .libs/lib_pyapi_la-conversion.o   -Wl,-rpath
>     -Wl,/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/monetdb5/tools/.libs
>     -Wl,-rpath
>     -Wl,/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/gdk/.libs
>     -Wl,-rpath
>     -Wl,/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/clients/mapilib/.libs
>     -Wl,-rpath
>     -Wl,/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/common/stream/.libs
>     -Wl,-rpath
>     -Wl,/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/lib
>     -L/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/gdk/.libs
>     -L/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/clients/mapilib/.libs
>     -L/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/common/stream/.libs
>     ../../../../../monetdb5/tools/.libs/libmonetdb5.so
>     /home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/gdk/.libs/libbat.so
>     /home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/clients/mapilib/.libs/libmapi.so
>     -lpcre -lxml2 ../../../../../gdk/.libs/libbat.so
>     /home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/common/stream/.libs/libstream.so
>     -lsnappy -lssl -lcrypto -lz -lpython2.7 -lpthread -ldl -lutil -lm 
>     -g -O2 -Wl,-export-dynamic   -pthread -Wl,-soname -Wl,lib_pyapi.so
>     -o .libs/lib_pyapi.so
>     /usr/bin/ld: cannot find -lpython2.7
>     collect2: ld returned 1 exit status
>     make[11]: *** [lib_pyapi.la <http://lib_pyapi.la/>] Error 1
>     make[11]: Leaving directory
>     `/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/sql/backends/monet5/UDF/pyapi'
>     make[10]: *** [all] Error 2
>     make[10]: Leaving directory
>     `/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/sql/backends/monet5/UDF/pyapi'
>     make[9]: *** [all-recursive] Error 1
>     make[9]: Leaving directory
>     `/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/sql/backends/monet5/UDF'
>     make[8]: *** [all] Error 2
>     make[8]: Leaving directory
>     `/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/sql/backends/monet5/UDF'
>     make[7]: *** [all-recursive] Error 1
>     make[7]: Leaving directory
>     `/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/sql/backends/monet5'
>     make[6]: *** [all] Error 2
>     make[6]: Leaving directory
>     `/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/sql/backends/monet5'
>     make[5]: *** [all-recursive] Error 1
>     make[5]: Leaving directory
>     `/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/sql/backends'
>     make[4]: *** [all] Error 2
>     make[4]: Leaving directory
>     `/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/sql/backends'
>     make[3]: *** [all-recursive] Error 1
>     make[3]: Leaving directory
>     `/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/sql'
>     make[2]: *** [all] Error 2
>     make[2]: Leaving directory
>     `/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD/sql'
>     make[1]: *** [all-recursive] Error 1
>     make[1]: Leaving directory
>     `/home/SECDEV.LOCAL/akravchenko/MonetDB-Jul2017-SP1/BUILD'
>     make: *** [all] Error 2
>
>     Thank you,
>     Anton
>
>
>
>
> _______________________________________________
> users-list mailing list
> users-list@monetdb.org
> https://www.monetdb.org/mailman/listinfo/users-list
>

--
Sjoerd Mullender


_______________________________________________
users-list mailing list
users-list@monetdb.org
https://www.monetdb.org/mailman/listinfo/users-list