MonetDB: Jul2017 - Initialize sys.argv, because tensorflow

Joeri van Ruth joeri.van.ruth at monetdbsolutions.com
Thu Jun 22 11:07:26 CEST 2017


Speaking of configure options, if I build and install MonetDB with
a --bin-dir outside the regular --prefix, mserver can't find its
scripts directory.  The result is that it does not start listening
on its requested mapi port, without a clear indication in the logs
as to what's going wrong.

Maybe we should remove the --bin-dir option altogether because if
we want to allow it, figuring out where to find supporting files is
much harder.  Looking in 

    `dirname "$0"`/../lib

is a little inflexible but at least it's clear what's going on.

By the way, i can image there is or has been a reason for not
logging anything when sql.mal cannot be found but in this scenario
it was inconvenient.  Would logging a warning clutter the log
file with useless warnings?

I'll file a bugzilla issue when I have a better idea of what the
desired behaviour actually is.  What do you think?

Joeri


On Mon, Jun 19, 2017 at 09:41:11AM +0200, Sjoerd Mullender wrote:
> On 16/06/17 14:58, Stefan Manegold wrote:
> > Dear all,
> > 
> > IMHO, each individual binary should indeed be robust again manual renaming,
> > i.e., by itself work properly no matter its own name.
> > 
> > Also, if we do not disable configure options such as --program-prefix
> > or --program-suffix, we should properly support them, i.e., with that very
> > build, binaries should not only be named but als called using the name as
> > set/modified by these options.
> > I have no preference as to how this can/should be implemented,
> > but it should either work, or we should disable these configure options.
> 
> We should probably have a configure option in monetdbd's configuration
> file that says which program to call when it wants to start an mserver.
> The default can then be argv[0] of monetdbd with monetdbd changed to
> mserver5 (as is done now).
> 
> > 
> > Best,
> > Stefan
> > 
> > ----- On Jun 16, 2017, at 2:24 PM, Eyal Rozenberg E.Rozenberg at cwi.nl wrote:
> > 
> >> Do you only want flexibility in the naming of mserver5, or of other
> >> binaries as well?
> >>
> >> Applications with a single binary, should typically be flexible enough
> >> to allow for users just renaming them willy-nilly, and get their own
> >> names from argv[0] and path to the binary from the CWD environment
> >> variable plus argv[0] (or by resolving the link /proc/self/exe on
> >> Linux). But when you have multiple binaries that can't work - which
> >> makes you wonder if changing binary names at all is such a good idea to
> >> begin with.
> >>
> >> Eyal
> >>
> >>
> >> On 06/16/2017 12:44 PM, Panagiotis Koutsourakis wrote:
> >>> Hello developers,
> >>>
> >>> Taking the opportunity from this commit, I wanted to initiate a
> >>> discussion with respect the executable names inside the code. A few
> >>> weeks ago I faced a problem when using the configure options
> >>> --program-prefix and --program-suffix, that change the executable names.
> >>> The problem was that the name mserver5 was hard coded and the daemon
> >>> could not find the actual executable.
> >>>
> >>> I see two possible solutions:
> >>> 1. Implement a mechanism that make a note in the monetdb_config.h
> >>> 2. Implement a run time mechanism to compute the relevant file names.
> >>>
> >>> Solution #1 is probably a bit more difficult because the prefix/suffix
> >>> transformation of the file names happens during the installation.
> >>>
> >>> Solution #2 is already partially implemented in the daemon (see the
> >>> function get_bin_path in the file mutils.c), but will probably need to
> >>> be generalized somehow.
> >>>
> >>> Comments? Ideas?
> >>>
> >>> Best,
> >>> Panos.
> >>>
> >>> Mark Raasveldt @ 2017-06-15 13:22 GMT:
> >>>
> >>>> Changeset: 44e0bc7dab59 for MonetDB
> >>>> URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=44e0bc7dab59
> >>>> Modified Files:
> >>>> 	sql/backends/monet5/UDF/pyapi/pyapi.c
> >>>> Branch: Jul2017
> >>>> Log Message:
> >>>>
> >>>> Initialize sys.argv, because tensorflow requires this.
> >>>>
> >>>>
> >>>> diffs (20 lines):
> >>>>
> >>>> diff --git a/sql/backends/monet5/UDF/pyapi/pyapi.c
> >>>> b/sql/backends/monet5/UDF/pyapi/pyapi.c
> >>>> --- a/sql/backends/monet5/UDF/pyapi/pyapi.c
> >>>> +++ b/sql/backends/monet5/UDF/pyapi/pyapi.c
> >>>> @@ -1425,9 +1425,16 @@ PYFUNCNAME(PyAPIprelude)(void *ret) {
> >>>>  	MT_lock_init(&queryLock, "query_lock");
> >>>>  	MT_lock_set(&pyapiLock);
> >>>>  	if (!pyapiInitialized) {
> >>>> +#ifdef IS_PY3K
> >>>> +		wchar_t* program = Py_DecodeLocale("mserver5", NULL);
> >>>> +		wchar_t* argv[] = { program };
> >>>> +#else
> >>>> +		char* argv[] = {"mserver5"};
> >>>> +#endif
> >>>>  		str msg = MAL_SUCCEED;
> >>>>  		PyObject *tmp;
> >>>>  		Py_Initialize();
> >>>> +		PySys_SetArgvEx(1, argv, 0);
> >>>>  		_import_array();
> >>>>  		msg = _connection_init();
> >>>>  		if (msg != MAL_SUCCEED) {
> >>>> _______________________________________________
> >>>> checkin-list mailing list
> >>>> checkin-list at monetdb.org
> >>>> https://www.monetdb.org/mailman/listinfo/checkin-list
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> developers-list mailing list
> >>>> developers-list at monetdb.org
> >>>> https://www.monetdb.org/mailman/listinfo/developers-list
> >> _______________________________________________
> >> developers-list mailing list
> >> developers-list at monetdb.org
> >> https://www.monetdb.org/mailman/listinfo/developers-list
> > 
> 
> 
> -- 
> Sjoerd Mullender
> 




> _______________________________________________
> developers-list mailing list
> developers-list at monetdb.org
> https://www.monetdb.org/mailman/listinfo/developers-list



More information about the developers-list mailing list