prevent mitosis from partitioning (filter) function arguments
I have implemented a custom join as a filter function. By design, it cannot work correctly when the input tables are partitioned by mitosis.
How can I prevent mitosis from partitioning its inputs?
Roberto
UPDATE:
I solved this for now by hardcoding in opt_mitosis a bailout condition for that specific function.
Wouldn't it be useful to use a function marker for this, along the lines of {inline|unsafe} markers?
Something like:
blocking command myfunction(...);
and then disable mitosis for all blocking functions (i.e. functions that need to consume their whole input)?
On the same topic, I also noticed that mitosis currently isn't disabled on unsafe functions. Is that correct? If a function is unsafe, calling it multiple times could be... .. well ... unsafe? ;-)
Roberto
On 9 February 2017 at 15:10, Roberto Cornacchia < roberto.cornacchia@gmail.com> wrote:
I have implemented a custom join as a filter function. By design, it cannot work correctly when the input tables are partitioned by mitosis.
How can I prevent mitosis from partitioning its inputs?
Roberto
On 9 Feb 2017, at 16:09, Roberto Cornacchia roberto.cornacchia@gmail.com wrote:
UPDATE:
I solved this for now by hardcoding in opt_mitosis a bailout condition for that specific function.
Wouldn't it be useful to use a function marker for this, along the lines of {inline|unsafe} markers?
Something like:
blocking command myfunction(...);
and then disable mitosis for all blocking functions (i.e. functions that need to consume their whole input)?
On the same topic, I also noticed that mitosis currently isn't disabled on unsafe functions. Is that correct? If a function is unsafe, calling it multiple times could be... .. well ... unsafe? ;-)
Hmmm, normally ‘unsafe’ is the keyword you need. Doesn’t it work for your function?
I took a look at the impl. of quantile, because its input is surely not partitioned. but I can’t quickly find how that requirement is past to mitosis...
Roberto
On 9 February 2017 at 15:10, Roberto Cornacchia roberto.cornacchia@gmail.com wrote: I have implemented a custom join as a filter function. By design, it cannot work correctly when the input tables are partitioned by mitosis.
How can I prevent mitosis from partitioning its inputs?
Roberto
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
On 10 February 2017 at 04:52, Ying Zhang Y.Zhang@cwi.nl wrote:
On 9 Feb 2017, at 16:09, Roberto Cornacchia <
roberto.cornacchia@gmail.com> wrote:
UPDATE:
I solved this for now by hardcoding in opt_mitosis a bailout condition
for that specific function.
Wouldn't it be useful to use a function marker for this, along the lines
of {inline|unsafe} markers?
Something like:
blocking command myfunction(...);
and then disable mitosis for all blocking functions (i.e. functions that
need to consume their whole input)?
On the same topic, I also noticed that mitosis currently isn't disabled
on unsafe functions. Is that correct? If a function is unsafe, calling it multiple times could be... .. well ... unsafe? ;-)
Hmmm, normally ‘unsafe’ is the keyword you need.
In my view, yes and no.
Yes, 'unsafe' should be a trigger for mitosis, bur apparently it isn't.
No, it isn't what I'm looking for, because it has much stronger consequences. If I'm not mistaken, unsafe means "has side effects", which implies: 1) not thread-safe 2) needs all input at once 3) its result cannot be reused in the plan.
The third is a show-stopper for me. That is why is was suggesting one more keyword, which would have only the first two effects from the list above.
Doesn’t it work for your function?
No, 'unsafe' doesn't affect my function, i.e. it still partitions the input and calls it multiple times. I did not check whether it at least serializes its execution, which would at least cover point 1 above.
I took a look at the impl. of quantile, because its input is surely not
partitioned. but I can’t quickly find how that requirement is past to mitosis...
Roberto
On 9 February 2017 at 15:10, Roberto Cornacchia <
roberto.cornacchia@gmail.com> wrote:
I have implemented a custom join as a filter function. By design, it cannot work correctly when the input tables are partitioned by mitosis.
How can I prevent mitosis from partitioning its inputs?
Roberto
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
About quantile, it is excluded in opt_mitosis by this:
if (p->argc > 2 && getModuleId(p) == aggrRef && getFunctionId(p) != subcountRef && getFunctionId(p) != subminRef && getFunctionId(p) != submaxRef && getFunctionId(p) != subavgRef && getFunctionId(p) != subsumRef && getFunctionId(p) != subprodRef) return 0; It excludes all aggregates except those listed. For my function, I did something similar here.
On 10 February 2017 at 08:45, Roberto Cornacchia < roberto.cornacchia@gmail.com> wrote:
On 10 February 2017 at 04:52, Ying Zhang Y.Zhang@cwi.nl wrote:
On 9 Feb 2017, at 16:09, Roberto Cornacchia <
roberto.cornacchia@gmail.com> wrote:
UPDATE:
I solved this for now by hardcoding in opt_mitosis a bailout condition
for that specific function.
Wouldn't it be useful to use a function marker for this, along the
lines of {inline|unsafe} markers?
Something like:
blocking command myfunction(...);
and then disable mitosis for all blocking functions (i.e. functions
that need to consume their whole input)?
On the same topic, I also noticed that mitosis currently isn't disabled
on unsafe functions. Is that correct? If a function is unsafe, calling it multiple times could be... .. well ... unsafe? ;-)
Hmmm, normally ‘unsafe’ is the keyword you need.
In my view, yes and no.
Yes, 'unsafe' should be a trigger for mitosis, bur apparently it isn't.
No, it isn't what I'm looking for, because it has much stronger consequences. If I'm not mistaken, unsafe means "has side effects", which implies:
- not thread-safe
- needs all input at once
- its result cannot be reused in the plan.
The third is a show-stopper for me. That is why is was suggesting one more keyword, which would have only the first two effects from the list above.
Doesn’t it work for your function?
No, 'unsafe' doesn't affect my function, i.e. it still partitions the input and calls it multiple times. I did not check whether it at least serializes its execution, which would at least cover point 1 above.
I took a look at the impl. of quantile, because its input is surely not
partitioned. but I can’t quickly find how that requirement is past to mitosis...
Roberto
On 9 February 2017 at 15:10, Roberto Cornacchia <
roberto.cornacchia@gmail.com> wrote:
I have implemented a custom join as a filter function. By design, it cannot work correctly when the input tables are
partitioned
by mitosis.
How can I prevent mitosis from partitioning its inputs?
Roberto
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
On 10 Feb 2017, at 10:25, Roberto Cornacchia roberto.cornacchia@gmail.com wrote:
About quantile, it is excluded in opt_mitosis by this:
if (p->argc > 2 && getModuleId(p) == aggrRef &&
getFunctionId(p) != subcountRef && getFunctionId(p) != subminRef && getFunctionId(p) != submaxRef && getFunctionId(p) != subavgRef && getFunctionId(p) != subsumRef && getFunctionId(p) != subprodRef)return 0;
It excludes all aggregates except those listed. For my function, I did something similar here.
Hmm, this is not the cleanest / most elegant solution. For UDFs, a user won’t want to “mess” with the MonetDB internal code.
Since the third behaviour of ‘unsafe’ is a particular show stopper for you, I’m afraid MonetDB doesn’t have the exact feature you’re looking for. Fire a feature request?
On 10 February 2017 at 08:45, Roberto Cornacchia roberto.cornacchia@gmail.com wrote: On 10 February 2017 at 04:52, Ying Zhang Y.Zhang@cwi.nl wrote:
On 9 Feb 2017, at 16:09, Roberto Cornacchia roberto.cornacchia@gmail.com wrote:
UPDATE:
I solved this for now by hardcoding in opt_mitosis a bailout condition for that specific function.
Wouldn't it be useful to use a function marker for this, along the lines of {inline|unsafe} markers?
Something like:
blocking command myfunction(...);
and then disable mitosis for all blocking functions (i.e. functions that need to consume their whole input)?
On the same topic, I also noticed that mitosis currently isn't disabled on unsafe functions. Is that correct? If a function is unsafe, calling it multiple times could be... .. well ... unsafe? ;-)
Hmmm, normally ‘unsafe’ is the keyword you need.
In my view, yes and no.
Yes, 'unsafe' should be a trigger for mitosis, bur apparently it isn't.
No, it isn't what I'm looking for, because it has much stronger consequences. If I'm not mistaken, unsafe means "has side effects", which implies:
- not thread-safe
- needs all input at once
- its result cannot be reused in the plan.
The third is a show-stopper for me. That is why is was suggesting one more keyword, which would have only the first two effects from the list above.
Doesn’t it work for your function?
No, 'unsafe' doesn't affect my function, i.e. it still partitions the input and calls it multiple times. I did not check whether it at least serializes its execution, which would at least cover point 1 above.
I took a look at the impl. of quantile, because its input is surely not partitioned. but I can’t quickly find how that requirement is past to mitosis...
Roberto
On 9 February 2017 at 15:10, Roberto Cornacchia roberto.cornacchia@gmail.com wrote: I have implemented a custom join as a filter function. By design, it cannot work correctly when the input tables are partitioned by mitosis.
How can I prevent mitosis from partitioning its inputs?
Roberto
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
Hi Jennie, thanks for the feedback.
I will file a feature request indeed
About 'unsafe', is there a bug here? I.e. should mitosis be disabled on functions marked as unsafe, as I would expect?
On 12 Feb 2017 8:02 am, "Ying Zhang" Y.Zhang@cwi.nl wrote:
On 10 Feb 2017, at 10:25, Roberto Cornacchia <
roberto.cornacchia@gmail.com> wrote:
About quantile, it is excluded in opt_mitosis by this:
if (p->argc > 2 && getModuleId(p) == aggrRef &&
getFunctionId(p) != subcountRef && getFunctionId(p) != subminRef && getFunctionId(p) != submaxRef && getFunctionId(p) != subavgRef && getFunctionId(p) != subsumRef && getFunctionId(p) != subprodRef)return 0;
It excludes all aggregates except those listed. For my function, I did something similar here.
Hmm, this is not the cleanest / most elegant solution. For UDFs, a user won’t want to “mess” with the MonetDB internal code.
Since the third behaviour of ‘unsafe’ is a particular show stopper for you, I’m afraid MonetDB doesn’t have the exact feature you’re looking for. Fire a feature request?
On 10 February 2017 at 08:45, Roberto Cornacchia <
roberto.cornacchia@gmail.com> wrote:
On 10 February 2017 at 04:52, Ying Zhang Y.Zhang@cwi.nl wrote:
On 9 Feb 2017, at 16:09, Roberto Cornacchia <
roberto.cornacchia@gmail.com> wrote:
UPDATE:
I solved this for now by hardcoding in opt_mitosis a bailout condition
for that specific function.
Wouldn't it be useful to use a function marker for this, along the
lines of {inline|unsafe} markers?
Something like:
blocking command myfunction(...);
and then disable mitosis for all blocking functions (i.e. functions
that need to consume their whole input)?
On the same topic, I also noticed that mitosis currently isn't
disabled on unsafe functions. Is that correct? If a function is unsafe, calling it multiple times could be... .. well ... unsafe? ;-)
Hmmm, normally ‘unsafe’ is the keyword you need.
In my view, yes and no.
Yes, 'unsafe' should be a trigger for mitosis, bur apparently it isn't.
No, it isn't what I'm looking for, because it has much stronger
consequences.
If I'm not mistaken, unsafe means "has side effects", which implies:
- not thread-safe
- needs all input at once
- its result cannot be reused in the plan.
The third is a show-stopper for me. That is why is was suggesting one
more keyword, which would have only the first two effects from the list above.
Doesn’t it work for your function?
No, 'unsafe' doesn't affect my function, i.e. it still partitions the
input and calls it multiple times. I did not check whether it at least serializes its execution, which would at least cover point 1 above.
I took a look at the impl. of quantile, because its input is surely not
partitioned.
but I can’t quickly find how that requirement is past to mitosis...
Roberto
On 9 February 2017 at 15:10, Roberto Cornacchia <
roberto.cornacchia@gmail.com> wrote:
I have implemented a custom join as a filter function. By design, it cannot work correctly when the input tables are
partitioned
by mitosis.
How can I prevent mitosis from partitioning its inputs?
Roberto
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
participants (2)
-
Roberto Cornacchia -
Ying Zhang