Create R function that can be applied to an entire table

Mark Raasveldt m.raasveldt at cwi.nl
Wed Apr 6 22:13:55 CEST 2016


Hey Willem,

This is a bug, or more accurately, lazy programming ;) the function just assume nobody will need more than 10,000 characters for the argument names. If you want to do a lazy fix you can just increase the size of the argnames array. If you want to do a not lazy fix you can check the size of each of the argument names and then allocate enough space for them.

I have to ask, though, how many columns are you using at once to trigger this error? If this is the only issue you run into when operating on that many columns in the same query I think that would be a small miracle ;)

Regards,

Mark



----- Original Message -----
From: "Willem Ligtenberg" <willem.ligtenberg at openanalytics.eu>
To: "users-list" <users-list at monetdb.org>
Sent: Wednesday, April 6, 2016 9:45:26 PM
Subject: Create R function that can be applied to an entire table

Hi,

I guess I am pushing the user defined R functions to their limits.
I know want to implement some batch correcting in R and then apply that to an entire table.
I have written R code, that will generate a function definition that has matching parameters.
And then I want to call the function as follows:

CREATE TABLE data_bc AS SELECT * FROM batch_correction( (SELECT * FROM data) ) WITH DATA;

I then get the following error:
Error in .local(conn, statement, ...) : 
  Unable to execute statement 'CREATE TABLE hcs_batch AS SELECT * FROM batch_correction( (SELECT * FROM hcs_norm) ) WITH DATA;'.
Server says '!Command too large'

Looking at the source, this error is raised in rapi.c
on either line 467 or 473.

After editing the source to figure out which of the lines was triggered, I figured out is was on line:
467 it is triggered, so the following code raises the issue that the command is too large:

        pos = 0;
        for (i = pci->retc + 2; i < pci->argc && pos < sizeof(argnames); i++) {
                pos += snprintf(argnames + pos, sizeof(argnames) - pos, "%s%s",
                                                args[i], i < pci->argc - 1 ? ", " : "");
        }
        if (pos >= sizeof(argnames)) {
                msg = createException(MAL, "rapi.eval", "Command too large1");
                goto wrapup;
        }

(yes, I added the 1 to large, to make it unique)

If this is not a bug, any help in getting my code to work is greatly appreciated.

Kind regards,

Willem
_______________________________________________
users-list mailing list
users-list at monetdb.org
https://www.monetdb.org/mailman/listinfo/users-list


More information about the users-list mailing list