Program contains errors

4 Aug 2016

On 14 Aug 2016, at 09:44, Stefano Fioravanzo <fioravanzos@gmail.com
> wrote:

Hello,

After another trial with the queries that produced the error
mentioned in the last mail, I got another error line which is:
*** Error in `mserver5': corrupted double-linked list:
0x00007feb1c20aec0 ***

As this issue seems to be a dead end, I’m just hoping that
something could ring a bell to someone for a solution.

Thanks,
Stefano

    On 10 Aug 2016, at 18:34, Stefano Fioravanzo <
    fioravanzos@gmail.com> wrote:

    Hello Stefan,

    Every time I have loaded the raw data from my csv files, so the
    are no issues in using different versions.

    Sadly the merovingian log does not give any information at all
    when this error occurs, and neither does the server console.
    Is there a way (in the stable version) to enable some verbose
    output, or to know in some way what actual error gave the
    message ‘program contains error’?

    To check if it is a space problem, I have created 80 GB of data
    using github.com/niocs/tqgen
    Command used: ./tqgen  --OutFilePat  /home/ubuntu/fake_data/
    tq.YYYYMMDD.csv  --NumStk 1000 --Seed 234532 --DateBeg 20200101
    --DateEnd 20211131
    And then to import: for i in /home/ubuntu/fake_data/*; do
    mclient -d testdatabase -s "copy offset 2 into fake_data from
    '$i' using delimiters ',' NULL AS '';"; done

    Well, even with this data some queries returned the same error,
    ‘Program contains error’.

    After a few more tests, after recreating the database and
    reimporting my data, the server console gave me this error on a
    query, I hove it can be more informative:

    !WARNING: gdk_bat.c:2083: assertion `b->batCount <= b->
    batCapacity' failed
    !WARNING: gdk_bat.c:2084: assertion `b->T->heap.size >= b->T->
    heap.free' failed
    !WARNING: gdk_bat.c:2083: assertion `b->batCount <= b->
    batCapacity' failed
    !WARNING: gdk_bat.c:2084: assertion `b->T->heap.size >= b->T->
    heap.free' failed
    Segmentation fault

    So there seem to be a problem with space indeed.
    The table used in the query that produced the error came from a
    csv of 80GB (reserved data which I cannot share), I have 30GB
    of RAM (There is nothing else running on the machine), 60GB of
    swap, 450GB of disk space….
    Any suggestion?

    Stefano

        On 05 Aug 2016, at 14:07, Stefan Manegold <
        Stefan.Manegold@cwi.nl> wrote:

        Hi Stefano,

        did/do you re-create/re-load your database from scratch for
        each version of MonetDB you tested,
        or do you access that same database (same dbfarm) using the
        different versions?

        Please be aware that we only support database upgrades
        between two consecutive releases of MonetDB.
        All other cases are not supported and their behavior is
        undefined, and might thus include
        anything from the server refusing to start to "silent"
        corruption of the database.

        Having said that, it's generally hard for us to analyze
        problems, let alone solve them,
        without being provided detailed instructions (if necessary
        including data) how to reproduce them.

        More over, the assertion you get with the default branch
        has nothing to do with not enough space;
        it indicated that internally some administrative
        information is inconsistent --- why is hard,
        if not impossible, to understand without being able to
        reproduce the problem.

        Also, with the 'Program contains error' message of the
        client, their might be more detailed
        information in the merovingian log or on the server
        console?

        Best,
        Stefan

        ----- On Aug 4, 2016, at 10:09 PM, Stefano Fioravanzo
        fioravanzos@gmail.com wrote:

            Hello,

            I will briefly recap the problem I have already
            reported a few weeks ago: I am
            using monetdb default branch (I did not compile the
            stable version because I
            need embedded python) and I have a large table (~80
            GB). A simple query like
            SELECT * FROM large_table WHERE field=x, fails.
            mserver5 reports this error:
            mserver5: gdk_select.c:867: fullscan_int: Assertion
            `cnt < (bn)->S.capacity'
            failed.

            It seems that monet thinks it doesn’t have enough space
            to do a full scan of the
            table, maybe? (The machine has 800GB of free disk space
            and 30GB of RAM -
            Ubuntu 14.04).
            I was using a default branch of december 2015, I have
            tried to use the latest
            default branch (4/08/16), recompiled, installed but
            nothing, always the same
            error.

            So, as suggested, I have compiled the latest stable
            release (Jun2016-SP1). Now
            even a query which does not require monet to read the
            whole table (like SELECT
            * FROM large_table LIMIT 10 - This query worked fine in
            the default branch
            version) fails with ‘Program contains error’ message by
            client.
            Selecting a single column of the table works just fine.
            Moreover, with the stable release, the same issue
            arises with the second biggest
            table of the database (~35 GB).
            [Here you can find the configuration of MonetDB:
            http://pastie.org/private/r8gtssg944aqwnigrysa]

            All queries on all the other tables in the database
            (they are all much smaller
            than these two) work fine. I do not understand if the
            issue is in the size of
            table or somewhere else. How can I know what ‘error’
            the message ‘program
            contains errors’ is referring to?

            NOTE: At the beginning, using the default compiled
            branch everything worked just
            fine! Every query on my large_table worked as expected.
            Then I had to recreate
            the database from scratch so I deleted the dbfarm and
            created a new one. That’s
            when this headache started…

            At this point I don’t know what to do, any help would
            be MUCH appreciated.

            Thanks,

            Stefano

            _______________________________________________
            users-list mailing list
            users-list@monetdb.org
            https://www.monetdb.org/mailman/listinfo/users-list

        --
        | Stefan.Manegold@CWI.nl | DB Architectures   (DA) |
        | www.CWI.nl/~manegold/  | Science Park 123 (L321) |
        | +31 (0)20 592-4212     | 1098 XG Amsterdam  (NL) |
        _______________________________________________
        users-list mailing list
        users-list@monetdb.org
        https://www.monetdb.org/mailman/listinfo/users-list

Stefano Fioravanzo

Stefan Manegold

Stefano Fioravanzo

Stefano Fioravanzo

Anderson, David B

Stefano Fioravanzo

Niels Nes

Stefano Fioravanzo

tags (0)

participants (4)