[MonetDB-users] Embedded Use: Persisting and Metadata
Finally, I have managed to get the Mbeddded example working in VC++ 2005 Express Edition on Windows XP ! Thanks to the folks in the mailing list.
Use Case: MonetDB5 , Embedded Mode , SQL
To move towards the application I intend to experiment on, I need some more information: 1) How to make data durable? In the simple example, I tried the following: a) First Run: Create table and insert rows. Select to check if everything is ok. b) Second Run: Only perform selection
I get an error in the second run.
2) How to get meta data of the database? a) I want to know the size of the tables and meta tables in the data base. In particular I am interested in seeing the growth of the database on addition of content. b) Typically, there is a master table in the database that records information of the schema persisted. What is the name of the Master Table and its schema in MonetDB? c) SQLite has an independent analyzer tool which dumps the state of the storage from which I can extract most of the data I need. Does MonetDB have a similar mechanism?
~Yuva
code snippet >>>>>
dbh = embedded_sql (set, setlen); if (dbh == NULL || mapi_error(dbh)) die(dbh, hdl); /* switch off autocommit */ if (mapi_setAutocommit(dbh, 0) != MOK || mapi_error(dbh)) die(dbh,NULL); if ((hdl = mapi_query(dbh, "create table emp" " (name varchar(20),age int)")) == NULL || mapi_error(dbh)) die(dbh, hdl); close_handle(dbh,hdl); for(i=0; i< 1000; i++) { char query[100]; _snprintf(query, 100,"insert into emp values('user%d', %d)", i, (i % 82)); if ((hdl = mapi_query(dbh, query)) == NULL || mapi_error(dbh)) die(dbh, hdl); close_handle(dbh,hdl); } if ((hdl = mapi_query(dbh, "select * from emp")) == NULL || mapi_error(dbh)) die(dbh, hdl); i=0; while (mapi_fetch_row(hdl)) { char *age = mapi_fetch_field(hdl, 1); i= i+ atoi(age); } if (mapi_error(dbh)) die(dbh, hdl); close_handle(dbh,hdl); printf("The sum is %d \n",i); mapi_disconnect(dbh); return 0;
On Fri, Feb 08, 2008 at 01:08:13PM +0530, Yuvaraj Athur Raghuvir wrote:
Finally, I have managed to get the Mbeddded example working in VC++ 2005 Express Edition on Windows XP ! Thanks to the folks in the mailing list. Use Case: MonetDB5 , Embedded Mode , SQL To move towards the application I intend to experiment on, I need some more information:
- How to make data durable? In the simple example, I tried
the following: a) First Run: Create table and insert rows. Select to check if everything is ok. b) Second Run: Only perform selection
I get an error in the second run.
- How to get meta data of the database?
a) I want to know the size of the tables and meta tables in the data base. In particular I am interested in seeing the growth of the database on addition of content.
The growth depends on your column types used. Ie if you use lets say integers, then each new row of 8 columns, would simply increase the storage requirements with 8 * 4 bytes. etc.
b) Typically, there is a master table in the database that records information of the schema persisted. What is the name of the Master Table and its schema in MonetDB?
You mean the table 'tables' ? A select * from tables gives you a list of all your tables. (see also schemas, columns , etc)
c) SQLite has an independent analyzer tool which dumps the state of the storage from which I can extract most of the data I need. Does MonetDB have a similar mechanism?
There is no storage analyzer for MonetDB.
~Yuva
code snippet >>>>>
dbh = embedded_sql (set, setlen); if (dbh == NULL || mapi_error(dbh)) die(dbh, hdl); /* switch off autocommit */ if (mapi_setAutocommit(dbh, 0) != MOK || mapi_error(dbh))
Okay here you disabled autocommit, which is fine.
die(dbh,NULL); if ((hdl = mapi_query(dbh, "create table emp" " (name varchar(20),age int)")) == NULL || mapi_error(dbh)) die(dbh, hdl); close_handle(dbh,hdl); for(i=0; i< 1000; i++) { char query[100]; _snprintf(query, 100,"insert into empvalues('user%d', %d)", i, (i % 82)); if ((hdl = mapi_query(dbh, query)) == NULL || mapi_error(dbh)) die(dbh, hdl); close_handle(dbh,hdl); }
But then I would expect an 'commit;' here.
if ((hdl = mapi_query(dbh, "select * from emp")) == NULL ||mapi_error(dbh)) die(dbh, hdl); i=0; while (mapi_fetch_row(hdl)) { char *age = mapi_fetch_field(hdl, 1); i= i+ atoi(age); } if (mapi_error(dbh)) die(dbh, hdl); close_handle(dbh,hdl); printf("The sum is %d \n",i); mapi_disconnect(dbh); return 0;
Niels
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
Thanks.
Regarding db size, I would expect that the storage strategy will have additional overheads due to bookkeeping requirements necessary for reconstruction of the relations between the data elements. Since there are no indices (I assume this due to columnar storage) I can discount on the space required for indices.
Are there Mapi calls that I can use to determine the size of the tables? I would like to horizontally partition the data across multiple stores based on table size and growth estimates.
~Yuva
On 08/02/2008, Yuvaraj Athur Raghuvir yuvaraj.a.r@gmail.com wrote:
Finally, I have managed to get the Mbeddded example working in VC++ 2005 Express Edition on Windows XP ! Thanks to the folks in the mailing list.
Use Case: MonetDB5 , Embedded Mode , SQL
To move towards the application I intend to experiment on, I need some more information:
- How to make data durable? In the simple example, I tried the following:
a) First Run: Create table and insert rows. Select to check if everything is ok. b) Second Run: Only perform selection
I get an error in the second run.
- How to get meta data of the database?
a) I want to know the size of the tables and meta tables in the data base. In particular I am interested in seeing the growth of the database on addition of content. b) Typically, there is a master table in the database that records information of the schema persisted. What is the name of the Master Table and its schema in MonetDB? c) SQLite has an independent analyzer tool which dumps the state of the storage from which I can extract most of the data I need. Does MonetDB have a similar mechanism?
~Yuva
code snippet >>>>>
dbh = embedded_sql (set, setlen); if (dbh == NULL || mapi_error(dbh)) die(dbh, hdl); /* switch off autocommit */ if (mapi_setAutocommit(dbh, 0) != MOK || mapi_error(dbh)) die(dbh,NULL); if ((hdl = mapi_query(dbh, "create table emp" " (name varchar(20),age int)")) == NULL || mapi_error(dbh)) die(dbh, hdl); close_handle(dbh,hdl); for(i=0; i< 1000; i++) { char query[100]; _snprintf(query, 100,"insert into emp values(\'user%d\', %d)", i, (i% 82)); if ((hdl = mapi_query(dbh, query)) == NULL || mapi_error(dbh)) die(dbh, hdl); close_handle(dbh,hdl); } if ((hdl = mapi_query(dbh, "select * from emp")) == NULL || mapi_error(dbh)) die(dbh, hdl); i=0; while (mapi_fetch_row(hdl)) { char *age = mapi_fetch_field(hdl, 1); i= i+ atoi(age); } if (mapi_error(dbh)) die(dbh, hdl); close_handle(dbh,hdl); printf("The sum is %d \n",i); mapi_disconnect(dbh); return 0;
Yuvaraj Athur Raghuvir wrote:
Thanks.
Regarding db size, I would expect that the storage strategy will have additional overheads due to bookkeeping requirements necessary for reconstruction of the relations between the data elements. Since there are no indices (I assume this due to columnar storage) I can discount on the space required for indices.
Indices are automatically created and dropped as a side effect of queries. They disappear at the end of a session.
Are there Mapi calls that I can use to determine the size of the tables? I would like to horizontally partition the data across multiple stores based on table size and growth estimates.
the SQL level does not open up the kernel information about storage requirements. At the MAL level several operators are available to assess storage cost.
~Yuva
On 08/02/2008, Yuvaraj Athur Raghuvir yuvaraj.a.r@gmail.com wrote:
Finally, I have managed to get the Mbeddded example working in VC++ 2005 Express Edition on Windows XP ! Thanks to the folks in the mailing list.
Use Case: MonetDB5 , Embedded Mode , SQL
To move towards the application I intend to experiment on, I need some more information:
- How to make data durable? In the simple example, I tried the following:
a) First Run: Create table and insert rows. Select to check if everything is ok. b) Second Run: Only perform selection
I get an error in the second run.
- How to get meta data of the database?
a) I want to know the size of the tables and meta tables in the data base. In particular I am interested in seeing the growth of the database on addition of content. b) Typically, there is a master table in the database that records information of the schema persisted. What is the name of the Master Table and its schema in MonetDB? c) SQLite has an independent analyzer tool which dumps the state of the storage from which I can extract most of the data I need. Does MonetDB have a similar mechanism?
~Yuva
code snippet >>>>>
dbh = embedded_sql (set, setlen); if (dbh == NULL || mapi_error(dbh)) die(dbh, hdl); /* switch off autocommit */ if (mapi_setAutocommit(dbh, 0) != MOK || mapi_error(dbh)) die(dbh,NULL); if ((hdl = mapi_query(dbh, "create table emp" " (name varchar(20),age int)")) == NULL || mapi_error(dbh)) die(dbh, hdl); close_handle(dbh,hdl); for(i=0; i< 1000; i++) { char query[100]; _snprintf(query, 100,"insert into emp values(\'user%d\', %d)", i, (i% 82)); if ((hdl = mapi_query(dbh, query)) == NULL || mapi_error(dbh)) die(dbh, hdl); close_handle(dbh,hdl); } if ((hdl = mapi_query(dbh, "select * from emp")) == NULL || mapi_error(dbh)) die(dbh, hdl); i=0; while (mapi_fetch_row(hdl)) { char *age = mapi_fetch_field(hdl, 1); i= i+ atoi(age); } if (mapi_error(dbh)) die(dbh, hdl); close_handle(dbh,hdl); printf("The sum is %d \n",i); mapi_disconnect(dbh); return 0;
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
From a C/C++ code, how do I access the MAL level storage operators? I would
assume that using the right libraries I should be able to get this data into my C/C++ code. Could you please point me to the right resources here? Thanks.
Further, I want to know the storage cost on disk for the entire database. As I see the data is stored as a "farm". Does this mean that I can sum the sizes of all files under the root folder to get the disk storage cost?
~Yuva
On Feb 9, 2008 12:10 AM, Martin Kersten Martin.Kersten@cwi.nl wrote:
Yuvaraj Athur Raghuvir wrote:
Thanks.
Regarding db size, I would expect that the storage strategy will have additional overheads due to bookkeeping requirements necessary for reconstruction of the relations between the data elements. Since there are no indices (I assume this due to columnar storage) I can discount on the space required for indices.
Indices are automatically created and dropped as a side effect of queries. They disappear at the end of a session.
Are there Mapi calls that I can use to determine the size of the tables? I would like to horizontally partition the data across multiple stores based on table size and growth estimates.
the SQL level does not open up the kernel information about storage requirements. At the MAL level several operators are available to assess storage cost.
~Yuva
On 08/02/2008, Yuvaraj Athur Raghuvir yuvaraj.a.r@gmail.com wrote:
Finally, I have managed to get the Mbeddded example working in VC++
2005
Express Edition on Windows XP ! Thanks to the folks in the mailing
list.
Use Case: MonetDB5 , Embedded Mode , SQL
To move towards the application I intend to experiment on, I need some
more
information:
- How to make data durable? In the simple example, I tried the
following:
a) First Run: Create table and insert rows. Select to check if
everything
is ok. b) Second Run: Only perform selection
I get an error in the second run.
- How to get meta data of the database?
a) I want to know the size of the tables and meta tables in the data
base.
In particular I am interested in seeing the growth of the database on addition of content. b) Typically, there is a master table in the database that records information of the schema persisted. What is the name of the Master
Table
and its schema in MonetDB? c) SQLite has an independent analyzer tool which dumps the state of the storage from which I can extract most of the data I need. Does MonetDB
have
a similar mechanism?
~Yuva
> code snippet >>>>> >
dbh = embedded_sql (set, setlen); if (dbh == NULL || mapi_error(dbh)) die(dbh, hdl); /* switch off autocommit */ if (mapi_setAutocommit(dbh, 0) != MOK || mapi_error(dbh)) die(dbh,NULL); if ((hdl = mapi_query(dbh, "create table emp" " (name varchar(20),age int)")) == NULL || mapi_error(dbh)) die(dbh, hdl); close_handle(dbh,hdl); for(i=0; i< 1000; i++) { char query[100]; _snprintf(query, 100,"insert into emp values(\'user%d\', %d)",i, (i
% 82)); if ((hdl = mapi_query(dbh, query)) == NULL || mapi_error(dbh)) die(dbh, hdl); close_handle(dbh,hdl); } if ((hdl = mapi_query(dbh, "select * from emp")) == NULL || mapi_error(dbh)) die(dbh, hdl); i=0; while (mapi_fetch_row(hdl)) { char *age = mapi_fetch_field(hdl, 1); i= i+ atoi(age); } if (mapi_error(dbh)) die(dbh, hdl); close_handle(dbh,hdl); printf("The sum is %d \n",i); mapi_disconnect(dbh); return 0;
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
Yuvaraj Athur Raghuvir wrote:
From a C/C++ code, how do I access the MAL level storage operators? I
to find operators of interest, you may start at the appendix of the manual: http://monetdb.cwi.nl/projects/monetdb/MonetDB/Documentation/Instruction-Hel... and perform a grep/find
would assume that using the right libraries I should be able to get this data into my C/C++ code. Could you please point me to the right resources here? Thanks.
Further, I want to know the storage cost on disk for the entire database. As I see the data is stored as a "farm". Does this mean that I can sum the sizes of all files under the root folder to get the disk storage cost?
yes, that is the persistent storage requirement of the database
~Yuva
On Feb 9, 2008 12:10 AM, Martin Kersten <Martin.Kersten@cwi.nl mailto:Martin.Kersten@cwi.nl> wrote:
Yuvaraj Athur Raghuvir wrote: > Thanks. > > Regarding db size, I would expect that the storage strategy will have > additional overheads due to bookkeeping requirements necessary for > reconstruction of the relations between the data elements. Since there > are no indices (I assume this due to columnar storage) I can discount > on the space required for indices. > Indices are automatically created and dropped as a side effect of queries. They disappear at the end of a session. > Are there Mapi calls that I can use to determine the size of the > tables? I would like to horizontally partition the data across > multiple stores based on table size and growth estimates. > the SQL level does not open up the kernel information about storage requirements. At the MAL level several operators are available to assess storage cost. > ~Yuva > > > On 08/02/2008, Yuvaraj Athur Raghuvir <yuvaraj.a.r@gmail.com <mailto:yuvaraj.a.r@gmail.com>> wrote: > >> Finally, I have managed to get the Mbeddded example working in VC++ 2005 >> Express Edition on Windows XP ! Thanks to the folks in the mailing list. >> >> Use Case: MonetDB5 , Embedded Mode , SQL >> >> To move towards the application I intend to experiment on, I need some more >> information: >> 1) How to make data durable? In the simple example, I tried the following: >> a) First Run: Create table and insert rows. Select to check if everything >> is ok. >> b) Second Run: Only perform selection >> >>>> I get an error in the second run. >>>> >> 2) How to get meta data of the database? >> a) I want to know the size of the tables and meta tables in the data base. >> In particular I am interested in seeing the growth of the database on >> addition of content. >> b) Typically, there is a master table in the database that records >> information of the schema persisted. What is the name of the Master Table >> and its schema in MonetDB? >> c) SQLite has an independent analyzer tool which dumps the state of the >> storage from which I can extract most of the data I need. Does MonetDB have >> a similar mechanism? >> >> ~Yuva >> >> >>>>>> code snippet >>>>> >>>>>> >> dbh = embedded_sql (set, setlen); >> if (dbh == NULL || mapi_error(dbh)) >> die(dbh, hdl); >> /* switch off autocommit */ >> if (mapi_setAutocommit(dbh, 0) != MOK || mapi_error(dbh)) >> die(dbh,NULL); >> if ((hdl = mapi_query(dbh, "create table emp" >> " (name varchar(20),age int)")) == NULL || mapi_error(dbh)) >> die(dbh, hdl); >> close_handle(dbh,hdl); >> for(i=0; i< 1000; i++) { >> char query[100]; >> _snprintf(query, 100,"insert into emp values(\'user%d\', %d)", i, (i >> % 82)); >> if ((hdl = mapi_query(dbh, query)) == NULL || mapi_error(dbh)) >> die(dbh, hdl); >> close_handle(dbh,hdl); >> } >> if ((hdl = mapi_query(dbh, "select * from emp")) == NULL || >> mapi_error(dbh)) >> die(dbh, hdl); >> i=0; >> while (mapi_fetch_row(hdl)) { >> char *age = mapi_fetch_field(hdl, 1); >> i= i+ atoi(age); >> } >> if (mapi_error(dbh)) >> die(dbh, hdl); >> close_handle(dbh,hdl); >> printf("The sum is %d \n",i); >> mapi_disconnect(dbh); >> return 0; >> >> > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > MonetDB-users mailing list > MonetDB-users@lists.sourceforge.net <mailto:MonetDB-users@lists.sourceforge.net> > https://lists.sourceforge.net/lists/listinfo/monetdb-users > ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net <mailto:MonetDB-users@lists.sourceforge.net> https://lists.sourceforge.net/lists/listinfo/monetdb-users
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
I have the following calls of interest: bat.getSpaceReserved bat.getSpaceUsed bbp.getDiskSpace
Is there sample code that shows how to invoke these MAL calls and interpret the results from C/C++ code?
My search in the lib files I have used so far do not have these calls.
Do let me how to proceed here. ~Yuva
On 11/02/2008, Martin Kersten Martin.Kersten@cwi.nl wrote:
Yuvaraj Athur Raghuvir wrote:
From a C/C++ code, how do I access the MAL level storage operators? I
to find operators of interest, you may start at the appendix of the manual: http://monetdb.cwi.nl/projects/monetdb/MonetDB/Documentation/Instruction-Hel... and perform a grep/find
would assume that using the right libraries I should be able to get this data into my C/C++ code. Could you please point me to the right resources here? Thanks.
Further, I want to know the storage cost on disk for the entire database. As I see the data is stored as a "farm". Does this mean that I can sum the sizes of all files under the root folder to get the disk storage cost?
yes, that is the persistent storage requirement of the database
~Yuva
On Feb 9, 2008 12:10 AM, Martin Kersten <Martin.Kersten@cwi.nl mailto:Martin.Kersten@cwi.nl> wrote:
Yuvaraj Athur Raghuvir wrote: > Thanks. > > Regarding db size, I would expect that the storage strategy willhave
> additional overheads due to bookkeeping requirements necessary for > reconstruction of the relations between the data elements. Since there > are no indices (I assume this due to columnar storage) I candiscount
> on the space required for indices. > Indices are automatically created and dropped as a side effect of queries. They disappear at the end of a session. > Are there Mapi calls that I can use to determine the size of the > tables? I would like to horizontally partition the data across > multiple stores based on table size and growth estimates. > the SQL level does not open up the kernel information about storage requirements. At the MAL level several operators are available to assess storagecost.
> ~Yuva > > > On 08/02/2008, Yuvaraj Athur Raghuvir <yuvaraj.a.r@gmail.com <mailto:yuvaraj.a.r@gmail.com>> wrote: > >> Finally, I have managed to get the Mbeddded example working in VC++ 2005 >> Express Edition on Windows XP ! Thanks to the folks in the mailing list. >> >> Use Case: MonetDB5 , Embedded Mode , SQL >> >> To move towards the application I intend to experiment on, I need some more >> information: >> 1) How to make data durable? In the simple example, I tried the following: >> a) First Run: Create table and insert rows. Select to check if everything >> is ok. >> b) Second Run: Only perform selection >> >>>> I get an error in the second run. >>>> >> 2) How to get meta data of the database? >> a) I want to know the size of the tables and meta tables in the data base. >> In particular I am interested in seeing the growth of the database on >> addition of content. >> b) Typically, there is a master table in the database that records >> information of the schema persisted. What is the name of the Master Table >> and its schema in MonetDB? >> c) SQLite has an independent analyzer tool which dumps the state of the >> storage from which I can extract most of the data I need. Does MonetDB have >> a similar mechanism? >> >> ~Yuva >> >> >>>>>> code snippet >>>>> >>>>>> >> dbh = embedded_sql (set, setlen); >> if (dbh == NULL || mapi_error(dbh)) >> die(dbh, hdl); >> /* switch off autocommit */ >> if (mapi_setAutocommit(dbh, 0) != MOK || mapi_error(dbh)) >> die(dbh,NULL); >> if ((hdl = mapi_query(dbh, "create table emp" >> " (name varchar(20),age int)")) == NULL || mapi_error(dbh)) >> die(dbh, hdl); >> close_handle(dbh,hdl); >> for(i=0; i< 1000; i++) { >> char query[100]; >> _snprintf(query, 100,"insert into emp values(\'user%d\', %d)", i, (i >> % 82)); >> if ((hdl = mapi_query(dbh, query)) == NULL || mapi_error(dbh)) >> die(dbh, hdl); >> close_handle(dbh,hdl); >> } >> if ((hdl = mapi_query(dbh, "select * from emp")) == NULL || >> mapi_error(dbh)) >> die(dbh, hdl); >> i=0; >> while (mapi_fetch_row(hdl)) { >> char *age = mapi_fetch_field(hdl, 1); >> i= i+ atoi(age); >> } >> if (mapi_error(dbh)) >> die(dbh, hdl); >> close_handle(dbh,hdl); >> printf("The sum is %d \n",i); >> mapi_disconnect(dbh); >> return 0; >> >> > >
> This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > MonetDB-users mailing list > MonetDB-users@lists.sourceforge.net <mailto:MonetDB-users@lists.sourceforge.net> > https://lists.sourceforge.net/lists/listinfo/monetdb-users >
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net <mailto:MonetDB-users@lists.sourceforge.net> https://lists.sourceforge.net/lists/listinfo/monetdb-users
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
Yuvaraj Athur Raghuvir wrote:
I have the following calls of interest: bat.getSpaceReserved bat.getSpaceUsed bbp.getDiskSpace
Is there sample code that shows how to invoke these MAL calls and interpret the results from C/C++ code?
looking at the thread, i see that you use an embedded SQL version. This means all interactions with the kernel are through SQL statements and JDBC. Internally, the SQL compiler produces an intermediate code, called MAL, which is interpreted by the kernel. The operations mentioned above are internal MAL calls, which are bound to specific C routines which you can find out using the MonetDB help functions, e.g.
mserver5 ?bat.getSpace()
command bat.getSpaceUsed(b:bat[:any_1,:any_2]):lng address BKCgetSpaceUsed; comment Determine the total space (in bytes) occupied by a BAT.
command bat.getSpaceReserved(b:bat[:any_1,:any_2]):lng address BKCgetSpaceReserved; comment Determine the total space (in bytes) reserved for a BAT.
?bbp.getDiskSpace()
command bbp.getDiskSpace():int address CMDbbpDiskSpace; comment Estimate the amount of diskspace occupied by dbfarm
However, calling those functions directly from you C environment requires deep knowledge of the total architecture. The easiest and safest is
bbp_export str CMDbbpDiskSpace(int *ret);
the others require BAT identifiers, which you don't have under control.
My search in the lib files I have used so far do not have these calls.
Do let me how to proceed here. ~Yuva
On 11/02/2008, Martin Kersten Martin.Kersten@cwi.nl wrote:
Yuvaraj Athur Raghuvir wrote:
From a C/C++ code, how do I access the MAL level storage operators? I
to find operators of interest, you may start at the appendix of the manual: http://monetdb.cwi.nl/projects/monetdb/MonetDB/Documentation/Instruction-Hel... and perform a grep/find
would assume that using the right libraries I should be able to get this data into my C/C++ code. Could you please point me to the right resources here? Thanks.
Further, I want to know the storage cost on disk for the entire database. As I see the data is stored as a "farm". Does this mean that I can sum the sizes of all files under the root folder to get the disk storage cost?
yes, that is the persistent storage requirement of the database
~Yuva
On Feb 9, 2008 12:10 AM, Martin Kersten <Martin.Kersten@cwi.nl mailto:Martin.Kersten@cwi.nl> wrote:
Yuvaraj Athur Raghuvir wrote: > Thanks. > > Regarding db size, I would expect that the storage strategy willhave
> additional overheads due to bookkeeping requirements necessary for > reconstruction of the relations between the data elements. Since there > are no indices (I assume this due to columnar storage) I candiscount
> on the space required for indices. > Indices are automatically created and dropped as a side effect of queries. They disappear at the end of a session. > Are there Mapi calls that I can use to determine the size of the > tables? I would like to horizontally partition the data across > multiple stores based on table size and growth estimates. > the SQL level does not open up the kernel information about storage requirements. At the MAL level several operators are available to assess storagecost.
> ~Yuva > > > On 08/02/2008, Yuvaraj Athur Raghuvir <yuvaraj.a.r@gmail.com <mailto:yuvaraj.a.r@gmail.com>> wrote: > >> Finally, I have managed to get the Mbeddded example working in VC++ 2005 >> Express Edition on Windows XP ! Thanks to the folks in the mailing list. >> >> Use Case: MonetDB5 , Embedded Mode , SQL >> >> To move towards the application I intend to experiment on, I need some more >> information: >> 1) How to make data durable? In the simple example, I tried the following: >> a) First Run: Create table and insert rows. Select to check if everything >> is ok. >> b) Second Run: Only perform selection >> >>>> I get an error in the second run. >>>> >> 2) How to get meta data of the database? >> a) I want to know the size of the tables and meta tables in the data base. >> In particular I am interested in seeing the growth of the database on >> addition of content. >> b) Typically, there is a master table in the database that records >> information of the schema persisted. What is the name of the Master Table >> and its schema in MonetDB? >> c) SQLite has an independent analyzer tool which dumps the state of the >> storage from which I can extract most of the data I need. Does MonetDB have >> a similar mechanism? >> >> ~Yuva >> >> >>>>>> code snippet >>>>> >>>>>> >> dbh = embedded_sql (set, setlen); >> if (dbh == NULL || mapi_error(dbh)) >> die(dbh, hdl); >> /* switch off autocommit */ >> if (mapi_setAutocommit(dbh, 0) != MOK || mapi_error(dbh)) >> die(dbh,NULL); >> if ((hdl = mapi_query(dbh, "create table emp" >> " (name varchar(20),age int)")) == NULL || mapi_error(dbh)) >> die(dbh, hdl); >> close_handle(dbh,hdl); >> for(i=0; i< 1000; i++) { >> char query[100]; >> _snprintf(query, 100,"insert into emp values(\'user%d\', %d)", i, (i >> % 82)); >> if ((hdl = mapi_query(dbh, query)) == NULL || mapi_error(dbh)) >> die(dbh, hdl); >> close_handle(dbh,hdl); >> } >> if ((hdl = mapi_query(dbh, "select * from emp")) == NULL || >> mapi_error(dbh)) >> die(dbh, hdl); >> i=0; >> while (mapi_fetch_row(hdl)) { >> char *age = mapi_fetch_field(hdl, 1); >> i= i+ atoi(age); >> } >> if (mapi_error(dbh)) >> die(dbh, hdl); >> close_handle(dbh,hdl); >> printf("The sum is %d \n",i); >> mapi_disconnect(dbh); >> return 0; >> >> > >
> This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > MonetDB-users mailing list > MonetDB-users@lists.sourceforge.net <mailto:MonetDB-users@lists.sourceforge.net> > https://lists.sourceforge.net/lists/listinfo/monetdb-users >
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net <mailto:MonetDB-users@lists.sourceforge.net> https://lists.sourceforge.net/lists/listinfo/monetdb-users
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
On 08-02-2008 13:08:13 +0530, Yuvaraj Athur Raghuvir wrote:
To move towards the application I intend to experiment on, I need some more information:
- How to make data durable? In the simple example, I tried the
following:
[snip]
code snippet >>>>>
dbh = embedded_sql (set, setlen); if (dbh == NULL || mapi_error(dbh)) die(dbh, hdl); /* switch off autocommit */ if (mapi_setAutocommit(dbh, 0) != MOK || mapi_error(dbh)) die(dbh,NULL);
You never commit in your example, so it seems logical to me that your data isn't durable. Either don't disable auto-commit, or manually commit after each transaction.
participants (4)
-
Fabian Groffen -
Martin Kersten -
Niels Nes -
Yuvaraj Athur Raghuvir