Extract Table Statistics after Data import using Monet.Read.Csv
Hello,
I am trying to import a very large dataset of approx.. 170 million records.
The dataset is divided into have 12 tables of approx. 2 GB each. In order to recreate the entire dataset, I want to use the MERGE function to merge all the 12 tables into one dataset for further analysis.
However, when I try to add the table to merge table, it gives an error of type mismatch in columns.
Can you please help me how to determine the column type after a ‘csv’ file has been imported using monet.red.csv ?
I would like to use the same R interface to MonetDB as I am still learning MonetDB and not conversant with the DB.
Thanks & Regards, Jigar Shah Sent from Mail for Windows 10
Hi Jigar,
On 29.12.2015, at 01:21, Jigar Shah jigar.shah1984@hotmail.com wrote:
The dataset is divided into have 12 tables of approx. 2 GB each. In order to recreate the entire dataset, I want to use the MERGE function to merge all the 12 tables into one dataset for further analysis.
You can pass a vector of _all_ your file names to monetdb.read.csv(), this will load all of them into a new table and does not require the merge. The different schemas come from the function guessing the column types from the file.
Hannes
for an example of this, see
https://github.com/ajdamico/asdfree/tree/master/Basic%20Stand%20Alone%20Medi... On Dec 29, 2015 2:50 AM, "Hannes Mühleisen" hannes.muehleisen@cwi.nl wrote:
Hi Jigar,
On 29.12.2015, at 01:21, Jigar Shah jigar.shah1984@hotmail.com wrote:
The dataset is divided into have 12 tables of approx. 2 GB each. In
order to recreate the entire dataset, I want to use the MERGE function to merge all the 12 tables into one dataset for further analysis.
You can pass a vector of _all_ your file names to monetdb.read.csv(), this will load all of them into a new table and does not require the merge. The different schemas come from the function guessing the column types from the file.
Hannes
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
participants (3)
-
Anthony Damico -
Hannes Mühleisen -
Jigar Shah