My 5 cents: I suspect in the context the UPSERT is misleading. What you need is rather a partitioning functionality in terms of what Oracle has. You need to break the data in to multiple partitions (based on whatever criteria as for example date range, ip range, etc). The DML/query is (that is in Oracle) transparent so you do not care where (ie what partition) the data goes/is physically but the database automatically assigns the data into a correct partition. Of course you have multiple options how treat the particular partition data from the performance perspective - you may want to refresh for example only last month of data. Now if you partition by month you can truncate only one month of data and load it again etc.
I really miss this in MonetDB.
Of course i may be wrong :)
milan On Apr 17, 2015 11:11 AM, "Ying Zhang" Y.Zhang@cwi.nl wrote:
Hello Shamsul,
On Apr 16, 2015, at 15:47, shamsul hassan shamsulbuddy@gmail.com
wrote:
there is only one thing which has stopped me for using MonetDB in
production is the lack of UPSERT operation.
In my case new data comes every 30 mins
How many data (in #records or size) do you receive?
and I cannot truncate and load the whole table again.
What has "truncate and load the whole table" to do with workaround the lack of UPSERT?
Ideally an UPSERT would have helped here as it will only touch those
records which are new or changed without truncating the whole table.
In case of new records, UPSERT is a syntactical convenience for, e.g., load the arrived records in a temp table, compute an except against the base table, and insert the results into base table.
In case of changed records, you probably have some customised definition of "changed" that calls for some computation to identify and update them?
I raised the question few days before in Community list but didn't get
any useful answer.
I probably overlooked it, but I don't recall any e-mail on the mailing list on the UPSERT topic in the last days. I do remember some truncate table discussions quite some time ago...
If i get any workaround for these UPSERTS the i would be as happy as pig
in dirty water :)
Please be aware that column-stores such as MonetDB has some fundamental differences than the row-stores you are probably very familiar with. And MonetDB has a rather different transaction scheme than many other DBMSs. Therefore, some advantages using UPSERT has in other DBMSs (such as the ones mentioned in https://wiki.postgresql.org/wiki/UPSERT#.22UPSERT.22_definition), may not apply in MonetDB.
With kind regards,
Jennie
Thanks
On Thu, Apr 16, 2015 at 2:37 PM, Ruben Silva <
ruben.silva@cortex-intelligence.com> wrote:
Hello Luciano,
I don't have a full grasp of the new stuff introduced into MonetDB in
the last year, but as far as I know there are still major differences between the two systems for me the most important is scalability. Vertica has no single point of failure when working as a cluster. However the CE version is limited to three nodes, so scalability is also limited in that case.
Vertica relies heavily on projections as a way to improve performance.
If your data model is relatively stable it is something fairly easy to use, but if your data model changes at a high rate you will have a considerable amount of additional work managing the projections. Also if you use many projections they will hurt your DML statements performance.
MonetDB feeds on CPU and has no boundaries for its appetite, consuming
as much resources as possible in order to respond to a single statement. That can be very good or very bad, it depends on your use case. Vertica has resource pools that are used to regulate the stuff (IO, Ram, CPU) that each query is allowed to use.
I don't know your use case (and that is crucial for a good advice), but
if you are considering a scenario where you have a single machine for the database system, then go with MonetDB without any doubt. It is just faster and simpler.
Cumprimentos (Regards),
2015-04-16 14:04 GMT+01:00 Luciano Sasso luciano@gsgroup.com.br: Hi,
The team of staff this questioning me about using Vertica, but it has
limitations on
the CE version. Someone already used the Vertica has some advantage over
MonetDB?
-- <Slice_01.png> <Slice_02.jpg> Luciano Sasso Vieira Data Scientist & Solutions Architect
luciano@gsgroup.com.br | tel: 17 3353-0833 | cel: 17 99706-9335 <Slice_04.png> www.gsgroup.com.br <Slice_06.png>
Este email foi escaneado pelo Avast antivírus. www.avast.com
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list
users-list mailing list users-list@monetdb.org https://www.monetdb.org/mailman/listinfo/users-list