Peter,
I would actually like reimplement the MIL primitives in Lisp. I cannot implement trading systems in MIL and writing a Lisp kernel on top of the GDK would let me run trading systems in the engine, without the overhead of pushing data around through ODBC, MAPI, etc.
I don't want to be limited by main memory so I guess I'm up against implementing X100 but on a far smaller scale :-).
There's still a lot to be reused from the GDK since it implements the very basic primitives (join, aggregate, select, etc.). From what I'm reading X100 seems to use fragmented BATs to avoid memory limitations which makes me think that I could do the same.
My basic goal is to store market data and process it efficiently. Most of the storing is appending to BATs where the [void,timestamp] bat is sorted by timestamp (inverted?). Most of the processing is running trading strategies over one or many BATs at the same time (e.g. price, volume of MSFT and IBM), using a window of X BUNs and sliding that window forward one BUN at a time.
Finally, I would like to do similarity matching of the different time series. That's icing on the cake and I can figure out how to do it later. There's a lot of similarity between what MonetDB is doing and what I need to do, I'm just working with Lisp.
What do you think?
Thanks, Joel
On Jul 30, 2005, at 8:39 PM, Peter Boncz wrote:
Then more general. I just read your posts on this mailing list and I do not have a full grasp of your plans in order to properly advise you.