The major feature release of June2016 is behind us, and now it’s time to look ahead for the next one. The list of topics looked into and reported on Bugzilla covers the full range of functional SQL enhancements, quality assurance, kernel algorithms, code cleaning, and packaging. Too much to tackle all at once. Therefore, our prioritized list contains the following topics for consideration of inclusion in the next feature release.
We will extend the SQL functionality where appropriate for our research purposes and by voiced requests from the community. In particular, we aim for support for renaming tables/columns and for UPSERT soon.
Efficient data ingestion is by large a prime feature of a DBMS. The textual approach based on CSV is fairly well defined, but there are also many bad CSV sources, and hence many ways in which the CSV file might not be compatible with our parser. A simpler API is required, which MonetDB already provides in the form of the Binary COPY INTO. This allows for client programs to produce the binary column representation. Subsequent inclusion into the database then merely consists of taking a copy of this binary version.
We plan to apply this binary column API all the way, removing most of the rendering/parsing overhead of the client-server protocol in handling result sets. The client side takes more responsibility and thus reduces the server CPU load.
A swiss-army knife for loading is the recently announced Python loader extension. It is a tightly integrated, open-ended solution for loading data from any external source. Loading regular JSON data has never been simpler.
The MonetDB optimizer stack will be extended to better serve ad-hoc queries. The current approach to convert every SQL query into a MAL function, by turning literal constants into parameters, is dropped for ad-hoc queries. Optimization will take place just before execution, which gives a better handle on exploiting the query literals, empty columns, and possibly better use of intermediates.
Quality assurance and upgrading stability is a key priority for us. With the advent of running MonetDB within paid-for virtual machines, the system may experience situations with too tight resource settings. Steps are taken to further reduce (tiny) memory leaks and to increase robustness against failing mallocs.
Last but not least, we plan to install soon a daily updated, read-only mirror of MonetDB in GitHub, which should simplify the life for developers preferring this repository infrastructure.
Much of our effort to improve MonetDB is grounded in and paid for by research projects, which by their very nature call for publications rather than code. So far we managed to maintain a well-appreciated system, used both for research, education, and business settings. Secured research funding permits us to continue our support for this open-source project for many years to come at a leisure pace. Urgent issues and feature requests are handled through MonetDB Solutions.
We appreciate all feedback about the use of the MonetDB system.