Blogs

Martin Kersten presented his idea to cope with the ever increasing big data growth in a keynote at the 32nd IEEE International Conference on Data Engineering, May 16-20, 2016, in Helsinki, Finland.

MonetDB is used for doing database research, but is also widely used in production for a large set of different applications. One of them is astronomy. The fact that MonetDB is a column store makes it very well suited for the large analytical workloads of modern astronomy. So it is not surprising that MonetDB was chosen to be part of the software pipeline of the LOFAR radio telescope. In the previous blogpost "Time-domain radio astronomy with MonetDB" by Bart Scheers, a more detailed explanation can be found.

The international low-frequency radio telescope LOFAR (located in The Netherlands) is one of the first telescopes to completely integrate observations with real-time computation and data storage facilities in its overall design. Signals received by thousands of antennas are locally pre-processed and digitised before they are transported over a 10Gb/s link to a remote supercomputer. There, the raw data is processed further, e.g., imaged, after which dedicated software pipelines pick up the calibrated data again to do their science.

In a previous blogpost we introduced MonetDB/Python. Using MonetDB/Python, users can execute their own vectorized Python functions within MonetDB without having to worry about slow data transfer. In this post we only really showcased simple Python functions, such as computing the quantile or summing up a set of integers. We don't really need Python UDFs to do these simple operations. We can easily do them using SQL as well.

MonetDB has a Ruby connector in its collection of supported clients since 2008. Recently we have begun to further improve  this client and made it compatible with Ruby versions 2.0 and above. These improvements are already part of the Jul2015-SP2 release.