[MonetDB-users] column sortedness
I'm running some experiments with a 1B row fact table (10 columns).
- How can I tell if a column is sorted?
- If I generate files in sorted order and use binary bulk loading, will monetdb consider the column sorted? If not, can I force it?
I've been reading code, but understanding all the assignments to bn->tsorted is going to take a while. I get the general theme of setting tsorted during bat operations, but I've been unsuccessful in finding where it is initialized when a column is first accessed.
TIA, -david
Bump
On Thu, Dec 29, 2011 at 9:09 PM, David Pennell dpennell@good-cloud.comwrote:
I'm running some experiments with a 1B row fact table (10 columns).
How can I tell if a column is sorted?
If I generate files in sorted order and use binary bulk loading, will
monetdb consider the column sorted? If not, can I force it?
I've been reading code, but understanding all the assignments to bn->tsorted is going to take a while. I get the general theme of setting tsorted during bat operations, but I've been unsuccessful in finding where it is initialized when a column is first accessed.
TIA, -david
In general, the sortedness property for arbitrary (intermediate) columns can only be checked using MAL to ask for the properties.
The sortedness is determined at runtime amongst others using function BATpropcheck() and indeed the property is propagated in many places.
In the default branch, the SQL storage() function has been extended to show this property.
A sorted input file copied into MonetDB leads to a sorted table representation.
regards, Martin
On 1/13/12 1:22 AM, David Pennell wrote:
Bump
On Thu, Dec 29, 2011 at 9:09 PM, David Pennell <dpennell@good-cloud.com mailto:dpennell@good-cloud.com> wrote:
I'm running some experiments with a 1B row fact table (10 columns). - How can I tell if a column is sorted? - If I generate files in sorted order and use binary bulk loading, will monetdb consider the column sorted? If not, can I force it? I've been reading code, but understanding all the assignments to bn->tsorted is going to take a while. I get the general theme of setting tsorted during bat operations, but I've been unsuccessful in finding where it is initialized when a column is first accessed. TIA, -david
RSA(R) Conference 2012 Mar 27 - Feb 2 Save $400 by Jan. 27 Register now! http://p.sf.net/sfu/rsa-sfdev2dev2
MonetDB-users mailing list MonetDB-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-users
participants (2)
-
David Pennell -
Martin Kersten