Iterators mk Tue, 03/30/2010 - 16:23

BAT Iterators

Many low level algorithms rely on an iterator to break a collection into smaller pieces. Each piece is subsequently processed by a block. For very large BATs it may make sense to break it into chunks and process them separately to solve a query. An iterator pair is provided to chop a BAT into fixed size elements. Each chunk is made available as a BATview. It provides read-only access to an underlying BAT. Adjusting the bounds is cheap, once the BATview descriptor has been constructed.

The smallest granularity is a single (oid,value) pair, which can be used to realize an iterator over the individual BAT elements. For larger sized chunks, the operators return a view.

All iterators require storage space to administer the location of the next element. The BAT iterator module uses a simple lng variable, which also acts as a cursor for barrier statements.

The larger chunks produced are currently static, i.e. their size is a parameter of the call. Dynamic chunk sizes are interesting for time-series query processing. (See another module)

module iterator;

command new(b:bat[:oid,:any_2], size:lng) (:lng,:bat[:oid,:any_2])
address ITRnewChunk
comment "Create an iterator with fixed granule size. The result is a view.";

command next(b:bat[:oid,:any_2], size:lng) (:lng, :bat[:oid,:any_2])
address ITRnextChunk
comment "Produce the next chunk for processing.";

pattern new(b:bat[:oid,:any_2]) (h:oid, t:any_2)
address ITRbunIterator
comment "Process the buns one by one extracted from a void table.";

pattern next(b:bat[:oid,:any_2]) (h:oid, t:any_2)
address ITRbunNext
comment "Produce the next bun for processing.";

command next(step:oid,last:oid):oid
address ITRnext_oid;
command next(step:sht,last:sht):sht
address ITRnext_sht;
command next(step:int,last:int):int
address ITRnext_int;
command next(step:lng,last:lng):lng
address ITRnext_lng;
command next(step:flt,last:flt):flt
address ITRnext_flt;
command next(step:dbl,last:dbl):dbl
address ITRnext_dbl
comment "Advances the iterator with a fixed value";