Struct BoundedWindowAggStream 

pub struct BoundedWindowAggStream {
    schema: SchemaRef,
    input: SendableRecordBatchStream,
    input_buffer: RecordBatch,
    partition_buffers: PartitionBatches,
    window_agg_states: Vec<PartitionWindowAggStates>,
    finished: bool,
    window_expr: Vec<Arc<dyn WindowExpr>>,
    baseline_metrics: BaselineMetrics,
    search_mode: Box<dyn PartitionSearcher>,
}

Stream for the bounded window aggregation plan.

Fields

schema: SchemaRef

input: SendableRecordBatchStream

input_buffer: RecordBatch

The record batch executor receives as input (i.e. the columns needed while calculating aggregation results).

partition_buffers: PartitionBatches

We separate input_buffer based on partitions (as determined by PARTITION BY columns) and store them per partition in partition_batches. We use this variable when calculating results for each window expression. This enables us to use the same batch for different window expressions without copying.

window_agg_states: Vec<PartitionWindowAggStates>

An executor can run multiple window expressions if the PARTITION BY and ORDER BY sections are same. We keep state of the each window expression inside window_agg_states.

finished: bool

window_expr: Vec<Arc<dyn WindowExpr>>

baseline_metrics: BaselineMetrics

search_mode: Box<dyn PartitionSearcher>

Search mode for partition columns. This determines the algorithm with which we group each partition.

Implementations

impl BoundedWindowAggStream

fn prune_state(&mut self, n_out: usize) -> Result<()>

Prunes sections of the state that are no longer needed when calculating results (as determined by window frame boundaries and number of results generated).

impl BoundedWindowAggStream

fn new( schema: SchemaRef, window_expr: Vec<Arc<dyn WindowExpr>>, input: SendableRecordBatchStream, baseline_metrics: BaselineMetrics, search_mode: Box<dyn PartitionSearcher>, ) -> Result<Self>

Create a new BoundedWindowAggStream

fn compute_aggregates(&mut self) -> Result<Option<RecordBatch>>

fn poll_next_inner( &mut self, cx: &mut Context<'_>, ) -> Poll<Option<Result<RecordBatch>>>

fn prune_partition_batches(&mut self)

Prunes the sections of the record batch (for each partition) that we no longer need to calculate the window function result.

fn prune_input_batch(&mut self, n_out: usize) -> Result<()>

Prunes the section of the input batch whose aggregate results are calculated and emitted.

fn prune_out_columns(&mut self)

Prunes emitted parts from WindowAggState out_col field.

Trait Implementations

impl RecordBatchStream for BoundedWindowAggStream

fn schema(&self) -> SchemaRef

Get the schema

impl Stream for BoundedWindowAggStream

type Item = Result<RecordBatch, DataFusionError>

Values yielded by the stream.

fn poll_next( self: Pin<&mut Self>, cx: &mut Context<'_>, ) -> Poll<Option<Self::Item>>

Attempt to pull out the next value of this stream, registering the current task for wakeup if the value is not yet available, and returning None if the stream is exhausted.

fn size_hint(&self) -> (usize, Option<usize>)

Returns the bounds on the remaining length of the stream.