Struct StreamedBatch 

pub(super) struct StreamedBatch {
    pub batch: RecordBatch,
    pub idx: usize,
    pub join_arrays: Vec<ArrayRef>,
    pub output_indices: Vec<StreamedJoinedChunk>,
    pub buffered_batch_idx: Option<usize>,
    pub join_filter_matched_idxs: HashSet<u64>,
}

Represents a record batch from streamed input.

Also stores information of matching rows from buffered batches.

Fields

batch: RecordBatch

The streamed record batch

idx: usize

The index of row in the streamed batch to compare with buffered batches

join_arrays: Vec<ArrayRef>

The join key arrays of streamed batch which are used to compare with buffered batches and to produce output. They are produced by evaluating on expressions.

output_indices: Vec<StreamedJoinedChunk>

Chunks of indices from buffered side (may be nulls) joined to streamed

buffered_batch_idx: Option<usize>

Index of currently scanned batch from buffered data

join_filter_matched_idxs: HashSet<u64>

Indices that found a match for the given join filter Used for semi joins to keep track the streaming index which got a join filter match and already emitted to the output.

Implementations

impl StreamedBatch

fn new(batch: RecordBatch, on_column: &[Arc<dyn PhysicalExpr>]) -> Self

fn new_empty(schema: SchemaRef) -> Self

fn append_output_pair( &mut self, buffered_batch_idx: Option<usize>, buffered_idx: Option<usize>, )

Appends new pair consisting of current streamed index and buffered_idx index of buffered batch with buffered_batch_idx index.