Struct BatchSplitter 

pub(crate) struct BatchSplitter {
    batch: Option<RecordBatch>,
    batch_size: usize,
    row_index: usize,
}

Splits large batches into smaller batches with a maximum number of rows.

Fields

batch: Option<RecordBatch>

RecordBatch to be split

batch_size: usize

Maximum number of rows in a split batch

row_index: usize

Current row index

Implementations

impl BatchSplitter

pub(crate) fn new(batch_size: usize) -> Self

Creates a new BatchSplitter with the specified batch size.

Trait Implementations

impl BatchTransformer for BatchSplitter

fn set_batch(&mut self, batch: RecordBatch)

Sets the next RecordBatch to be processed.

fn next(&mut self) -> Option<(RecordBatch, bool)>

Retrieves the next RecordBatch from the transformer. Returns None if all batches have been produced. The boolean flag indicates whether the batch is the last one.

impl Clone for BatchSplitter

fn clone(&self) -> BatchSplitter

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for BatchSplitter

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.