Struct PartialSortStream 

struct PartialSortStream {
    input: SendableRecordBatchStream,
    expr: LexOrdering,
    common_prefix_length: usize,
    in_mem_batch: RecordBatch,
    fetch: Option<usize>,
    is_closed: bool,
    baseline_metrics: BaselineMetrics,
}

Fields

input: SendableRecordBatchStream

The input plan

expr: LexOrdering

Sort expressions

common_prefix_length: usize

Length of prefix common to input ordering and required ordering of plan should be more than 0 otherwise PartialSort is not applicable

in_mem_batch: RecordBatch

Used as a buffer for part of the input not ready for sort

fetch: Option<usize>

Fetch top N results

is_closed: bool

Whether the stream has finished returning all of its data or not

baseline_metrics: BaselineMetrics

Execution metrics

Implementations

impl PartialSortStream

fn poll_next_inner( self: &mut Pin<&mut Self>, cx: &mut Context<'_>, ) -> Poll<Option<Result<RecordBatch>>>

fn sort_in_mem_batch(self: &mut Pin<&mut Self>) -> Result<RecordBatch>

Returns a sorted RecordBatch from in_mem_batches and clears in_mem_batches

If fetch is specified for PartialSortStream sort_in_mem_batch will limit the last RecordBatch returned and will mark the stream as closed

fn get_slice_point( &self, common_prefix_len: usize, batch: &RecordBatch, ) -> Result<Option<usize>>

Return the end index of the second last partition if the batch can be partitioned based on its already sorted columns

Return None if the batch cannot be partitioned, which means the batch does not have the information for a safe sort

Trait Implementations

impl RecordBatchStream for PartialSortStream

fn schema(&self) -> SchemaRef

Returns the schema of this RecordBatchStream.

impl Stream for PartialSortStream

type Item = Result<RecordBatch, DataFusionError>

Values yielded by the stream.

fn poll_next( self: Pin<&mut Self>, cx: &mut Context<'_>, ) -> Poll<Option<Self::Item>>

Attempt to pull out the next value of this stream, registering the current task for wakeup if the value is not yet available, and returning None if the stream is exhausted.

fn size_hint(&self) -> (usize, Option<usize>)

Returns the bounds on the remaining length of the stream.