Struct GroupValuesRows 

pub struct GroupValuesRows {
    schema: SchemaRef,
    row_converter: RowConverter,
    map: HashTable<(u64, usize)>,
    map_size: usize,
    group_values: Option<Rows>,
    hashes_buffer: Vec<u64>,
    rows_buffer: Rows,
    random_state: RandomState,
}

A GroupValues making use of [Rows]

This is a general implementation of GroupValues that works for any combination of data types and number of columns, including nested types such as structs and lists.

It uses the arrow-rs [Rows] to store the group values, which is a row-wise representation.

Fields

schema: SchemaRef

The output schema

row_converter: RowConverter

Converter for the group values

map: HashTable<(u64, usize)>

Logically maps group values to a group_index in Self::group_values and in each accumulator

Uses the raw API of hashbrown to avoid actually storing the keys (group values) in the table

keys: u64 hashes of the GroupValue values: (hash, group_index)

map_size: usize

The size of map in bytes

group_values: Option<Rows>

The actual group by values, stored in arrow Row format. group_values[i] holds the group value for group_index i.

The row format is used to compare group keys quickly and store them efficiently in memory. Quick comparison is especially important for multi-column group keys.

hashes_buffer: Vec<u64>

reused buffer to store hashes

rows_buffer: Rows

reused buffer to store rows

random_state: RandomState

Random state for creating hashes

Implementations

impl GroupValuesRows

pub fn try_new(schema: SchemaRef) -> Result<Self>

Trait Implementations

impl GroupValues for GroupValuesRows

fn intern(&mut self, cols: &[ArrayRef], groups: &mut Vec<usize>) -> Result<()>

Calculates the group id for each input row of cols, assigning new group ids as necessary.

fn size(&self) -> usize

Returns the number of bytes of memory used by this GroupValues

fn is_empty(&self) -> bool

Returns true if this GroupValues is empty

fn len(&self) -> usize

The number of values (distinct group values) stored in this GroupValues

fn emit(&mut self, emit_to: EmitTo) -> Result<Vec<ArrayRef>>

Emits the group values

fn clear_shrink(&mut self, batch: &RecordBatch)

Clear the contents and shrink the capacity to the size of the batch (free up memory usage)