Struct GroupValuesPrimitive 

pub struct GroupValuesPrimitive<T: ArrowPrimitiveType> {
    data_type: DataType,
    map: HashTable<(usize, u64)>,
    null_group: Option<usize>,
    values: Vec<T::Native>,
    random_state: RandomState,
}

A GroupValues storing a single column of primitive values

This specialization is significantly faster than using the more general purpose Rows format

Fields

data_type: DataType

The data type of the output array

map: HashTable<(usize, u64)>

Stores the (group_index, hash) based on the hash of its value

We also store hash is for reducing cost of rehashing. Such cost is obvious in high cardinality group by situation. More details can see: https://github.com/apache/datafusion/issues/15961

null_group: Option<usize>

The group index of the null value if any

values: Vec<T::Native>

The values for each group index

random_state: RandomState

The random state used to generate hashes

Implementations

impl<T: ArrowPrimitiveType> GroupValuesPrimitive<T>

pub fn new(data_type: DataType) -> Self

Trait Implementations

impl<T: ArrowPrimitiveType> GroupValues for GroupValuesPrimitive<T>

where T::Native: HashValue,

fn intern(&mut self, cols: &[ArrayRef], groups: &mut Vec<usize>) -> Result<()>

Calculates the group id for each input row of cols, assigning new group ids as necessary.

fn size(&self) -> usize

Returns the number of bytes of memory used by this GroupValues

fn is_empty(&self) -> bool

Returns true if this GroupValues is empty

fn len(&self) -> usize

The number of values (distinct group values) stored in this GroupValues

fn emit(&mut self, emit_to: EmitTo) -> Result<Vec<ArrayRef>>

Emits the group values

fn clear_shrink(&mut self, batch: &RecordBatch)

Clear the contents and shrink the capacity to the size of the batch (free up memory usage)