Struct PartitionPruningStatistics 

pub struct PartitionPruningStatistics {
    partition_values: Vec<ArrayRef>,
    num_containers: usize,
    partition_schema: SchemaRef,
}

Prune files based on their partition values.

This is used both at planning time and execution time to prune files based on their partition values. This feeds into CompositePruningStatistics to allow pruning with filters that depend both on partition columns and data columns (e.g. WHERE partition_col = data_col).

Fields

partition_values: Vec<ArrayRef>

Values for each column for each container.

The outer vectors represent the columns while the inner vectors represent the containers. The order must match the order of the partition columns in PartitionPruningStatistics::partition_schema.

num_containers: usize

The number of containers.

Stored since the partition values are column-major and if there are no columns we wouldn’t know the number of containers.

partition_schema: SchemaRef

The schema of the partition columns.

This must not be the schema of the entire file or table: it must only be the schema of the partition columns, in the same order as the values in PartitionPruningStatistics::partition_values.

Implementations

impl PartitionPruningStatistics

pub fn try_new( partition_values: Vec<Vec<ScalarValue>>, partition_fields: Vec<FieldRef>, ) -> Result<Self, DataFusionError>

Create a new instance of PartitionPruningStatistics.

Args:

• partition_values: A vector of vectors of ScalarValues. The outer vector represents the containers while the inner vector represents the partition values for each column. Note that this is the opposite of the order of the partition columns in PartitionPruningStatistics::partition_schema.
• partition_schema: The schema of the partition columns. This must not be the schema of the entire file or table: instead it must only be the schema of the partition columns, in the same order as the values in partition_values.

Trait Implementations

impl Clone for PartitionPruningStatistics

fn clone(&self) -> PartitionPruningStatistics

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl PruningStatistics for PartitionPruningStatistics

fn min_values(&self, column: &Column) -> Option<ArrayRef>

Return the minimum values for the named column, if known.

fn max_values(&self, column: &Column) -> Option<ArrayRef>

Return the maximum values for the named column, if known.

fn num_containers(&self) -> usize

Return the number of containers (e.g. Row Groups) being pruned with these statistics.

fn null_counts(&self, _column: &Column) -> Option<ArrayRef>

Return the number of null values for the named column as an UInt64Array

fn row_counts(&self, _column: &Column) -> Option<ArrayRef>

Return the number of rows for the named column in each container as an UInt64Array.

fn contained( &self, column: &Column, values: &HashSet<ScalarValue>, ) -> Option<BooleanArray>

Returns [BooleanArray] where each row represents information known about specific literal values in a column.