Struct BloomFilterStatistics 

struct BloomFilterStatistics {
    column_sbbf: HashMap<String, (Sbbf, Type)>,
}

Implements PruningStatistics for Parquet Split Block Bloom Filters (SBBF)

Fields

column_sbbf: HashMap<String, (Sbbf, Type)>

Maps column name to the parquet bloom filter and parquet physical type

Implementations

impl BloomFilterStatistics

fn check_scalar(sbbf: &Sbbf, value: &ScalarValue, parquet_type: &Type) -> bool

Helper function for checking if [Sbbf] filter contains ScalarValue.

In case the type of scalar is not supported, returns true, assuming that the value may be present.

Trait Implementations

impl PruningStatistics for BloomFilterStatistics

fn contained( &self, column: &Column, values: &HashSet<ScalarValue>, ) -> Option<BooleanArray>

Use bloom filters to determine if we are sure this column can not possibly contain values

The contained API returns false if the bloom filters knows that ALL of the values in a column are not present.

fn min_values(&self, _column: &Column) -> Option<ArrayRef>

Return the minimum values for the named column, if known.

fn max_values(&self, _column: &Column) -> Option<ArrayRef>

Return the maximum values for the named column, if known.

fn num_containers(&self) -> usize

Return the number of containers (e.g. Row Groups) being pruned with these statistics.

fn null_counts(&self, _column: &Column) -> Option<ArrayRef>

Return the number of null values for the named column as an UInt64Array

fn row_counts(&self, _column: &Column) -> Option<ArrayRef>

Return the number of rows for the named column in each container as an UInt64Array.