Struct RowGroupAccessPlanFilter 

pub struct RowGroupAccessPlanFilter {
    access_plan: ParquetAccessPlan,
}

Reduces the ParquetAccessPlan based on row group level metadata.

This struct implements the various types of pruning that are applied to a set of row groups within a parquet file, progressively narrowing down the set of row groups (and ranges/selections within those row groups) that should be scanned, based on the available metadata.

Fields

access_plan: ParquetAccessPlan

Implementations

impl RowGroupAccessPlanFilter

pub fn new(access_plan: ParquetAccessPlan) -> RowGroupAccessPlanFilter

Create a new RowGroupPlanBuilder for pruning out the groups to scan based on metadata and statistics

pub fn is_empty(&self) -> bool

Return true if there are no row groups

pub fn remaining_row_group_count(&self) -> usize

Return the number of row groups that are currently expected to be scanned

pub fn build(self) -> ParquetAccessPlan

Returns the inner access plan

pub fn prune_by_range(&mut self, groups: &[RowGroupMetaData], range: &FileRange)

Prune remaining row groups to only those within the specified range.

Updates this set to mark row groups that should not be scanned

Panics

if groups.len() != self.len()

pub fn prune_by_statistics( &mut self, arrow_schema: &Schema, parquet_schema: &SchemaDescriptor, groups: &[RowGroupMetaData], predicate: &PruningPredicate, metrics: &ParquetFileMetrics, )

Prune remaining row groups using min/max/null_count statistics and the PruningPredicate to determine if the predicate can not be true.

Updates this set to mark row groups that should not be scanned

Note: This method currently ignores ColumnOrder https://github.com/apache/datafusion/issues/8335

Panics

if groups.len() != self.len()

pub async fn prune_by_bloom_filters<T>( &mut self, arrow_schema: &Schema, builder: &mut ArrowReaderBuilder<AsyncReader<T>>, predicate: &PruningPredicate, metrics: &ParquetFileMetrics, )

where T: AsyncFileReader + Send + 'static,

Prune remaining row groups using available bloom filters and the PruningPredicate.

Updates this set with row groups that should not be scanned

Panics

if the builder does not have the same number of row groups as this set

Trait Implementations

impl Clone for RowGroupAccessPlanFilter

fn clone(&self) -> RowGroupAccessPlanFilter

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for RowGroupAccessPlanFilter

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl PartialEq for RowGroupAccessPlanFilter

fn eq(&self, other: &RowGroupAccessPlanFilter) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for RowGroupAccessPlanFilter