Struct ParquetFileMetrics 

pub struct ParquetFileMetrics {

    pub files_ranges_pruned_statistics: PruningMetrics,
    pub predicate_evaluation_errors: Count,
    pub row_groups_pruned_bloom_filter: PruningMetrics,
    pub row_groups_pruned_statistics: PruningMetrics,
    pub bytes_scanned: Count,
    pub pushdown_rows_pruned: Count,
    pub pushdown_rows_matched: Count,
    pub row_pushdown_eval_time: Time,
    pub statistics_eval_time: Time,
    pub bloom_filter_eval_time: Time,
    pub page_index_rows_pruned: PruningMetrics,
    pub page_index_eval_time: Time,
    pub metadata_load_time: Time,
    pub predicate_cache_inner_records: Count,
    pub predicate_cache_records: Count,
}

Stores metrics about the parquet execution for a particular parquet file.

This component is a subject to change in near future and is exposed for low level integrations through ParquetFileReaderFactory.

Fields

files_ranges_pruned_statistics: PruningMetrics

Number of file ranges pruned or matched by partition or file level statistics. Pruning of files often happens at planning time but may happen at execution time if dynamic filters (e.g. from a join) result in additional pruning.

This does not necessarily equal the number of files pruned: files may be scanned in sub-ranges to increase parallelism, in which case this will represent the number of sub-ranges pruned, not the number of files. The number of files pruned will always be less than or equal to this number.

A single file may have some ranges that are not pruned and some that are pruned. For example, with a query like ORDER BY col LIMIT 10, the TopK dynamic filter pushdown optimization may fill up the TopK heap when reading the first part of a file, then skip the second part if file statistics indicate it cannot contain rows that would be in the TopK.

predicate_evaluation_errors: Count

Number of times the predicate could not be evaluated

row_groups_pruned_bloom_filter: PruningMetrics

Number of row groups whose bloom filters were checked, tracked with matched/pruned counts

row_groups_pruned_statistics: PruningMetrics

Number of row groups whose statistics were checked, tracked with matched/pruned counts

bytes_scanned: Count

Total number of bytes scanned

pushdown_rows_pruned: Count

Total rows filtered out by predicates pushed into parquet scan

pushdown_rows_matched: Count

Total rows passed predicates pushed into parquet scan

row_pushdown_eval_time: Time

Total time spent evaluating row-level pushdown filters

statistics_eval_time: Time

Total time spent evaluating row group-level statistics filters

bloom_filter_eval_time: Time

Total time spent evaluating row group Bloom Filters

page_index_rows_pruned: PruningMetrics

Total rows filtered or matched by parquet page index

page_index_eval_time: Time

Total time spent evaluating parquet page index filters

metadata_load_time: Time

Total time spent reading and parsing metadata from the footer

predicate_cache_inner_records: Count

Predicate Cache: number of records read directly from the inner reader. This is the number of rows decoded while evaluating predicates

predicate_cache_records: Count

Predicate Cache: number of records read from the cache. This is the number of rows that were stored in the cache after evaluating predicates reused for the output.

Implementations

impl ParquetFileMetrics

pub fn new( partition: usize, filename: &str, metrics: &ExecutionPlanMetricsSet, ) -> Self

Create new metrics

Trait Implementations

impl Clone for ParquetFileMetrics

fn clone(&self) -> ParquetFileMetrics

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ParquetFileMetrics

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.