Struct FilterCandidateBuilder 

struct FilterCandidateBuilder {
    expr: Arc<dyn PhysicalExpr>,
    file_schema: SchemaRef,
    table_schema: SchemaRef,
    schema_adapter_factory: Arc<dyn SchemaAdapterFactory>,
}

Helper to build a FilterCandidate.

This will do several things

1. Determine the columns required to evaluate the expression
2. Calculate data required to estimate the cost of evaluating the filter
3. Rewrite column expressions in the predicate which reference columns not in the particular file schema.

Schema Rewrite

When parquet files are read in the context of “schema evolution” there are potentially wo schemas:

1. The table schema (the columns of the table that the parquet file is part of)
2. The file schema (the columns actually in the parquet file)

There are times when the table schema contains columns that are not in the file schema, such as when new columns have been added in new parquet files but old files do not have the columns.

When a file is missing a column from the table schema, the value of the missing column is filled in by a SchemaAdapter (by default as NULL).

When a predicate is pushed down to the parquet reader, the predicate is evaluated in the context of the file schema. For each predicate we build a filter schema which is the projection of the table schema that contains only the columns that this filter references. If any columns from the file schema are missing from a particular file they are added by the SchemaAdapter, by default as NULL.

Fields

expr: Arc<dyn PhysicalExpr>

file_schema: SchemaRef

The schema of this parquet file. Columns may have different types from the table schema and there may be columns in the file schema that are not in the table schema or columns that are in the table schema that are not in the file schema.

table_schema: SchemaRef

The schema of the table (merged schema) – columns may be in different order than in the file and have columns that are not in the file schema

schema_adapter_factory: Arc<dyn SchemaAdapterFactory>

A SchemaAdapterFactory used to map the file schema to the table schema.

Implementations

impl FilterCandidateBuilder

pub fn new( expr: Arc<dyn PhysicalExpr>, file_schema: Arc<Schema>, table_schema: Arc<Schema>, schema_adapter_factory: Arc<dyn SchemaAdapterFactory>, ) -> Self

pub fn build( self, metadata: &ParquetMetaData, ) -> Result<Option<FilterCandidate>>

Attempt to build a FilterCandidate from the expression

Return values

• Ok(Some(candidate)) if the expression can be used as an ArrowFilter
• Ok(None) if the expression cannot be used as an ArrowFilter
• Err(e) if an error occurs while building the candidate