Struct SchemaMapping 

pub struct SchemaMapping {
    projected_table_schema: SchemaRef,
    field_mappings: Vec<Option<usize>>,
    cast_column: Arc<CastColumnFn>,
}

The SchemaMapping struct holds a mapping from the file schema to the table schema and any necessary type conversions.

map_batch is used by the ParquetOpener to produce a RecordBatch which has the projected schema, since that’s the schema which is supposed to come out of the execution of this query. Thus map_batch uses projected_table_schema as it can only operate on the projected fields.

Fields

projected_table_schema: SchemaRef

The schema of the table. This is the expected schema after conversion and it should match the schema of the query result.

field_mappings: Vec<Option<usize>>

Mapping from field index in projected_table_schema to index in projected file_schema.

They are Options instead of just plain usizes because the table could have fields that don’t exist in the file.

cast_column: Arc<CastColumnFn>

Function used to adapt a column from the file schema to the table schema when it exists in both schemas

Implementations

impl SchemaMapping

pub fn new( projected_table_schema: SchemaRef, field_mappings: Vec<Option<usize>>, cast_column: Arc<CastColumnFn>, ) -> Self

Creates a new SchemaMapping instance

Initializes the field mappings needed to transform file data to the projected table schema

Trait Implementations

impl Debug for SchemaMapping

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl SchemaMapper for SchemaMapping

fn map_batch(&self, batch: RecordBatch) -> Result<RecordBatch>

Adapts a RecordBatch to match the projected_table_schema using the stored mapping and conversions. The produced RecordBatch has a schema that contains only the projected columns.

fn map_column_statistics( &self, file_col_statistics: &[ColumnStatistics], ) -> Result<Vec<ColumnStatistics>>

Adapts file-level column Statistics to match the table_schema