Struct SchemaMapping 

pub struct SchemaMapping {
    projected_table_schema: Arc<Schema>,
    field_mappings: Vec<Option<usize>>,
    cast_column: Arc<dyn Fn(&Arc<dyn Array>, &Field, &CastOptions<'_>) -> Result<Arc<dyn Array>, DataFusionError> + Sync + Send>,
}

The SchemaMapping struct holds a mapping from the file schema to the table schema and any necessary type conversions.

map_batch is used by the ParquetOpener to produce a RecordBatch which has the projected schema, since that’s the schema which is supposed to come out of the execution of this query. Thus map_batch uses projected_table_schema as it can only operate on the projected fields.

Fields

projected_table_schema: Arc<Schema>

field_mappings: Vec<Option<usize>>

cast_column: Arc<dyn Fn(&Arc<dyn Array>, &Field, &CastOptions<'_>) -> Result<Arc<dyn Array>, DataFusionError> + Sync + Send>

Implementations

impl SchemaMapping

pub fn new( projected_table_schema: Arc<Schema>, field_mappings: Vec<Option<usize>>, cast_column: Arc<dyn Fn(&Arc<dyn Array>, &Field, &CastOptions<'_>) -> Result<Arc<dyn Array>, DataFusionError> + Sync + Send>, ) -> SchemaMapping

Creates a new SchemaMapping instance

Initializes the field mappings needed to transform file data to the projected table schema

Trait Implementations

impl Debug for SchemaMapping

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl SchemaMapper for SchemaMapping

fn map_batch(&self, batch: RecordBatch) -> Result<RecordBatch, DataFusionError>

Adapts a RecordBatch to match the projected_table_schema using the stored mapping and conversions. The produced RecordBatch has a schema that contains only the projected columns.

fn map_column_statistics( &self, file_col_statistics: &[ColumnStatistics], ) -> Result<Vec<ColumnStatistics>, DataFusionError>

Adapts file-level column Statistics to match the table_schema