Module schema_adapter 

SchemaAdapter and SchemaAdapterFactory to adapt file-level record batches to a table schema.

Adapter provides a method of translating the RecordBatches that come out of the physical format into how they should be used by DataFusion. For instance, a schema can be stored external to a parquet file that maps parquet logical types to arrow types.

Structs

DefaultSchemaAdapter 🔒

This SchemaAdapter requires both the table schema and the projected table schema. See SchemaMapping for more details

DefaultSchemaAdapterFactory

Default SchemaAdapterFactory for mapping schemas.

SchemaMapping

The SchemaMapping struct holds a mapping from the file schema to the table schema and any necessary type conversions.

Traits

SchemaAdapter

Creates SchemaMappers to map file-level [RecordBatch]es to a table schema, which may have a schema obtained from merging multiple file-level schemas.

SchemaAdapterFactory

Factory for creating SchemaAdapter

SchemaMapper

Maps, columns from a specific file schema to the table schema.

Functions

can_cast_field 🔒

Checks if a file field can be cast to a table field

create_field_mapping 🔒

Helper function that creates field mappings between file schema and table schema

Type Aliases

CastColumnFn

Function used by SchemaMapping to adapt a column from the file schema to the table schema.