ParquetFileReaderFactory

datafusion_datasource_parquet

Trait ParquetFileReaderFactory 

Source 
pub trait ParquetFileReaderFactory:
    Debug
    + Send
    + Sync
    + 'static {
    // Required method
    fn create_reader(
        &self,
        partition_index: usize,
        partitioned_file: PartitionedFile,
        metadata_size_hint: Option<usize>,
        metrics: &ExecutionPlanMetricsSet,
    ) -> Result<Box<dyn AsyncFileReader + Send>>;
}
Expand description

Interface for reading parquet files.

The combined implementations of ParquetFileReaderFactory and [AsyncFileReader] can be used to provide custom data access operations such as pre-cached metadata, I/O coalescing, etc.

See DefaultParquetFileReaderFactory for a simple implementation.

Required Methods§

Source

fn create_reader( &self, partition_index: usize, partitioned_file: PartitionedFile, metadata_size_hint: Option<usize>, metrics: &ExecutionPlanMetricsSet, ) -> Result<Box<dyn AsyncFileReader + Send>>

Provides an AsyncFileReader for reading data from a parquet file specified

§Notes

If the resulting [AsyncFileReader] returns ParquetMetaData without page index information, the reader will load it on demand. Thus it is important to ensure that the returned ParquetMetaData has the necessary information if you wish to avoid a subsequent I/O

§Arguments
  • partition_index - Index of the partition (for reporting metrics)
  • file - The file to be read
  • metadata_size_hint - If specified, the first IO reads this many bytes from the footer
  • metrics - Execution metrics

Implementors§