Expand description
DFParquetMetadata for fetching Parquet file metadata, statistics
and schema information.
Structsยง
- Cached
Parquet Meta Data - Wrapper to implement [
FileMetadata] for [ParquetMetaData]. - DFParquet
Metadata - Handles fetching Parquet file schema, metadata and statistics from object store.
- Statistics
Accumulators ๐ - Holds the accumulator state for collecting statistics from row groups
Functionsยง
- create_
max_ ๐min_ accs - get_
col_ ๐stats - has_
any_ ๐exact_ match - Checks if any occurrence of
valueinarraycorresponds to atrueentry in theexactnessarray. - min_
max_ ๐aggregate_ data_ type - Min/max aggregation can take Dictionary encode input but always produces unpacked (aka non Dictionary) output. We need to adjust the output data type to reflect this. The reason min/max aggregate produces unpacked output because there is only one min/max value per group; there is no needs to keep them Dictionary encoded
- summarize_
min_ ๐max_ null_ counts