Struct CacheManagerConfig 

pub struct CacheManagerConfig {
    pub table_files_statistics_cache: Option<Arc<dyn CacheAccessor<Path, Arc<Statistics>, Extra = ObjectMeta>>>,
    pub list_files_cache: Option<Arc<dyn CacheAccessor<Path, Arc<Vec<ObjectMeta>>, Extra = ObjectMeta>>>,
    pub file_metadata_cache: Option<Arc<dyn FileMetadataCache<Extra = ObjectMeta>>>,
    pub metadata_cache_limit: usize,
}

Fields

table_files_statistics_cache: Option<Arc<dyn CacheAccessor<Path, Arc<Statistics>, Extra = ObjectMeta>>>

Enable cache of files statistics when listing files. Avoid get same file statistics repeatedly in same datafusion session. Default is disable. Fow now only supports Parquet files.

list_files_cache: Option<Arc<dyn CacheAccessor<Path, Arc<Vec<ObjectMeta>>, Extra = ObjectMeta>>>

Enable cache of file metadata when listing files. This setting avoids listing file meta of the same path repeatedly in same session, which may be expensive in certain situations (e.g. remote object storage). Note that if this option is enabled, DataFusion will not see any updates to the underlying location.
Default is disable.

file_metadata_cache: Option<Arc<dyn FileMetadataCache<Extra = ObjectMeta>>>

Cache of file-embedded metadata, used to avoid reading it multiple times when processing a data file (e.g., Parquet footer and page metadata). If not provided, the CacheManager will create a DefaultFilesMetadataCache.

metadata_cache_limit: usize

Limit of the file-embedded metadata cache, in bytes.

Implementations

impl CacheManagerConfig

pub fn with_files_statistics_cache( self, cache: Option<Arc<dyn CacheAccessor<Path, Arc<Statistics>, Extra = ObjectMeta>>>, ) -> CacheManagerConfig

Set the cache for files statistics.

Default is None (disabled).

pub fn with_list_files_cache( self, cache: Option<Arc<dyn CacheAccessor<Path, Arc<Vec<ObjectMeta>>, Extra = ObjectMeta>>>, ) -> CacheManagerConfig

Set the cache for listing files.

Default is None (disabled).

pub fn with_file_metadata_cache( self, cache: Option<Arc<dyn FileMetadataCache<Extra = ObjectMeta>>>, ) -> CacheManagerConfig

Sets the cache for file-embedded metadata.

Default is a DefaultFilesMetadataCache.

pub fn with_metadata_cache_limit(self, limit: usize) -> CacheManagerConfig

Sets the limit of the file-embedded metadata cache, in bytes.

Trait Implementations

impl Clone for CacheManagerConfig

fn clone(&self) -> CacheManagerConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Default for CacheManagerConfig

fn default() -> CacheManagerConfig

Returns the “default value” for a type.