Struct ListingSchemaProvider 

pub struct ListingSchemaProvider {
    authority: String,
    path: Path,
    factory: Arc<dyn TableProviderFactory>,
    store: Arc<dyn ObjectStore>,
    tables: Arc<Mutex<HashMap<String, Arc<dyn TableProvider>>>>,
    format: String,
}

A SchemaProvider that scans an ObjectStore to automatically discover tables

A subfolder relationship is assumed, i.e. given:

• authority = s3://host.example.com:3000
• path = /data/tpch
• factory = DeltaTableFactory

A table called “customer” will be registered for the folder: s3://host.example.com:3000/data/tpch/customer

assuming it contains valid deltalake data, i.e:

• s3://host.example.com:3000/data/tpch/customer/part-00000-xxxx.snappy.parquet
• s3://host.example.com:3000/data/tpch/customer/_delta_log/

Fields

authority: String

path: Path

factory: Arc<dyn TableProviderFactory>

store: Arc<dyn ObjectStore>

tables: Arc<Mutex<HashMap<String, Arc<dyn TableProvider>>>>

format: String

Implementations

impl ListingSchemaProvider

pub fn new( authority: String, path: Path, factory: Arc<dyn TableProviderFactory>, store: Arc<dyn ObjectStore>, format: String, ) -> ListingSchemaProvider

Create a new ListingSchemaProvider

Arguments: authority: The scheme (i.e. s3://) + host (i.e. example.com:3000) path: The root path that contains subfolders which represent tables factory: The TableProviderFactory to use to instantiate tables for each subfolder store: The ObjectStore containing the table data format: The FileFormat of the tables has_header: Indicates whether the created external table has the has_header flag enabled

pub async fn refresh(&self, state: &dyn Session) -> Result<(), DataFusionError>

Reload table information from ObjectStore

Trait Implementations

impl Debug for ListingSchemaProvider

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl SchemaProvider for ListingSchemaProvider

fn as_any(&self) -> &(dyn Any + 'static)

Returns this SchemaProvider as Any so that it can be downcast to a specific implementation.

fn table_names(&self) -> Vec<String>

Retrieves the list of available table names in this schema.

fn table<'life0, 'life1, 'async_trait>( &'life0 self, name: &'life1 str, ) -> Pin<Box<dyn Future<Output = Result<Option<Arc<dyn TableProvider>>, DataFusionError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, ListingSchemaProvider: 'async_trait,

Retrieves a specific table from the schema by name, if it exists, otherwise returns None.

fn register_table( &self, name: String, table: Arc<dyn TableProvider>, ) -> Result<Option<Arc<dyn TableProvider>>, DataFusionError>

If supported by the implementation, adds a new table named name to this schema.

fn deregister_table( &self, name: &str, ) -> Result<Option<Arc<dyn TableProvider>>, DataFusionError>

If supported by the implementation, removes the name table from this schema and returns the previously registered TableProvider, if any.

fn table_exist(&self, name: &str) -> bool

Returns true if table exist in the schema provider, false otherwise.

fn owner_name(&self) -> Option<&str>

Returns the owner of the Schema, default is None. This value is reported as part of `information_tables.schemata

fn table_type<'life0, 'life1, 'async_trait>( &'life0 self, name: &'life1 str, ) -> Pin<Box<dyn Future<Output = Result<Option<TableType>, DataFusionError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, Self: 'async_trait,

Retrieves the type of a specific table from the schema by name, if it exists, otherwise returns None. Implementations for which this operation is cheap but Self::table is expensive can override this to improve operations that only need the type, e.g. SELECT * FROM information_schema.tables.