Struct StreamTable 

pub struct StreamTable(Arc<StreamConfig>);

A TableProvider for an unbounded stream source

Currently only reading from / appending to a single file in-place is supported, but other stream sources and sinks may be added in future.

Applications looking to read/write datasets comprising multiple files, e.g. Hadoop-style data stored in object storage, should instead consider ListingTable.

Tuple Fields

0: Arc<StreamConfig>

Implementations

impl StreamTable

pub fn new(config: Arc<StreamConfig>) -> StreamTable

Create a new StreamTable for the given StreamConfig

Trait Implementations

impl Debug for StreamTable

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl TableProvider for StreamTable

fn as_any(&self) -> &(dyn Any + 'static)

Returns the table provider as Any so that it can be downcast to a specific implementation.

fn schema(&self) -> Arc<Schema>

Get a reference to the schema for this table

fn constraints(&self) -> Option<&Constraints>

Get a reference to the constraints of the table. Returns:

fn table_type(&self) -> TableType

Get the type of this table for metadata/catalog purposes.

fn scan<'life0, 'life1, 'life2, 'life3, 'async_trait>( &'life0 self, state: &'life1 dyn Session, projection: Option<&'life2 Vec<usize>>, _filters: &'life3 [Expr], limit: Option<usize>, ) -> Pin<Box<dyn Future<Output = Result<Arc<dyn ExecutionPlan>, DataFusionError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait, StreamTable: 'async_trait,

Create an ExecutionPlan for scanning the table with optionally specified projection, filter and limit, described below.

fn insert_into<'life0, 'life1, 'async_trait>( &'life0 self, _state: &'life1 dyn Session, input: Arc<dyn ExecutionPlan>, _insert_op: InsertOp, ) -> Pin<Box<dyn Future<Output = Result<Arc<dyn ExecutionPlan>, DataFusionError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, StreamTable: 'async_trait,

Return an ExecutionPlan to insert data into this table, if supported.

fn get_table_definition(&self) -> Option<&str>

Get the create statement used to create this table, if available.

fn get_logical_plan(&self) -> Option<Cow<'_, LogicalPlan>>

Get the LogicalPlan of this table, if available.

fn get_column_default(&self, _column: &str) -> Option<&Expr>

Get the default value for a column, if available.

fn scan_with_args<'a, 'life0, 'life1, 'async_trait>( &'life0 self, state: &'life1 dyn Session, args: ScanArgs<'a>, ) -> Pin<Box<dyn Future<Output = Result<ScanResult, DataFusionError>> + Send + 'async_trait>>

where 'a: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, Self: 'async_trait,

Create an ExecutionPlan for scanning the table using structured arguments.

fn supports_filters_pushdown( &self, filters: &[&Expr], ) -> Result<Vec<TableProviderFilterPushDown>, DataFusionError>

Specify if DataFusion should provide filter expressions to the TableProvider to apply during the scan.

fn statistics(&self) -> Option<Statistics>

Get statistics for this table, if available Although not presently used in mainline DataFusion, this allows implementation specific behavior for downstream repositories, in conjunction with specialized optimizer rules to perform operations such as re-ordering of joins.