Struct FileStreamProvider 

pub struct FileStreamProvider {
    location: PathBuf,
    encoding: StreamEncoding,
    pub schema: Arc<Schema>,
    header: bool,
    batch_size: usize,
}

Stream data from the file at location

• Data will be read sequentially from the provided location
• New data will be appended to the end of the file

The encoding can be configured with Self::with_encoding and defaults to StreamEncoding::Csv

Fields

location: PathBuf

encoding: StreamEncoding

schema: Arc<Schema>

Get a reference to the schema for this file stream

header: bool

batch_size: usize

Implementations

impl FileStreamProvider

pub fn new_file(schema: Arc<Schema>, location: PathBuf) -> FileStreamProvider

Stream data from the file at location

• Data will be read sequentially from the provided location
• New data will be appended to the end of the file

The encoding can be configured with Self::with_encoding and defaults to StreamEncoding::Csv

pub fn with_batch_size(self, batch_size: usize) -> FileStreamProvider

Set the batch size (the number of rows to load at one time)

pub fn with_header(self, header: bool) -> FileStreamProvider

Specify whether the file has a header (only applicable for StreamEncoding::Csv)

pub fn with_encoding(self, encoding: StreamEncoding) -> FileStreamProvider

Specify an encoding for the stream

Trait Implementations

impl Debug for FileStreamProvider

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl StreamProvider for FileStreamProvider

fn schema(&self) -> &Arc<Schema>

Get a reference to the schema for this stream

fn reader( &self, ) -> Result<Box<dyn RecordBatchReader<Item = Result<RecordBatch, ArrowError>>>, DataFusionError>

Provide RecordBatchReader

fn writer(&self) -> Result<Box<dyn RecordBatchWriter>, DataFusionError>

Provide RecordBatchWriter

fn stream_write_display( &self, _t: DisplayFormatType, f: &mut Formatter<'_>, ) -> Result<(), Error>

Display implementation when using as a DataSink