stateless_serialize_and_write_files

datafusion_datasource::write::orchestration

Function stateless_serialize_and_write_files 

Source 
pub(crate) async fn stateless_serialize_and_write_files(
    rx: Receiver<(Receiver<RecordBatch>, Arc<dyn BatchSerializer>, Box<dyn AsyncWrite + Send + Unpin>)>,
    tx: Sender<u64>,
) -> Result<()>
Expand description

Contains the common logic for serializing RecordBatches and writing the resulting bytes to an ObjectStore. Serialization is assumed to be stateless, i.e. each RecordBatch can be serialized without any dependency on the RecordBatches before or after.