Struct IPCStreamWriter 

struct IPCStreamWriter {
    pub writer: StreamWriter<File>,
    pub num_batches: usize,
    pub num_rows: usize,
    pub num_bytes: usize,
}

Write in Arrow IPC Stream format to a file.

Stream format is used for spill because it supports dictionary replacement, and the random access of IPC File format is not needed (IPC File format doesn’t support dictionary replacement).

Fields

writer: StreamWriter<File>

Inner writer

num_batches: usize

Batches written

num_rows: usize

Rows written

num_bytes: usize

Bytes written

Implementations

impl IPCStreamWriter

pub fn new( path: &Path, schema: &Schema, compression_type: SpillCompression, ) -> Result<Self>

Create new writer

pub fn write(&mut self, batch: &RecordBatch) -> Result<(usize, usize)>

Writes a single batch to the IPC stream and updates the internal counters.

Returns a tuple containing the change in the number of rows and bytes written.

pub fn finish(&mut self) -> Result<()>

Finish the writer