fn repeat_arrs_from_indices(
batch: &[ArrayRef],
indices: &PrimitiveArray<Int64Type>,
repeat_mask: &[bool],
) -> Result<Vec<Arc<dyn Array>>>Expand description
Create a batch of arrays based on an input batch and a indices array.
The indices array is used by the take kernel to repeat values in the arrays
that are marked with true in the repeat_mask. Arrays marked with false
in the repeat_mask will be replaced with arrays filled with nulls of the
appropriate length.
For example if we have the following batch:
c1: [1], null, [2, 3, 4], null, [5, 6]
c2: 'a', 'b', 'c', null, 'd'then the unnested_list_arrays contains the unnest column that will replace c1 in
the final batch if preserve_nulls is true:
c1: 1, null, 2, 3, 4, null, 5, 6And the indices array contains the indices that are used by take kernel to
repeat the values in c2:
0, 1, 2, 2, 2, 3, 4, 4so that the final batch will look like:
c1: 1, null, 2, 3, 4, null, 5, 6
c2: 'a', 'b', 'c', 'c', 'c', null, 'd', 'd'The repeat_mask determines whether an array’s values are repeated or replaced with nulls.
For example, if the repeat_mask is:
[true, false]The final batch will look like:
c1: 1, null, 2, 3, 4, null, 5, 6 // Repeated using `indices`
c2: null, null, null, null, null, null, null, null // Replaced with nulls