Module unnest 

Define a plan for unnesting values in columns that contain a list type.

Structs

ListUnnest

UnnestExec

Unnest the given columns (either with type struct or list) For list unnesting, each row is vertically transformed into multiple rows For struct unnesting, each column is horizontally transformed into multiple columns, Thus the original RecordBatch with dimension (n x m) may have new dimension (n’ x m’)

UnnestMetrics 🔒

UnnestStream 🔒

A stream that issues [RecordBatch]es with unnested column data.

UnnestingResult 🔒

Traits

ListArrayType 🔒

Trait defining common methods used for unnesting, implemented by list array types.

Functions

build_batch 🔒

For each row in a RecordBatch, some list/struct columns need to be unnested.

create_take_indices 🔒

Creates take indices that will be used to expand all columns except for the list type columns that is being unnested. Every column value needs to be repeated multiple times according to the length array.

find_longest_length 🔒

Find the longest list length among the given list arrays for each row.

flatten_struct_cols 🔒

Given a set of struct column indices to flatten try converting the column in input into multiple subfield columns For example struct_col: [a: struct(item: int, name: string), b: int] with a batch {a: {item: 1, name: “a”}, b: 2}, {a: {item: 3, name: “b”}, b: 4] will be converted into {a.item: 1, a.name: “a”, b: 2}, {a.item: 3, a.name: “b”, b: 4}

list_unnest_at_level 🔒

This function is used to execute the unnesting on multiple columns all at once, but one level at a time, and is called n times, where n is the highest recursion level among the unnest exprs in the query.

repeat_arrs_from_indices 🔒

Create a batch of arrays based on an input batch and a indices array. The indices array is used by the take kernel to repeat values in the arrays that are marked with true in the repeat_mask. Arrays marked with false in the repeat_mask will be replaced with arrays filled with nulls of the appropriate length.

unnest_list_array 🔒

Unnest a list array according the target length array.

unnest_list_arrays 🔒

Unnest multiple list arrays according to the length array.