remove_dist_changing_operators

datafusion_physical_optimizer::enforce_distribution

Function remove_dist_changing_operators 

Source 
fn remove_dist_changing_operators(
    distribution_context: DistributionContext,
) -> Result<DistributionContext>
Expand description

Updates the physical plan inside DistributionContext so that distribution changing operators are removed from the top. If they are necessary, they will be added in subsequent stages.

Assume that following plan is given:

"RepartitionExec: partitioning=RoundRobinBatch(10), input_partitions=10",
"  RepartitionExec: partitioning=RoundRobinBatch(10), input_partitions=2",
"    DataSourceExec: file_groups={2 groups: \[\[x], \[y]]}, projection=\[a, b, c, d, e], output_ordering=\[a@0 ASC], file_type=parquet",

Since RepartitionExecs change the distribution, this function removes them and returns following plan:

"DataSourceExec: file_groups={2 groups: \[\[x], \[y]]}, projection=\[a, b, c, d, e], output_ordering=\[a@0 ASC], file_type=parquet",