Module enforce_sorting 

EnforceSorting optimizer rule inspects the physical plan with respect to local sorting requirements and does the following:

• Adds a SortExec when a requirement is not met,
• Removes an already-existing SortExec if it is possible to prove that this sort is unnecessary

The rule can work on valid and invalid physical plans with respect to sorting requirements, but always produces a valid physical plan in this sense.

A non-realistic but easy to follow example for sort removals: Assume that we somehow get the fragment

SortExec: expr=[nullable_col@0 ASC]
  SortExec: expr=[non_nullable_col@1 ASC]

in the physical plan. The first sort is unnecessary since its result is overwritten by another SortExec. Therefore, this rule removes it from the physical plan.

Modules

replace_with_order_preserving_variants

Optimizer rule that replaces executors that lose ordering with their order-preserving variants when it is helpful; either in terms of performance or to accommodate unbounded streams by fixing the pipeline.

sort_pushdown

Structs

EnforceSorting

This rule inspects SortExec’s in the given physical plan in order to remove unnecessary sorts, and optimize sort performance across the plan.

Functions

adjust_window_sort_removal 🔒

Adjusts a WindowAggExec or a BoundedWindowAggExec to determine whether it may allow removing a sort.

analyze_immediate_sort_removal 🔒

Analyzes if there are any immediate sort removals by checking the SortExecs and their ordering requirement satisfactions with children If the sort is unnecessary, either replaces it with SortPreservingMergeExec/LimitExec or removes the SortExec. Otherwise, returns the original plan

ensure_sorting

This function enforces sorting requirements and makes optimizations without violating these requirements whenever possible. Requires a bottom-up traversal.

get_sort_exprs 🔒

Converts an ExecutionPlan trait object to a LexOrdering reference when possible.

parallelize_sorts

Transform CoalescePartitionsExec + SortExec cascades into SortExec

remove_bottleneck_in_subplan 🔒

Removes parallelization-reducing, avoidable CoalescePartitionsExecs from the plan in node. After the removal of such CoalescePartitionsExecs from the plan, some of the remaining RepartitionExecs might become unnecessary. Removes such RepartitionExecs from the plan as well.

remove_corresponding_sort_from_sub_plan 🔒

Removes the sort from the plan in node.

replace_with_partial_sort 🔒

Only interested with SortExecs and their unbounded children. If the plan is not a SortExec or its child is not unbounded, returns the original plan. Otherwise, by checking the requirement satisfaction searches for a replacement chance. If there’s one replaces the SortExec plan with a PartialSortExec

update_child_to_remove_unnecessary_sort 🔒

Updates child to remove the unnecessary sort below it.

update_coalesce_ctx_children 🔒

Discovers the linked Coalesce->Sort cascades.

update_sort_ctx_children_data 🔒

For a given node, update the PlanContext.data attribute.

Type Aliases

PlanWithCorrespondingCoalescePartitions

This object is used within the EnforceSorting rule to track the closest CoalescePartitionsExec descendant(s) for every child of a plan. The data attribute stores whether the plan is a CoalescePartitionsExec or is connected to a CoalescePartitionsExec via its children.

PlanWithCorrespondingSort

This context object is used within the EnforceSorting rule to track the closest SortExec descendant(s) for every child of a plan. The data attribute stores whether the plan is a SortExec or is connected to a SortExec via its children.