Module distributor_channels 

Special channel construction to distribute data from various inputs into N outputs minimizing buffering but preventing deadlocks when repartitioning

Design

+----+      +------+
| TX |==||  | Gate |
+----+  ||  |      |  +--------+  +----+
        ====|      |==| Buffer |==| RX |
+----+  ||  |      |  +--------+  +----+
| TX |==||  |      |
+----+      |      |
            |      |
+----+      |      |  +--------+  +----+
| TX |======|      |==| Buffer |==| RX |
+----+      +------+  +--------+  +----+

There are N virtual MPSC (multi-producer, single consumer) channels with unbounded capacity. However, if all buffers/channels are non-empty, than a global gate will be closed preventing new data from being written (the sender futures will be pending) until at least one channel is empty (and not closed).

Structs

Channel 🔒

Links senders and receivers.

ChannelState 🔒

DistributionReceiver

Receiver side of distribution channels.

DistributionSender

Sender side of distribution channels.

Gate 🔒

The “all channels have data” gate.

RecvFuture

Future backing recv.

SendError

Erroring during send.

SendFuture

Future backing send.

Functions

channels

Create n empty channels.

partition_aware_channels

Create n_out empty channels for each of the n_in inputs. This way, each distinct partition will communicate via a dedicated channel. This SPSC structure enables us to track which partition input data comes from.

Type Aliases

PartitionAwareReceivers 🔒

PartitionAwareSenders 🔒

SharedChannel 🔒

Shared channel.

SharedGate 🔒

Gate shared by all senders and receivers.