Read Primitives — Exchange Merge

Read Primitives benchmark · all Read Primitives queries

Sorted data from multiple nodes is combined while keeping the sort order

Representative SQL

Rendered for DataFusion with default parameters.

SELECT
  o.o_orderkey,
  l.l_linenumber,
  l.l_quantity
FROM orders AS o
JOIN lineitem AS l
  ON o.o_orderkey = l.l_orderkey
ORDER BY
  o.o_orderkey,
  l.l_linenumber

Representative DataFrame

Expression family (Polars, PySpark, DataFusion), default parameters.

def exchange_merge_expression_impl(ctx: DataFrameContext) -> Any:
    return (
        ctx.get_table("orders")
        .join(ctx.get_table("lineitem"), left_on="o_orderkey", right_on="l_orderkey")
        .select("o_orderkey", "l_linenumber", "l_quantity")
        .sort(["o_orderkey", "l_linenumber"])
    )

Get the exact query for your run

The blocks above are representative. To get the precise statement a specific platform runs at a specific scale, substitute your own values:

from benchbox import ReadPrimitives

bench = ReadPrimitives(scale_factor=0.01)            # your scale factor
queries = bench.get_queries(dialect="datafusion")   # your platform's dialect; drop dialect= for native SQL
print(queries['exchange_merge'])

For the DataFrame form:

from benchbox.core.query_catalog import get_dataframe_render

print(get_dataframe_render('read_primitives', 'exchange_merge', family="expression").source)  # or family="pandas"

Source

Query text: benchbox/core/read_primitives/queries.py