Read Primitives — Exchange Broadcast

Read Primitives benchmark · all Read Primitives queries

One small table is copied to all nodes that have the large table

Representative SQL

Rendered for DataFusion with default parameters.

/* + BROADCAST(small_parts) */
SELECT
  l.l_orderkey,
  SUM(l.l_quantity) AS total_qty
FROM lineitem AS l
JOIN (
  SELECT
    p_partkey
  FROM part
  WHERE
    p_size = 1
) AS small_parts
  ON l.l_partkey = small_parts.p_partkey
GROUP BY
  l.l_orderkey

Representative DataFrame

Expression family (Polars, PySpark, DataFusion), default parameters.

def exchange_broadcast_expression_impl(ctx: DataFrameContext) -> Any:
    small_parts = ctx.get_table("part").filter(ctx.col("p_size") == ctx.lit(1)).select("p_partkey")
    return (
        ctx.get_table("lineitem")
        .join(small_parts, left_on="l_partkey", right_on="p_partkey")
        .group_by("l_orderkey")
        .agg(ctx.col("l_quantity").sum().alias("total_qty"))
    )

Get the exact query for your run

The blocks above are representative. To get the precise statement a specific platform runs at a specific scale, substitute your own values:

from benchbox import ReadPrimitives

bench = ReadPrimitives(scale_factor=0.01)            # your scale factor
queries = bench.get_queries(dialect="datafusion")   # your platform's dialect; drop dialect= for native SQL
print(queries['exchange_broadcast'])

For the DataFrame form:

from benchbox.core.query_catalog import get_dataframe_render

print(get_dataframe_render('read_primitives', 'exchange_broadcast', family="expression").source)  # or family="pandas"

Source

Query text: benchbox/core/read_primitives/queries.py