Read Primitives — Optimizer: Group-Join Fusion

Read Primitives benchmark · all Read Primitives queries

Test join+aggregate fusion into a single grouped-join pass

Representative SQL

Rendered for DataFusion with default parameters.

SELECT
  o.o_orderkey,
  o.o_custkey,
  o.o_orderdate,
  COUNT(l.l_linenumber) AS line_count,
  SUM(l.l_extendedprice * (
    1 - l.l_discount
  )) AS revenue
FROM orders AS o
JOIN lineitem AS l
  ON o.o_orderkey = l.l_orderkey
GROUP BY
  o.o_orderkey,
  o.o_custkey,
  o.o_orderdate
ORDER BY
  revenue DESC
LIMIT 100

Representative DataFrame

Expression family (Polars, PySpark, DataFusion), default parameters.

def optimizer_groupjoin_expression_impl(ctx: DataFrameContext) -> Any:
    orders, lineitem = _tables(ctx, "orders", "lineitem")
    col = ctx.col
    lit = ctx.lit

    return (
        orders.join(lineitem, col("o_orderkey") == col("l_orderkey"))
        .group_by("o_orderkey", "o_custkey", "o_orderdate")
        .agg(
            col("l_linenumber").count().alias("line_count"),
            (col("l_extendedprice") * (lit(1) - col("l_discount"))).sum().alias("revenue"),
        )
        .sort("revenue", descending=True)
        .limit(100)
    )

Get the exact query for your run

The blocks above are representative. To get the precise statement a specific platform runs at a specific scale, substitute your own values:

from benchbox import ReadPrimitives

bench = ReadPrimitives(scale_factor=0.01)            # your scale factor
queries = bench.get_queries(dialect="datafusion")   # your platform's dialect; drop dialect= for native SQL
print(queries['optimizer_groupjoin'])

For the DataFrame form:

from benchbox.core.query_catalog import get_dataframe_render

print(get_dataframe_render('read_primitives', 'optimizer_groupjoin', family="expression").source)  # or family="pandas"

Source

Query text: benchbox/core/read_primitives/queries.py