Read Primitives — Min/Max Runtime Filter

Read Primitives benchmark · all Read Primitives queries

Bloom filter and runtime filter effectiveness for join optimization

Representative SQL

Rendered for DataFusion with default parameters.

SELECT
  l.*
FROM lineitem AS l
WHERE
  l.l_orderkey IN (
    SELECT
      o_orderkey
    FROM orders
    WHERE
      o_orderdate BETWEEN CAST('1995-01-01' AS DATE) AND CAST('1995-03-31' AS DATE)
  )

Representative DataFrame

Expression family (Polars, PySpark, DataFusion), default parameters.

def min_max_runtime_filter_expression_impl(ctx: DataFrameContext) -> Any:
    lineitem, orders = _tables(ctx, "lineitem", "orders")
    col = ctx.col
    lit = ctx.lit

    date_filtered_orders = orders.filter(
        (col("o_orderdate") >= lit(date(1995, 1, 1))) & (col("o_orderdate") <= lit(date(1995, 3, 31)))
    ).select("o_orderkey")

    return lineitem.join(date_filtered_orders, left_on="l_orderkey", right_on="o_orderkey", how="semi")

Get the exact query for your run

The blocks above are representative. To get the precise statement a specific platform runs at a specific scale, substitute your own values:

from benchbox import ReadPrimitives

bench = ReadPrimitives(scale_factor=0.01)            # your scale factor
queries = bench.get_queries(dialect="datafusion")   # your platform's dialect; drop dialect= for native SQL
print(queries['min_max_runtime_filter'])

For the DataFrame form:

from benchbox.core.query_catalog import get_dataframe_render

print(get_dataframe_render('read_primitives', 'min_max_runtime_filter', family="expression").source)  # or family="pandas"

Source

Query text: benchbox/core/read_primitives/queries.py