AMPLab Q1a — Scan Aggregation

AMPLab benchmark · all AMPLab queries

COUNT, AVG, MAX of pageRank with threshold filter

Representative SQL

Rendered for DataFusion with default parameters.

SELECT
  COUNT(*) AS total_pages,
  AVG(pageRank) AS avg_pagerank,
  MAX(pageRank) AS max_pagerank
FROM rankings
WHERE
  pageRank > 1000

Representative DataFrame

Expression family (Polars, PySpark, DataFusion), default parameters.

def q1a_expression_impl(ctx: DataFrameContext) -> Any:
    params = get_parameters("Q1a")
    threshold = params.get("pagerank_threshold", 1000)

    rankings = ctx.get_table("rankings")
    col = ctx.col
    lit = ctx.lit

    return rankings.filter(col("pageRank") > lit(threshold)).select(
        col("pageRank").count().alias("total_pages"),
        col("pageRank").mean().alias("avg_pagerank"),
        col("pageRank").max().alias("max_pagerank"),
    )

Get the exact query for your run

The blocks above are representative. To get the precise statement a specific platform runs at a specific scale, substitute your own values:

from benchbox import AMPLab

bench = AMPLab(scale_factor=0.01)            # your scale factor
queries = bench.get_queries(dialect="datafusion")   # your platform's dialect; drop dialect= for native SQL
print(queries['1a'])

For the DataFrame form:

from benchbox.core.query_catalog import get_dataframe_render

print(get_dataframe_render('amplab', '1a', family="expression").source)  # or family="pandas"

Source

Query text: benchbox/core/amplab/queries.py