AMPLab Q5 — Cross-Table Analytics

AMPLab benchmark · all AMPLab queries

Join uservisits+rankings, GROUP BY country with HAVING and COUNT DISTINCT

Representative SQL

Rendered for DataFusion with default parameters.

SELECT
  uv.countryCode,
  COUNT(DISTINCT uv.destURL) AS unique_pages,
  COUNT(*) AS total_visits,
  SUM(uv.adRevenue) AS total_revenue,
  AVG(r.pageRank) AS avg_pagerank,
  AVG(uv.duration) AS avg_duration
FROM uservisits AS uv
JOIN rankings AS r
  ON uv.destURL = r.pageURL
WHERE
  uv.visitDate >= '2000-01-01' AND r.pageRank > 1000
GROUP BY
  uv.countryCode
HAVING
  COUNT(*) > 10
ORDER BY
  total_revenue DESC
LIMIT 100

Representative DataFrame

Expression family (Polars, PySpark, DataFusion), default parameters.

def q5_expression_impl(ctx: DataFrameContext) -> Any:
    params = get_parameters("Q5")
    start_date = params.get("start_date", date(2000, 1, 1))
    threshold = params.get("pagerank_threshold", 1000)
    min_visits = params.get("min_visits", 10)
    limit_rows = params.get("limit_rows", 100)

    uservisits = ctx.get_table("uservisits")
    rankings = ctx.get_table("rankings")
    col = ctx.col
    lit = ctx.lit

    return (
        uservisits.filter(col("visitDate").cast_date() >= lit(start_date))
        .join(rankings, left_on="destURL", right_on="pageURL")
        .filter(col("pageRank") > lit(threshold))
        .group_by("countryCode")
        .agg(
            col("destURL").n_unique().alias("unique_pages"),
            col("countryCode").count().alias("total_visits"),
            col("adRevenue").sum().alias("total_revenue"),
            col("pageRank").mean().alias("avg_pagerank"),
            col("duration").mean().alias("avg_duration"),
        )
        .filter(col("total_visits") > lit(min_visits))
        .sort("total_revenue", descending=True)
        .limit(limit_rows)
    )

Get the exact query for your run

The blocks above are representative. To get the precise statement a specific platform runs at a specific scale, substitute your own values:

from benchbox import AMPLab

bench = AMPLab(scale_factor=0.01)            # your scale factor
queries = bench.get_queries(dialect="datafusion")   # your platform's dialect; drop dialect= for native SQL
print(queries['5'])

For the DataFrame form:

from benchbox.core.query_catalog import get_dataframe_render

print(get_dataframe_render('amplab', '5', family="expression").source)  # or family="pandas"

Source

Query text: benchbox/core/amplab/queries.py