AMPLab Q3 — Text Search

AMPLab benchmark · all AMPLab queries

Text search on searchWord with HAVING on visit_count

Representative SQL

Rendered for DataFusion with default parameters.

SELECT
  sourceIP,
  COUNT(*) AS visit_count,
  SUM(adRevenue) AS total_revenue,
  AVG(duration) AS avg_duration
FROM uservisits
WHERE
  visitDate BETWEEN '2000-01-01' AND '2000-01-03' AND searchWord LIKE '%database%'
GROUP BY
  sourceIP
HAVING
  COUNT(*) > 10
ORDER BY
  total_revenue DESC
LIMIT 100

Representative DataFrame

Expression family (Polars, PySpark, DataFusion), default parameters.

def q3_expression_impl(ctx: DataFrameContext) -> Any:
    params = get_parameters("Q3")
    start_date = params.get("start_date", date(2000, 1, 1))
    end_date = params.get("end_date", date(2000, 1, 3))
    search_term = params.get("search_term", "database")
    min_visits = params.get("min_visits", 10)
    limit_rows = params.get("limit_rows", 100)

    uservisits = ctx.get_table("uservisits")
    col = ctx.col
    lit = ctx.lit

    return (
        uservisits.filter(
            (col("visitDate").cast_date() >= lit(start_date))
            & (col("visitDate").cast_date() <= lit(end_date))
            & (col("searchWord").str.contains(search_term))
        )
        .group_by("sourceIP")
        .agg(
            col("sourceIP").count().alias("visit_count"),
            col("adRevenue").sum().alias("total_revenue"),
            col("duration").mean().alias("avg_duration"),
        )
        .filter(col("visit_count") > lit(min_visits))
        .sort("total_revenue", descending=True)
        .limit(limit_rows)
    )

Get the exact query for your run

The blocks above are representative. To get the precise statement a specific platform runs at a specific scale, substitute your own values:

from benchbox import AMPLab

bench = AMPLab(scale_factor=0.01)            # your scale factor
queries = bench.get_queries(dialect="datafusion")   # your platform's dialect; drop dialect= for native SQL
print(queries['3'])

For the DataFrame form:

from benchbox.core.query_catalog import get_dataframe_render

print(get_dataframe_render('amplab', '3', family="expression").source)  # or family="pandas"

Source

Query text: benchbox/core/amplab/queries.py