Read Primitives — Variance and StdDev

Read Primitives benchmark · all Read Primitives queries

Variance and standard deviation calculations

Representative SQL

Rendered for DataFusion with default parameters.

SELECT
  o_orderpriority,
  COUNT(*) AS order_count,
  AVG(o_totalprice) AS avg_price,
  VAR_SAMP(o_totalprice) AS price_variance,
  STDDEV(o_totalprice) AS price_stddev,
  STDDEV_POP(o_totalprice) AS price_stddev_pop,
  STDDEV_SAMP(o_totalprice) AS price_stddev_samp
FROM orders
WHERE
  o_orderdate >= CAST('1995-01-01' AS DATE)
GROUP BY
  o_orderpriority

Representative DataFrame

Expression family (Polars, PySpark, DataFusion), default parameters.

def statistical_variance_expression_impl(ctx: DataFrameContext) -> Any:
    import polars as pl

    return (
        ctx.get_table("orders")
        .native.filter(pl.col("o_orderdate") >= date(1995, 1, 1))
        .group_by("o_orderpriority")
        .agg(
            pl.len().alias("order_count"),
            pl.col("o_totalprice").mean().alias("avg_price"),
            pl.col("o_totalprice").var(ddof=1).alias("price_variance"),
            pl.col("o_totalprice").std(ddof=1).alias("price_stddev"),
            pl.col("o_totalprice").std(ddof=0).alias("price_stddev_pop"),
            pl.col("o_totalprice").std(ddof=1).alias("price_stddev_samp"),
        )
    )

Get the exact query for your run

The blocks above are representative. To get the precise statement a specific platform runs at a specific scale, substitute your own values:

from benchbox import ReadPrimitives

bench = ReadPrimitives(scale_factor=0.01)            # your scale factor
queries = bench.get_queries(dialect="datafusion")   # your platform's dialect; drop dialect= for native SQL
print(queries['statistical_variance_stddev'])

For the DataFrame form:

from benchbox.core.query_catalog import get_dataframe_render

print(get_dataframe_render('read_primitives', 'statistical_variance_stddev', family="expression").source)  # or family="pandas"

Source

Query text: benchbox/core/read_primitives/queries.py