report - Historical Analysis

Analyze benchmark results over time, generate platform rankings, and detect performance regressions. Results are stored in a local SQLite database at ~/.benchbox/results.db.

Basic Syntax

benchbox report <subcommand> [OPTIONS]

Subcommands

rankings - Platform Rankings

Generate platform rankings for a benchmark based on historical results.

benchbox report rankings --benchmark <name> --scale-factor <sf> [OPTIONS]

Required:

• --benchmark, -b TEXT: Benchmark name (e.g., TPC-H, TPC-DS)
• --scale-factor, -s FLOAT: Scale factor to rank

Optional:

• --metric, -m [geometric_mean|power_at_size|cost_efficiency]: Metric to rank by (default: geometric_mean)
• --min-samples INT: Minimum samples required to include a platform (default: 1)
• --lookback-days INT: Consider results from the last N days (default: 90)
• --db-path PATH: Path to result database

benchbox report rankings --benchmark TPC-H --scale-factor 1
benchbox report rankings -b TPC-DS -s 10 --metric power_at_size

trends - Performance Trends

Show performance trends for a platform over time, highlighting regressions.

benchbox report trends --platform <name> --benchmark <name> --scale-factor <sf> [OPTIONS]

Required:

• --platform, -p TEXT: Platform name
• --benchmark, -b TEXT: Benchmark name
• --scale-factor, -s FLOAT: Scale factor

Optional:

• --periods INT: Number of periods to analyze (default: 6)
• --period-days INT: Days per period (default: 30)
• --db-path PATH: Path to result database

benchbox report trends --platform DuckDB --benchmark TPC-H --scale-factor 1

regressions - Detect Regressions

Scan all platforms and benchmarks for significant performance slowdowns.

benchbox report regressions [OPTIONS]

Optional:

• --threshold FLOAT: Regression threshold percentage (default: 10.0)
• --lookback-days INT: Comparison period in days (default: 30)
• --db-path PATH: Path to result database

benchbox report regressions
benchbox report regressions --threshold 15 --lookback-days 14

import - Import Results

Import benchmark result files into the historical database.

benchbox report import <directory> [OPTIONS]

Required:

• DIRECTORY: Path to directory containing result JSON files

Optional:

• --pattern TEXT: Glob pattern for result files (default: **/*.json)
• --db-path PATH: Path to result database

benchbox report import benchmark_runs/results/
benchbox report import ./results --pattern "*.json"

stats - Database Statistics

Show summary statistics for the result database.

benchbox report stats [OPTIONS]

Optional:

• --db-path PATH: Path to result database

list - List Results

List stored benchmark results with optional filters.

benchbox report list [OPTIONS]

Optional:

• --platform, -p TEXT: Filter by platform
• --benchmark, -b TEXT: Filter by benchmark
• --limit INT: Maximum results to show (default: 20)
• --db-path PATH: Path to result database

benchbox report list
benchbox report list --platform DuckDB --limit 10

Related

• results - View and export recent benchmark results
• visualize - Generate charts from results