Your First Benchmark

Run a complete TPC-H benchmark in under 5 minutes with zero configuration.

What You’ll Do

1. Install BenchBox
2. Run a TPC-H power test on DuckDB
3. View your results

Step 1: Install BenchBox

Use uv (recommended) or pip:

uv add benchbox

pip install benchbox

Step 2: Run the Benchmark

benchbox run --platform duckdb --benchmark tpch --scale 0.01

This command:

• Generates TPC-H data at scale factor 0.01 (~10MB)
• Loads data into DuckDB (stored in a database file inside the run output directory)
• Executes all 22 TPC-H queries
• Reports timing and validation results

Example output (your numbers will differ):

Running TPC-H Power Test (Scale Factor: 0.01, Stream ID: 0)
Warm-up runs: 1, Measurement runs: 3
...
✅ TPC-H Power Test completed
  Queries executed: 22, Successful: 22
...
✅ Benchmark completed: PASSED
...
┌ TPC-H on DuckDB (SF 0.01) ────────┐
│ Queries:  22                       │
│ Best: Q6, Q11, Q17                 │
│ Worst: Q10, Q8, Q21                │
└────────────────────────────────────┘

Step 3: View Results

The first command shows recent results. The second exports the last result to JSON for analysis.

benchbox results --limit 1

benchbox export --last --format json

What Just Happened?

1. Data Generation: BenchBox used TPC-H’s data generator to create realistic business data (customers, orders, line items)

2. Schema Loading: Tables were created in DuckDB and data was loaded

3. Query Execution: All 22 TPC-H queries ran sequentially (power test)

4. Validation: Results were compared against expected row counts

Try Different Options

The first command uses a larger dataset, which takes longer and is more realistic. The second runs specific queries only. The third previews without running (dry run).

benchbox run --platform duckdb --benchmark tpch --scale 0.1

benchbox run --platform duckdb --benchmark tpch --queries Q1,Q6,Q17

benchbox run --dry-run ./preview --platform duckdb --benchmark tpch

Next Steps

• Understanding Results - Learn what the metrics mean
• Comparing Platforms - Run on multiple databases
• DataFrame Benchmarking - Use Polars/Pandas APIs