Driver Version Management¶

Tags intermediate guide driver versioning

When running benchmarks, BenchBox uses the Python database driver (e.g. duckdb, snowflake-connector-python) that is installed in the current environment. This guide explains how version selection works and how to reliably benchmark a specific driver version, including dev builds and pre-release releases.

How BenchBox Selects a Driver Version¶

BenchBox follows this order:

Explicit pin: If --platform-option driver_version=X.Y.Z is set, BenchBox validates that the requested version is installed. If driver_auto_install=true is also set, BenchBox installs it automatically via uv.
Current environment: If no version is requested, BenchBox uses whatever is installed in the current Python environment (detected via importlib.metadata).

The version that actually ran is always recorded in the result JSON under execution.driver_version_actual and displayed in the CLI announcement line:

Running tpch on duckdb at scale 1.0 [driver 1.4.3]

Why Your Manually-Installed Version May Not Be Respected¶

If you run BenchBox via uv run -- benchbox run ..., uv syncs your environment against uv.lock before Python starts. Any version you installed manually with uv pip install "duckdb==1.5.0" may be silently reverted to the lock-file-pinned version (e.g. 1.4.3).

The result JSON will still report the version that actually ran, but you may not notice the revert until you inspect it.

Reliable Patterns for Testing a Specific Version¶

Pattern 1: driver_auto_install (recommended)¶

BenchBox manages the install for the duration of the run. The lock file is not modified.

benchbox run --platform duckdb --benchmark tpch \
  --platform-option driver_version=1.5.0 \
  --platform-option driver_auto_install=true

This works for any platform, not just DuckDB:

# Snowflake connector
benchbox run --platform snowflake --benchmark tpch \
  --platform-option driver_version=3.12.0 \
  --platform-option driver_auto_install=true \
  --output s3://my-bucket/results/

# Polars DataFrame engine
benchbox run --platform polars-df --benchmark tpch \
  --platform-option driver_version=1.36.1 \
  --platform-option driver_auto_install=true

# ClickHouse Cloud (clickhouse-connect HTTP driver)
benchbox run --platform clickhouse-cloud --benchmark tpch \
  --platform-option driver_version=0.10.0 \
  --platform-option driver_auto_install=true \
  --platform-option host=your-instance.clickhouse.cloud \
  --platform-option password=your-password

Pattern 2: uv run –with (inline override)¶

Override the installed package for a single invocation without touching uv.lock:

uv run --with "duckdb==1.5.0" benchbox run --platform duckdb --benchmark tpch

Pattern 3: Update the lock file¶

If you want every uv run to use a specific version permanently:

uv add "duckdb==1.5.0" --allow-prereleases
# Re-lock and run normally
benchbox run --platform duckdb --benchmark tpch

This mutates uv.lock and is appropriate when you want a stable, reproducible environment rather than a one-off test.

Verifying Which Version Actually Ran¶

The CLI output shows the driver version immediately:

Running tpch on duckdb at scale 1.0 [driver 1.5.0]

For scripts and CI pipelines, check the result JSON:

jq '.execution.driver_version_actual' result.json
# or
jq '.platform.client_version' result.json

Both fields record the version that ran. driver_version_actual reflects what importlib.metadata reported from the live process; client_version is populated by the platform adapter from the driver itself.