BenchBox
Home Docs Blog GitHub
Contents Menu Expand Light mode Dark mode Auto light/dark, in light mode Auto light/dark, in dark mode Skip to content
BenchBox 0.2.0 documentation
BenchBox 0.2.0 documentation

Getting Started

  • BenchBox Documentation
  • Installation & Environment Setup
  • Getting Started in 5 Minutes
  • BenchBox CLI - Quick Reference
  • MCP Integration Guide
  • Tutorials
    • Your First Benchmark
    • Understanding Benchmark Results
    • Comparing Platforms
    • DataFrame Benchmarking Quickstart

User Guide

  • Configuration Handbook
  • Data Generation
  • Dry Run Mode
  • Examples Directory Guide
  • Understanding Benchmark Results
  • Troubleshooting Guide
  • Frequently Asked Questions (FAQ)
  • Concepts
    • BenchBox Architecture
    • Benchmarking Workflows
    • Data Model
    • Glossary
    • Database Benchmarking Tools Compared

Benchmarks

  • TPC Standards
    • TPC-H Benchmark
    • TPC-DS Benchmark
    • TPC-DI Benchmark
  • Academic Benchmarks
    • Star Schema Benchmark (SSB)
    • AMPLab Big Data Benchmark
    • Join Order Benchmark Implementation
  • Industry Benchmarks
    • ClickBench (ClickHouse Analytics Benchmark)
    • H2O DB Benchmark
    • CoffeeShop Benchmark
  • BenchBox Primitives
    • Read Primitives Benchmark
    • Write Primitives Benchmark
    • Transaction Primitives Benchmark
  • BenchBox Experimental
    • TPC-Havoc Benchmark
    • TPC-H Skew Benchmark
    • TPC-H Data Vault Benchmark
    • AI/ML Primitives Benchmark
    • Metadata Primitives Benchmark
    • NYC Taxi OLAP Benchmark
    • TSBS DevOps Benchmark

Platforms

  • Platform Selection Guide
  • Multi-Platform Database Support
  • Platform Comparison Matrix
  • SQL Platforms
    • Apache DataFusion Platform Guide
    • ClickHouse Local Mode
    • Polars Platform
    • PostgreSQL Platform
    • PrestoDB Platform Guide
    • Trino Platform
    • Apache Spark Platform
  • DataFrame Platforms
    • DataFrame Platforms
    • Polars Platform
    • Pandas DataFrame Platform
    • Modin DataFrame Platform
    • Dask DataFrame Platform
    • cuDF DataFrame Platform
    • PySpark Platform (SQL & DataFrame)
    • DataFusion DataFrame Platform
  • Cloud Platforms
    • Amazon Athena Platform
    • Firebolt Platform
    • Microsoft Fabric
    • Azure Analytics Platforms

Guides

  • TPC Benchmark Guides
    • TPC-H Official Benchmark Guide
    • TPC-DS Official Benchmark Guide
    • TPC-DI Deployment Guide
    • TPC-DI ETL Implementation Guide
    • TPC Maintenance Phase: Complete Guide
    • TPC Patterns Usage Guide
    • TPC Test Result Validation System
  • Table Format Guides
    • Parquet Deep Dive
    • Delta Lake Guide
    • Apache Iceberg Guide
    • Vortex Guide
  • Platform Comparison Guide
  • Cloud Storage Support
  • Data Compression in BenchBox
  • Advanced Topics
    • Power Run Iterations and Concurrent Query Execution
    • Optimizer Sniff Test Queries
    • Performance Monitoring
    • Advanced Performance Optimization Guide
    • Performance & Tuning
      • Performance Monitoring
      • Advanced Performance Optimization Guide
      • Power Run Iterations and Concurrent Query Execution
    • Open Table Formats Guide
    • Custom Benchmarks
    • CI/CD Integration Guide
    • Customization & Extensions
      • Custom Benchmarks
      • Optimizer Sniff Test Queries
  • BenchBox Visualization Architecture
    • Chart Generation Guide
    • Chart Types
    • Customization
    • Templates
    • CLI Reference: benchbox visualize

Reference

  • BenchBox CLI Reference
    • run - Run Benchmarks
    • convert - Convert Data Formats
    • shell - Interactive SQL Shell
    • platforms - Platform Management
    • Utility Commands
    • Tuning Commands
    • Results Commands
    • visualize - Generate Charts
    • report - Historical Analysis
    • metrics - Performance Metrics
    • aggregate - Aggregate Results
    • datagen - Data Generation
    • setup - Cloud Credentials
    • show-plan / plan-history - Query Plans
    • download-answers - Answer Files
    • Configuration
    • Common Workflows
    • Troubleshooting
  • Python API Reference
    • Base Benchmark API
    • Benchmark API Reference
      • Benchmark APIs
      • TPC-H Benchmark API
      • TPC-DS Benchmark API
      • TPC-DI Benchmark API
      • SSB (Star Schema Benchmark) API
      • ClickBench Benchmark API
      • Join Order Benchmark API
      • AMPLab Big Data Benchmark API
      • H2O.ai Database Benchmark API
      • Read Primitives Benchmark API
      • Write Primitives Benchmark API
    • Results API
    • Result Analysis API
    • DuckDB Platform Adapter
    • Apache DataFusion Platform Adapter
    • SQLite Platform Adapter
    • ClickHouse Platform Adapter
    • Databricks Platform Adapter
    • BigQuery Platform Adapter
    • Snowflake Platform Adapter
    • Amazon Redshift Platform Adapter
    • Cloud Storage Integration API
    • Data Validation Utilities API
    • Utilities & Helpers
      • Utility Functions API
      • Tuning Configuration API
      • Cloud Storage Integration API
      • Result Analysis API
      • Data Validation Utilities API
      • Performance Monitoring Utilities API
      • Additional Utilities API
    • Additional Utilities API
    • Performance Monitoring Utilities API
    • Tuning Configuration API
  • MCP Server Reference
  • Result Export Formats
  • Benchmark Result Schema (v1.1)
  • API Reference

Development

  • Getting Started with Development
    • Development Guide
    • BenchBox Testing Guide
    • Live Integration Tests
  • Architecture & Design
    • Design & Architecture
      • BenchBox Architecture
      • BenchBox Repository Structure
      • Future State Proposals
        • artifactlinks Future State
        • benchbox-maintainer Future State
        • benchbox-experimental Future State
        • Monitoring Optional Extra Future State
        • MCP API Formalization Future State
    • BenchBox Architecture
    • BenchBox Repository Structure
    • Import Patterns and Lazy Loading
  • Platform Development
    • Adding New Platform Adapters
    • Adding a New DataFrame Platform
    • Runtime Module Architecture Overview
    • DB API 2.0: Foundation of BenchBox Platform Support
  • BenchBox Architecture
  • Testing Documentation
    • End-to-End (E2E) Testing Guide
    • Docker Integration Tests
    • Live Integration Tests

Browse by Tag

  • By Audience
    • beginner (19)
    • intermediate (86)
    • advanced (28)
    • contributor (35)
  • By Benchmark
    • tpc-h (13)
    • tpc-ds (10)
    • tpc-di (5)
    • tpc-havoc (1)
    • tpch-skew (1)
    • ssb (2)
    • clickbench (2)
    • h2odb (2)
    • join-order (1)
    • amplab (1)
    • nyctaxi (1)
    • coffeeshop (1)
    • datavault (1)
    • tsbs-devops (1)
    • read-primitives (1)
    • write-primitives (1)
    • transaction-primitives (1)
    • metadata-primitives (1)
    • ai-primitives (1)
    • custom-benchmark (20)
  • By Platform
    • duckdb (4)
    • sqlite (2)
    • postgresql (2)
    • datafusion (1)
    • snowflake (3)
    • databricks (3)
    • bigquery (2)
    • redshift (1)
    • motherduck (1)
    • starburst (1)
    • clickhouse (3)
    • trino (1)
    • presto (1)
    • firebolt (1)
    • timescaledb (2)
    • influxdb (1)
    • athena (1)
    • aws-glue (1)
    • emr-serverless (1)
    • athena-spark (1)
    • dataproc (1)
    • dataproc-serverless (1)
    • azure (1)
    • fabric (1)
    • fabric-spark (1)
    • synapse-spark (1)
    • spark (2)
    • pyspark (1)
    • pandas (1)
    • polars (2)
    • dask (1)
    • modin (1)
    • cudf (1)
    • datafusion-df (1)
  • By Platform Type
    • sql-platform (26)
    • dataframe-platform (17)
    • cloud-platform (21)
    • embedded-platform (3)
    • cloud-storage (3)
  • By Content Type
    • guide (105)
    • tutorial (5)
    • reference (82)
    • concept (30)
    • quickstart (4)
  • By Feature
    • architecture (8)
    • cli (24)
    • cloud (5)
    • e2e (1)
    • python-api (37)
    • configuration (3)
    • data-generation (3)
    • performance (11)
    • tuning (3)
    • validation (13)
    • testing (7)
    • visualization (7)
Back to top
Edit this page
View this page

Tagged with: testingΒΆ

Pages with this tag

  • CI/CD Integration Guide

  • Pytest xdist Safety

  • BenchBox Testing Guide

  • Docker Integration Tests

  • End-to-End (E2E) Testing Guide

  • Testing Documentation

  • Live Integration Tests

Next
Tagged with: visualization
Previous
Tagged with: validation
Copyright © 2025, Joe Harris
Made with Sphinx and @pradyunsg's Furo