Developer Documentation

Documentation for contributors and developers working on BenchBox.

Roadmap

• Development Roadmap - Planned platform and benchmark additions

Getting Started with Development

• Development Guide - Setting up development environment and workflow
• Testing - Testing strategies, running tests, and test organization
• Test tiers and duration budgets - Measured T1/T2/T3 policy and quarantine markers
• Pytest xdist Safety - Why macOS worker counts are capped and how to validate changes

Contributing

• Adding New Platforms - How to add support for new database platforms
• New Platform Acceptance Checklist - Required registry, docs, tests, compatibility, and UAT gates
• Import Patterns - Lazy loading and dependency management patterns
• TPC Compilation Guide - Compiling TPC benchmark tools

Architecture & Design

• Architecture Overview - High-level system architecture
• Code Structure - Codebase organization
• DB API 2.0 - Python DB API 2.0 specification and platform integration
• Runtime Modules - Runtime module organization and responsibilities
• Run Lifecycle Map - Current benchbox run execution/export branches
• Data Sharing - Data sharing between benchmark phases
• Dependency Compatibility - Managing optional dependencies

Testing

• Testing Guide - Testing strategies and test organization
• Test tiers and duration budgets - Measured T1/T2/T3 policy and quarantine markers
• Pytest xdist Safety - Root cause, reproducer, and validation checklist for xdist lock-ups
• Testing Index - Test documentation overview
• Live Integration Tests - Running tests against live databases

Results & Validation

• Result Execution Environment Contract - Normalized client host, platform runtime, cloud/compute/storage facets, and anonymization policy
• Result Integrity Validation - Three-tier validator: structural, completeness, and believability checks for result JSON files

Reference

• Read Primitives Catalog - Catalog of primitive read operations
• Read Primitives: Platform Skip Reference - Why each query is skipped per platform and how to re-enable

Related Documentation

• Concepts - Core concepts and glossary

• Development Roadmap
• Getting Started with Development
• Development Guide
• Platform Development
• Adding New Platform Adapters
• Adding a New DataFrame Platform
• New Platform Acceptance Checklist
• Architecture & Design
• Data & Dependencies
• Import Patterns and Lazy Loading
• TPC Binary Auto-Compilation Guide
• BenchBox Testing Guide
• Test tiers, duration budgets, and quarantine
• Pytest xdist Safety
• Primitives Query Catalog
• Read Primitives: Platform Skip Reference
• Runtime Module Architecture Overview
• Run Lifecycle Branch Map
• Data Sharing Between Benchmarks
• Dependency Compatibility
• DB API 2.0: Foundation of BenchBox Platform Support
• BenchBox Test Quality Guidelines
• Result Execution Environment Contract
• Result Integrity Validation
• BenchBox Platform Configuration Audit