TPC-DI EQ2

TPC-DI benchmark · all TPC-DI queries

Representative SQL

Rendered for DataFusion with default parameters.

SELECT
  'Incremental Load Validation' AS validation_name,
  table_name,
  batch_id,
  new_records,
  updated_records,
  unchanged_records,
  total_records,
  new_records / total_records * 100 AS new_record_pct,
  updated_records / total_records * 100 AS updated_record_pct,
  CASE
    WHEN (
      new_records + updated_records
    ) > 0 AND unchanged_records >= 0
    THEN 'PASS'
    WHEN (
      new_records + updated_records
    ) = 0
    THEN 'NO CHANGES'
    ELSE 'FAIL'
  END AS incremental_status
FROM (
  SELECT
    'DimCustomer' AS table_name,
    BatchID AS batch_id,
    COUNT(*) AS total_records,
    SUM(
      CASE WHEN EffectiveDate >= '2023-01-01' AND EndDate = '9999-12-31' THEN 1 ELSE 0 END
    ) AS new_records,
    SUM(
      CASE
        WHEN EffectiveDate >= '2023-01-01' AND EndDate <> '9999-12-31'
        THEN 1
        ELSE 0
      END
    ) AS updated_records,
    SUM(CASE WHEN EffectiveDate < '2023-01-01' THEN 1 ELSE 0 END) AS unchanged_records
  FROM DimCustomer
  WHERE
    BatchID >= 1
  GROUP BY
    BatchID
  UNION ALL
  SELECT
    'DimAccount' AS table_name,
    BatchID AS batch_id,
    COUNT(*) AS total_records,
    SUM(
      CASE WHEN EffectiveDate >= '2023-01-01' AND EndDate = '9999-12-31' THEN 1 ELSE 0 END
    ) AS new_records,
    SUM(
      CASE
        WHEN EffectiveDate >= '2023-01-01' AND EndDate <> '9999-12-31'
        THEN 1
        ELSE 0
      END
    ) AS updated_records,
    SUM(CASE WHEN EffectiveDate < '2023-01-01' THEN 1 ELSE 0 END) AS unchanged_records
  FROM DimAccount
  WHERE
    BatchID >= 1
  GROUP BY
    BatchID
  UNION ALL
  SELECT
    'DimSecurity' AS table_name,
    BatchID AS batch_id,
    COUNT(*) AS total_records,
    SUM(
      CASE WHEN EffectiveDate >= '2023-01-01' AND EndDate = '9999-12-31' THEN 1 ELSE 0 END
    ) AS new_records,
    SUM(
      CASE
        WHEN EffectiveDate >= '2023-01-01' AND EndDate <> '9999-12-31'
        THEN 1
        ELSE 0
      END
    ) AS updated_records,
    SUM(CASE WHEN EffectiveDate < '2023-01-01' THEN 1 ELSE 0 END) AS unchanged_records
  FROM DimSecurity
  WHERE
    BatchID >= 1
  GROUP BY
    BatchID
) AS incremental_metrics
ORDER BY
  batch_id,
  table_name

Representative DataFrame

No DataFrame implementation is registered for this query.

Get the exact query for your run

The blocks above are representative. To get the precise statement a specific platform runs at a specific scale, substitute your own values:

from benchbox import TPCDI

bench = TPCDI(scale_factor=0.01)            # your scale factor
queries = bench.get_queries(dialect="datafusion")   # your platform's dialect; drop dialect= for native SQL
print(queries['EQ2'])

Source

Query text: benchbox/core/tpcdi/query_etl.py