After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 3e6e8f3.

Benchmarks with errors

These are errors that were caught while running the benchmarks. You can click each link to go to the Conbench entry for that benchmark, which might have more information about what the error was.

• Commit Run on arm64-t4g-2xlarge-linux at 2025-01-30 05:46:35Z

  • tpch (R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-03, scale_factor=1
  • tpch (R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-03, scale_factor=1

• Commit Run on amd64-m5-4xlarge-linux at 2025-01-30 05:38:26Z

  • tpch (R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-03, scale_factor=1
  • tpch (R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-03, scale_factor=10
  • tpch (R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-03, scale_factor=10
  • tpch (R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-03, scale_factor=1

• Commit Run on test-mac-arm at 2025-01-30 05:49:22Z

  • tpch (R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-03, scale_factor=1
  • tpch (R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-03, scale_factor=1

Benchmarks with performance regressions

There were 0 possible performance regressions, according to the lookback z-score method.

All benchmark runs analyzed:

• Commit Run on arm64-t4g-2xlarge-linux at 2025-01-30 05:46:35Z
• Commit Run on amd64-c6a-4xlarge-linux at 2025-01-30 05:26:31Z
• Commit Run on amd64-m5-4xlarge-linux at 2025-01-30 05:38:26Z
• Commit Run on test-mac-arm at 2025-01-30 05:49:22Z

Unstable benchmarks with performance regressions

The following benchmark results indicate a possible performance regression, but are known to sometimes produce false positives when applying the lookback z-score analysis.

• Commit Run on amd64-c6a-4xlarge-linux at 2025-01-30 05:26:31Z

  • ShortVectorPushBack (C++) with params=<STATIC_VECTOR(std::string)>, source=cpp-micro, suite=arrow-small-vector-benchmark

• Commit Run on test-mac-arm at 2025-01-30 05:49:22Z

  • ProjectionOverhead (C++) with params=zero_copy_expression/batch_size:1000000/real_time, source=cpp-micro, suite=arrow-acero-project-benchmark
  • ProjectionOverheadIsolated (C++) with params=complex_expression/batch_size:100000/real_time, source=cpp-micro, suite=arrow-acero-project-benchmark
  • ProjectionOverheadIsolated (C++) with params=ref_only_expression/batch_size:1000000/real_time, source=cpp-micro, suite=arrow-acero-project-benchmark