Skip to content

test: Use the range-based loop in all benchmarks - #1655

Closed
chfast wants to merge 1 commit into
masterfrom
test/bench-batch-loop
Closed

test: Use the range-based loop in all benchmarks#1655
chfast wants to merge 1 commit into
masterfrom
test/bench-batch-loop

Conversation

@chfast

@chfast chfast commented Aug 13, 2026

Copy link
Copy Markdown
Member

Replaces the 8 state.KeepRunningBatch(N) loops with the range-based for (auto _ : state) one.

KeepRunningBatch() never drove the loop body — the body already iterated over all the items itself. The batch size only told google/benchmark to divide the measured time by it, so this changes the reporting, not the executed work:

  • Time changes from per item to per loop iteration, i.e. it grows by the batch size.
  • avg_time_per_item (avg_time_per_input in precompiles_bench) is added and reports what the Time column reported before.
  • gas_used becomes avg_gas_used, the average gas of a single input instead of the whole batch, so that it matches the per-input time: avg_gas_used / avg_time_per_input is exactly gas_rate.
  • gas_rate is unchanged, it never depended on the batch size.

Before and after with the same compiler, precompile<PrecompileId::ecrecover, evmone>:

master   227733 ns    gas_used=30k     gas_rate=13.1736M/s
this PR 2304700 ns    avg_gas_used=3k  gas_rate=13.0173M/s  avg_time_per_input=230.462us

The per-item counter matters most for find_jumpdest and lru_cache, where the batch size is derived from the benchmark argument (indexes.size(), capacity): without it a larger argument would simply look slower per iteration.

Uniform loops also keep every benchmark measurable by tools instrumenting the range-based loop only, which is the case for the CodSpeed compatibility layer: #1653 currently needs a dedicated macro to swap the loop in its builds, and that macro can be dropped once this lands.

@codecov

codecov Bot commented Aug 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.72%. Comparing base (ebc25ef) to head (d8033c1).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1655   +/-   ##
=======================================
  Coverage   97.72%   97.72%           
=======================================
  Files         171      171           
  Lines       15631    15631           
  Branches     3617     3617           
=======================================
  Hits        15275    15275           
  Misses        269      269           
  Partials       87       87           
Flag Coverage Δ
eest-develop 88.59% <ø> (ø)
eest-develop-gmp 26.51% <ø> (ø)
eest-legacy 17.14% <ø> (ø)
eest-libsecp256k1 28.81% <ø> (ø)
eest-stable 88.59% <ø> (ø)
evmone-unittests 93.44% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
core 96.10% <ø> (ø)
tooling 91.92% <ø> (ø)
tests 99.80% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Standardizes batched benchmarks on range-based loops while preserving per-item throughput reporting.

Changes:

  • Replaces eight KeepRunningBatch() loops.
  • Adds processed-item counters.
  • Preserves gas metrics in precompile benchmarks.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
test/precompiles_bench/precompiles_bench.cpp Updates precompile loop and item reporting.
test/internal_benchmarks/lru_cache_bench.cpp Updates batched cache insertion benchmark.
test/internal_benchmarks/find_jumpdest_bench.cpp Updates random lookup benchmarks.
test/internal_benchmarks/evmmax_bench.cpp Updates modular arithmetic benchmarks.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Replace the 8 KeepRunningBatch() loops with the range-based one. The batch size
never drove the loop body, it only told google/benchmark to divide the reported
time by it, so the executed work is unchanged and only the Time column changes
from per item to per loop iteration. The per-item time is now reported explicitly
by the avg_time_per_item counter, and the precompiles report avg_gas_used per
input to match it: avg_gas_used / avg_time_per_input is exactly gas_rate.
Uniform loops also keep every benchmark measurable by the tools instrumenting
the range-based loop only.
@chfast
chfast force-pushed the test/bench-batch-loop branch from 8282d9e to d8033c1 Compare August 14, 2026 08:01
@chfast chfast closed this Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants