Skip to content

fix: Instrument the KeepRunningBatch() loops - #55

Open
chfast wants to merge 1 commit into
CodSpeedHQ:mainfrom
chfast:fix/keep-running-batch
Open

fix: Instrument the KeepRunningBatch() loops#55
chfast wants to merge 1 commit into
CodSpeedHQ:mainfrom
chfast:fix/keep-running-batch

Conversation

@chfast

@chfast chfast commented Aug 14, 2026

Copy link
Copy Markdown

Fixes #54.

The start_benchmark()/end_benchmark() hooks were attached to the StateIterator paths only, so the benchmarks looping with while (state.KeepRunningBatch(n)) were executed but never measured. They are moved to State::StartKeepRunning()/State::FinishKeepRunning(), which both the range-based and the KeepRunning*() loops go through, so both loop styles are now instrumented. The call order is preserved: start_benchmark() runs before the rest of StartKeepRunning() and measurement_start() after it, measurement_stop() runs before the rest of FinishKeepRunning() and end_benchmark() after it — the range-based loop therefore behaves exactly as before.

Verified with evmone, whose 8 KeepRunningBatch() benchmarks are reported for the first time with this change and whose range-based ones are unaffected:

Measured: ::{anonymous}::bench_ecrecovery::precompile[PrecompileId::ecrecover, evmone]
Measured: ::{anonymous}::bench_ecpairing::precompile[PrecompileId::ecpairing, evmmax_cpp]
Measured: ::{anonymous}::bench_kzg::precompile[PrecompileId::point_evaluation, evmone_blst]
…

The start_benchmark()/end_benchmark() hooks were attached to the StateIterator
only, so the benchmarks looping with while (state.KeepRunningBatch(n)) were
executed but never measured. Move them to StartKeepRunning()/FinishKeepRunning(),
which both the range-based and the KeepRunning*() loops go through.
chfast added a commit to ipsilon/evmone that referenced this pull request Aug 14, 2026
The KeepRunningBatch() loops are executed but never measured by the released
compatibility layer, which silently drops 28 benchmarks, all the precompile ones
among them. See CodSpeedHQ/codspeed-cpp#54 and the fix in
CodSpeedHQ/codspeed-cpp#55.
@greptile-apps

greptile-apps Bot commented Aug 14, 2026

Copy link
Copy Markdown

Greptile Summary

The PR relocates CodSpeed analysis hooks into Google Benchmark’s common State lifecycle so KeepRunning() and KeepRunningBatch() loops are measured without changing the range-based loop contract.

  • Moves benchmark-start and measurement-start hooks into State::StartKeepRunning().
  • Moves measurement-stop and benchmark-end hooks into State::FinishKeepRunning().
  • Removes the equivalent range-iterator-specific hooks.

Confidence Score: 5/5

The PR appears safe to merge, with supported benchmark loop styles now sharing a paired instrumentation lifecycle.

The relocated hooks retain their previous ordering for range-based loops and add the same start/stop coverage to KeepRunning and KeepRunningBatch without changing public interfaces or overlapping the separate wall-time build path.

Important Files Changed

Filename Overview
google_benchmark/include/benchmark/benchmark.h Removes range-iterator-specific instrumentation while retaining calls into the shared State lifecycle used by all supported loop styles.
google_benchmark/src/benchmark.cc Adds paired CodSpeed analysis hooks to StartKeepRunning and FinishKeepRunning with the range-loop ordering preserved.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Benchmark loop begins] --> B{Loop style}
  B -->|Range-based loop| C[State::end]
  B -->|KeepRunning or KeepRunningBatch| D[KeepRunningInternal]
  C --> E[StartKeepRunning]
  D --> E
  E --> F[Start CodSpeed benchmark and measurement]
  F --> G[Execute benchmark iterations]
  G --> H[FinishKeepRunning]
  H --> I[Stop measurement and end CodSpeed benchmark]
Loading

Reviews (1): Last reviewed commit: "fix: Instrument the KeepRunningBatch() l..." | Re-trigger Greptile

chfast added a commit to ipsilon/evmone that referenced this pull request Aug 14, 2026
The KeepRunningBatch() loops are executed but never measured by the released
compatibility layer, which silently drops 28 benchmarks, all the precompile ones
among them. The fork instruments them, see CodSpeedHQ/codspeed-cpp#54 and the
fix in CodSpeedHQ/codspeed-cpp#55, so the benchmark sources no longer need the
EVMONE_BENCH_LOOP_BATCH macro switching the loop in the CodSpeed builds.
chfast added a commit to ipsilon/evmone that referenced this pull request Aug 14, 2026
The KeepRunningBatch() loops are executed but never measured by the released
compatibility layer, which silently drops 28 benchmarks, all the precompile ones
among them. The fork instruments them, see CodSpeedHQ/codspeed-cpp#54 and the
fix in CodSpeedHQ/codspeed-cpp#55, so the benchmark sources no longer need the
EVMONE_BENCH_LOOP_BATCH macro switching the loop in the CodSpeed builds.
codspeed-hq Bot pushed a commit to ipsilon/evmone that referenced this pull request Aug 14, 2026
The KeepRunningBatch() loops are executed but never measured by the released
compatibility layer, which silently drops 28 benchmarks, all the precompile ones
among them. The fork instruments them, see CodSpeedHQ/codspeed-cpp#54 and the
fix in CodSpeedHQ/codspeed-cpp#55, so the benchmark sources no longer need the
EVMONE_BENCH_LOOP_BATCH macro switching the loop in the CodSpeed builds.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

KeepRunningBatch() benchmarks are silently not measured

1 participant