fix: Instrument the KeepRunningBatch() loops - #55
Open
chfast wants to merge 1 commit into
Open
Conversation
The start_benchmark()/end_benchmark() hooks were attached to the StateIterator only, so the benchmarks looping with while (state.KeepRunningBatch(n)) were executed but never measured. Move them to StartKeepRunning()/FinishKeepRunning(), which both the range-based and the KeepRunning*() loops go through.
chfast
added a commit
to ipsilon/evmone
that referenced
this pull request
Aug 14, 2026
The KeepRunningBatch() loops are executed but never measured by the released compatibility layer, which silently drops 28 benchmarks, all the precompile ones among them. See CodSpeedHQ/codspeed-cpp#54 and the fix in CodSpeedHQ/codspeed-cpp#55.
Greptile SummaryThe PR relocates CodSpeed analysis hooks into Google Benchmark’s common State lifecycle so KeepRunning() and KeepRunningBatch() loops are measured without changing the range-based loop contract.
Confidence Score: 5/5The PR appears safe to merge, with supported benchmark loop styles now sharing a paired instrumentation lifecycle. The relocated hooks retain their previous ordering for range-based loops and add the same start/stop coverage to KeepRunning and KeepRunningBatch without changing public interfaces or overlapping the separate wall-time build path.
|
| Filename | Overview |
|---|---|
| google_benchmark/include/benchmark/benchmark.h | Removes range-iterator-specific instrumentation while retaining calls into the shared State lifecycle used by all supported loop styles. |
| google_benchmark/src/benchmark.cc | Adds paired CodSpeed analysis hooks to StartKeepRunning and FinishKeepRunning with the range-loop ordering preserved. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Benchmark loop begins] --> B{Loop style}
B -->|Range-based loop| C[State::end]
B -->|KeepRunning or KeepRunningBatch| D[KeepRunningInternal]
C --> E[StartKeepRunning]
D --> E
E --> F[Start CodSpeed benchmark and measurement]
F --> G[Execute benchmark iterations]
G --> H[FinishKeepRunning]
H --> I[Stop measurement and end CodSpeed benchmark]
Reviews (1): Last reviewed commit: "fix: Instrument the KeepRunningBatch() l..." | Re-trigger Greptile
chfast
added a commit
to ipsilon/evmone
that referenced
this pull request
Aug 14, 2026
The KeepRunningBatch() loops are executed but never measured by the released compatibility layer, which silently drops 28 benchmarks, all the precompile ones among them. The fork instruments them, see CodSpeedHQ/codspeed-cpp#54 and the fix in CodSpeedHQ/codspeed-cpp#55, so the benchmark sources no longer need the EVMONE_BENCH_LOOP_BATCH macro switching the loop in the CodSpeed builds.
chfast
added a commit
to ipsilon/evmone
that referenced
this pull request
Aug 14, 2026
The KeepRunningBatch() loops are executed but never measured by the released compatibility layer, which silently drops 28 benchmarks, all the precompile ones among them. The fork instruments them, see CodSpeedHQ/codspeed-cpp#54 and the fix in CodSpeedHQ/codspeed-cpp#55, so the benchmark sources no longer need the EVMONE_BENCH_LOOP_BATCH macro switching the loop in the CodSpeed builds.
codspeed-hq Bot
pushed a commit
to ipsilon/evmone
that referenced
this pull request
Aug 14, 2026
The KeepRunningBatch() loops are executed but never measured by the released compatibility layer, which silently drops 28 benchmarks, all the precompile ones among them. The fork instruments them, see CodSpeedHQ/codspeed-cpp#54 and the fix in CodSpeedHQ/codspeed-cpp#55, so the benchmark sources no longer need the EVMONE_BENCH_LOOP_BATCH macro switching the loop in the CodSpeed builds.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #54.
The
start_benchmark()/end_benchmark()hooks were attached to theStateIteratorpaths only, so the benchmarks looping withwhile (state.KeepRunningBatch(n))were executed but never measured. They are moved toState::StartKeepRunning()/State::FinishKeepRunning(), which both the range-based and theKeepRunning*()loops go through, so both loop styles are now instrumented. The call order is preserved:start_benchmark()runs before the rest ofStartKeepRunning()andmeasurement_start()after it,measurement_stop()runs before the rest ofFinishKeepRunning()andend_benchmark()after it — the range-based loop therefore behaves exactly as before.Verified with evmone, whose 8
KeepRunningBatch()benchmarks are reported for the first time with this change and whose range-based ones are unaffected: