Conversation
The dfsrw workload paired one read with every write, pinning the mix at 1:1. Real workloads are rarely balanced, so --read-write-ratio now sets how many reads a write is followed by: 4 reads back each write four times, 0.25 reads back every fourth write, and the default of 1 leaves the workload as it was. Reads stay inside the thread that wrote the file, so a read still validates the CRC32 of the latest write of the path it reads, and an overwritten path returning older bytes is still detected. A ratio that is not a whole number is accumulated per thread rather than rounded on every write, otherwise every ratio below 0.5 would read back nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
chihsuan
reviewed
Sep 10, 2026
Contributor
|
cc @yandrey321 if you’d like to take a look. This follow-up implements the configurable write/read ratio you suggested here #10651 (comment). Thanks! |
…ctly - Rename --read-write-ratio to --reads-per-write, and describe the reads as validation reads that each pick a file the thread wrote at random, rather than as read-backs of the write just made. - Carry the ratio in whole millionths of a read instead of accumulating it as a double, which drifted: adding 0.1 ten times gives 0.9999999999999999, so the read the tenth write was due slipped to an eleventh. The option is now a BigDecimal, which also removes the NaN and infinity checks. - Parameterize the corruption point of everyReadOfATaskValidatesContent over the first, second and third read, so it no longer covers only the last. - Extend FlippingInputStream from FSInputStream, which supplies the positioned reads and drops three forwarding methods. Adds carriesFractionalRatioWithoutDrift, which fails on the old accumulator.
yandrey321
reviewed
Sep 11, 2026
Review feedback: express the read/write mix as a percentage and draw per operation instead of issuing a fixed number of reads after every write. Every operation of the run now independently draws whether it writes a file or reads one back, so -n counts operations of both kinds and the split holds on average rather than exactly. This removes the fixed-point carry machinery that was needed to spread a fractional reads-per-write ratio across writes. Two invariants the ratio model got for free need holding explicitly: a thread whose history is still empty has nothing to read and writes instead, and the file a write goes to is now named after the write sequence in the thread's own marker rather than the task counter, which no longer counts that thread's writes. That keeps a thread within --max-files-per-thread however its reads and writes happen to fall.
… change The file a write goes to is named after the task counter again, as it was before --read-percent. Deriving it from the thread's own write sequence broke the path recycling the framework provides: taskLoop passes counter % testNo, so a time-based run cycles through -n ids and writes over its paths, while the write sequence only wraps after --max-files-per-thread. testValidateOverwrittenPaths saw 317 paths where it expects at most 8, and the tool stopped exercising overwritten reads in --duration runs, which is the point of that mode. The memory bound the write sequence was meant to protect holds either way, since the id is taken modulo --max-files-per-thread. testWriteReadValidate expected -n files. -n now counts operations of both kinds, so it takes the number of writes from the metrics and checks that reads and writes together account for every operation. testPathsWrapAtMaxFilesPerThread now runs at --read-percent 0. It asserts an exact file count, and with reads drawn in, whether every path of the cycle got written at all would have been left to chance.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Follow-up from HDDS-14524.
The
dfsrw(dfs-read-write-validator) freon workload issues exactly one readafter every write, so its read/write mix is pinned at 1:1. Real workloads are
rarely balanced, and the benchmark is more useful when the mix can be tuned
toward read-heavy or write-heavy.
This PR adds a
--read-write-ratiooption that sets how many reads each write isfollowed by:
--read-write-ratio1.0(default)40.25The existing per-path CRC32 validation and stale-read detection are unaffected. A
read still picks a random file from the history of the thread that wrote it and
compares against the checksum of the most recent write of that path, so both
corruption and an overwritten path returning older bytes are still detected.
Why a ratio rather than a
--read-threads/--write-threadssplitA read-only thread would have no write history of its own, so it would have to
read paths owned by other threads. Those paths are overwritten concurrently,
which is precisely the situation the stale-read check treats as a failure.
Splitting threads would therefore have meant weakening the validation, so every
read is kept inside the thread that wrote the file.
Fractional ratios
A ratio that is not a whole number is accumulated per thread rather than rounded
on each write:
Rounding per write would collapse every ratio below 0.5 to zero reads, leaving a
"validator" that never validates. Carrying the remainder makes
0.25read backevery fourth write and
1.5alternate between one and two reads. At the defaultof
1.0the credit lands on exactly 1.0 every time, so the default path isunchanged.
Notes
-nkeeps its meaning: one task is one write, and the number of files writtenfor a given
-ndoes not change with the ratio.file-writeandfile-read-validatetimers are unchanged, so the twosides of the mix stay separately measurable.
the existing
--size,--bufferand--max-files-per-threadchecks.What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-16354
How was this patch tested?
New unit test
TestHadoopFsReadWriteRatioinhadoop-ozone/freon.The existing integration test
TestHadoopFsReadWriteValidatoris unchanged andcontinues to cover the default behaviour against a real cluster.