Skip to content

uucore: Remove ARGV cache#13276

Open
AnuthaDev wants to merge 1 commit into
uutils:mainfrom
AnuthaDev:arghh-alloc
Open

uucore: Remove ARGV cache#13276
AnuthaDev wants to merge 1 commit into
uutils:mainfrom
AnuthaDev:arghh-alloc

Conversation

@AnuthaDev

@AnuthaDev AnuthaDev commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Problem

uucore keeps the CLI arguments cached in a static variable (ARGV), stating that repeatedly calling env::args_os is expensive. However env::args_os is only called once during the binary lifecycle. Storing the ARGV is useless overhead.

Fix

  • Removed the ARGV variable and added a different mechanism to initialize UTIL_NAME and EXECUTION_PHRASE.
  • Added init_util_name() , init_execution_phrase() that use the result of the first args_os call to set these variables. This is to prevent an additional call to args_os for determining these values.
  • Retained the get_or_init for these values instead of panicking on uninitialized values so any downstream users of uucore keep working, albeit with a performance hit.

Breaking changes

Two semantic changes for downstream consumers of the uucore crate:

  1. uucore::args_os() is no longer cheap to call repeatedly. It previously returned clones from a process-lifetime cache; it now copies all of argv (and re-expands globs via wild on Windows) on every call. Call it once and reuse the result.
  2. If a downstream binary calls the new init_* functions, those values win over derivation from argv. Callers that never init see identical behavior to before.

Measurements

  1. Create a test input file: printf "00 %.0s" {1..100000} > args.txt
  2. Run the binary:
valgrind --tool=dhat ./target/release/coreutils echo $(cat args.txt) > /dev/null

main:

Total:     5,286,993 bytes in 200,206 blocks
At t-gmax: 5,000,674 bytes in 100,006 blocks
At t-end:  2,652,146 bytes in 100,083 blocks
Reads:     10,776,549 bytes
Writes:    7,975,882 bytes

This PR:

Total:     2,687,044 bytes in 100,207 blocks
At t-gmax: 2,663,057 bytes in 100,147 blocks
At t-end:  52,103 bytes in 82 blocks
Reads:     10,576,682 bytes
Writes:    5,375,912 bytes

GNU echo:

Total:     8,089 bytes in 31 blocks
At t-gmax: 8,084 bytes in 30 blocks
At t-end:  0 bytes in 0 blocks
Reads:     304,472 bytes
Writes:    303,826 bytes

The uutils version still allocates 332 times more (down from 653 times), still lots of scope for reducing allocations.

Impact

This change reduces memory allocation for all the coreutils binaries. The savings scale with argv: total allocations drop by one full argv copy, and the duplicate no longer persists for the process lifetime.

Practical benefit: Calling ls pattern* in a directory with a large number of items matching the pattern allocates much less.

Testing

  • All existing tests pass

@sylvestre sylvestre left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the AI comment #0 isn't useful.
Please write it for a human as I am not an AI

Comment thread src/bin/coreutils.rs Outdated
@sylvestre

Copy link
Copy Markdown
Contributor

Please update comment 0

@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown

GNU testsuite comparison:

Skipping an intermittent issue tests/date/resolution (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/tail/tail-n0f (passes in this run but fails in the 'main' branch)

@codspeed-hq

codspeed-hq Bot commented Jul 4, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 5.72%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 4 improved benchmarks
✅ 327 untouched benchmarks
⏩ 46 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation complex_relative_date 147.1 µs 135.7 µs +8.43%
Simulation hostname_ip_lookup[100000] 108.8 µs 100.9 µs +7.74%
Simulation unexpand_large_file[10] 291.9 ms 282.3 ms +3.41%
Simulation unexpand_many_lines[100000] 139.3 ms 134.7 ms +3.41%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing AnuthaDev:arghh-alloc (1155881) with main (871b80c)

Open in CodSpeed

Footnotes

  1. 46 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@AnuthaDev

Copy link
Copy Markdown
Contributor Author

Updated the Summary section in description (if that's what you meant by comment 0)

@sylvestre

Copy link
Copy Markdown
Contributor

this one:
#13276 (comment)

@sylvestre

Copy link
Copy Markdown
Contributor

i still would like comment #0 to be shorter
also, would it be possible to add a memory benchmark confirming this improvement? (note: should be added in a different pr)
thanks

@AnuthaDev

AnuthaDev commented Jul 5, 2026

Copy link
Copy Markdown
Contributor Author

Improved the comment. Will create a separate PR to add a memory benchmark. Please review.

Comment thread src/bin/uudoc.rs Outdated
Comment thread src/bin/coreutils.rs Outdated
@sylvestre

Copy link
Copy Markdown
Contributor

Will create a separate PR to add a memory benchmark.

i am waiting for this to review this PR

@oech3

oech3 commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

@AnuthaDev

Copy link
Copy Markdown
Contributor Author

@sylvestre Please let me know if the existing benchmark pointed out by @oech3 is enough or I need to add a separate benchmark

@sylvestre

Copy link
Copy Markdown
Contributor

ok, thanks
there were below the threshold

@sylvestre sylvestre enabled auto-merge (squash) July 8, 2026 20:52
@AnuthaDev

Copy link
Copy Markdown
Contributor Author

@sylvestre I can see that you have enabled auto-merge, but its still pending on your approval since you had requested changes earlier.

@sylvestre sylvestre disabled auto-merge July 9, 2026 12:54
@sylvestre sylvestre enabled auto-merge (squash) July 9, 2026 12:54
@AnuthaDev

AnuthaDev commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

@sylvestre You need to click on "Approve PR" for it to get merged. Its blocked due to your earlier review.

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants