Serialize concurrent runs and atomic-write credential/config dotfiles - #9
Merged
Merged
Conversation
nijave
force-pushed
the
feature/amplify-locking-atomic-writes
branch
from
September 18, 2026 23:08
422d0fe to
deaf8a3
Compare
nijave
force-pushed
the
feature/kvinck-patches
branch
from
September 18, 2026 23:50
c677e9a to
c1c8ba1
Compare
Wrap the writes to ~/.aws/credentials (write_sts_token / set_default_profile) and ~/.okta-aws (write_role_to_profile / write_applink_to_profile) with a cross-process advisory lock (filelock) and an atomic temp-file + os.replace, so parallel okta-awscli invocations can no longer corrupt or clobber each other's config. The ~/.okta-aws writers now re-read under the lock and merge, preserving keys written by a peer. A lock-acquisition timeout surfaces as a friendly CLI error instead of a traceback. Adds a small oktaawscli/_locking module (locked(), atomic_write()) and a filelock runtime dependency. Adapted by hand from amplify-education/okta-awscli (their base is 0.4.x, so the change was reimplemented on our code rather than cherry-picked): 02474e0 Add _locking module with locked() and atomic_write() helpers 14897cc Lock and atomic-write write_sts_token 0d27063 Lock and atomic-write copy_to_default 144f2ef Lock and merge-on-write _save_config_value 33f9046 Handle filelock.Timeout with a friendly CLI error
nijave
force-pushed
the
feature/amplify-locking-atomic-writes
branch
from
September 18, 2026 23:50
deaf8a3 to
ac34fa4
Compare
Base automatically changed from
feature/kvinck-patches
to
feature/aws-partitions-eusc-cn
September 19, 2026 01:42
Base automatically changed from
feature/aws-partitions-eusc-cn
to
chore/drop-python-3.9
September 19, 2026 01:43
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Ports the atomic-write + cross-process file-locking robustness improvement from the heavily-diverged fork
amplify-education/okta-awscli. This is the self-contained core of amplify's concurrency suite: it makes the two dotfiles this tool writes —~/.aws/credentialsand~/.okta-aws— safe to write from parallelokta-awscliinvocations (e.g. CI/automation running many profiles at once), which previously could interleave and corrupt the files or silently clobber each other's updates.How it works:
oktaawscli/_locking.pymodule:locked(path)returns afilelock.FileLockon<path>.lock;atomic_write(path)writes to a temp file next to the target andos.replace()s it in (POSIX-atomic same-FS rename), removing the temp and leaving the original untouched if the writer raises.aws_auth.py:write_sts_tokenandset_default_profilenow write the credentials file under a lock viaatomic_write.check_sts_token's default-mirror write is also now taken under the lock.okta_auth_config.py:write_role_to_profile/write_applink_to_profileacquire the lock, re-read the config inside the lock and merge (so a peer's concurrent write to another key/section is preserved), thenatomic_write.okta_awscli.py: a lock-acquisition timeout surfaces as a friendly CLI error instead of an unhandledfilelock.Timeouttraceback.filelockas a runtime dependency (pyproject.toml,requirements.txtpinned to3.16.1for Python 3.8 compatibility,uv.lockregenerated — filelock-only diff).Provenance — adapted by hand (needs review)
amplify's base is an old 0.4.x tree with a very different
AwsAuth/auth architecture (different constructor and method signatures, an Okta session cache, inline MFA), so nothing cherry-picked cleanly — this was reimplemented by hand on our currentmainand should be reviewed as new code, not a mechanical port. Credit toamplify-education/okta-awscli, source commits:02474e0Add_lockingmodule withlocked()andatomic_write()helpers14897ccLock and atomic-writewrite_sts_token0d27063Lock and atomic-writecopy_to_default(ourset_default_profile)144f2efLock and merge-on-write_save_config_value(our config writers)33f9046Handlefilelock.Timeoutwith a friendly CLI errorBehavior changes (this is NOT a no-op refactor)
This PR intentionally changes observable behavior:
okta-awscliprocesses now block on a per-file advisory lock (~/.aws/credentials.lock,~/.okta-aws.lock) while writing, instead of racing. Lock timeout is 60s.Could not acquire lock on <file> - another okta-awscli process is holding it. Try again.and exits 1 (previously there was no lock, so no such path).os.replace, so the file inode changes on each write and the new file carriesmkstemp's0600mode rather than the umask-derived mode of the old in-placeopen(..., 'w+')(typically0644). For a credentials file0600is arguably desirable, but it is a change.~/.okta-aws— the config writers now re-read the file under the lock before writing, so they no longer overwrite the whole file from a possibly-stale in-memory parser; keys/sections written by a concurrent process are preserved.filelock.The written contents (keys/sections/values) are unchanged in the single-process case; only the write mechanism and concurrency semantics change.
What I deliberately did NOT port (out of scope / too entangled)
primary_authlock and Okta rate-limit (E0000047) retry (31b7308,c66b1b7): these are woven into amplify's divergedokta_auth.py(session cache,_exit_on_okta_error/_okta_json_requesthelpers, inline MFA) that our base doesn't have. Porting them would mean dragging in that machinery or a risky reimplementation of the auth flow — not self-contained. Left for a separate, dedicated PR.2175a65): our base has no~/.okta-tokensession cache, so it doesn't apply.102f37990): the repo's.gitignoreexcludestests/*, so committing tests would need a.gitignorechange (separate concern). I wrote and ran unit tests for the new_lockingmodule locally (7 tests, all pass) but did not commit them, to respect the existing convention.Verification
python -m py_compileon all changed modules: pass.import oktaawscli+ all changed submodules: pass.pylint --errors-onlyon the changed files: clean. (Note:pylint --errors-only oktaawsclireports a pre-existingE0606inokta_auth.py:76that exists onmainand is untouched by this PR.)_locking(locked/atomic_write, timeout, atomic-rollback): 7/7 pass.$HOME:write_sts_tokenwrites the profile + default mirror atomically with no leftover temp files;~/.okta-awswriters preserve a pre-existing key (merge-on-write verified).