Skip to content

Serialize concurrent runs and atomic-write credential/config dotfiles - #9

Merged
nijave merged 1 commit into
mainfrom
feature/amplify-locking-atomic-writes
Sep 19, 2026
Merged

nijave merged 1 commit into
mainfrom
feature/amplify-locking-atomic-writes

Conversation

@nijave

@nijave nijave commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

What & why

Ports the atomic-write + cross-process file-locking robustness improvement from the heavily-diverged fork amplify-education/okta-awscli. This is the self-contained core of amplify's concurrency suite: it makes the two dotfiles this tool writes — ~/.aws/credentials and ~/.okta-aws — safe to write from parallel okta-awscli invocations (e.g. CI/automation running many profiles at once), which previously could interleave and corrupt the files or silently clobber each other's updates.

How it works:

  • New oktaawscli/_locking.py module: locked(path) returns a filelock.FileLock on <path>.lock; atomic_write(path) writes to a temp file next to the target and os.replace()s it in (POSIX-atomic same-FS rename), removing the temp and leaving the original untouched if the writer raises.
  • aws_auth.py: write_sts_token and set_default_profile now write the credentials file under a lock via atomic_write. check_sts_token's default-mirror write is also now taken under the lock.
  • okta_auth_config.py: write_role_to_profile / write_applink_to_profile acquire the lock, re-read the config inside the lock and merge (so a peer's concurrent write to another key/section is preserved), then atomic_write.
  • okta_awscli.py: a lock-acquisition timeout surfaces as a friendly CLI error instead of an unhandled filelock.Timeout traceback.
  • Adds filelock as a runtime dependency (pyproject.toml, requirements.txt pinned to 3.16.1 for Python 3.8 compatibility, uv.lock regenerated — filelock-only diff).

Provenance — adapted by hand (needs review)

amplify's base is an old 0.4.x tree with a very different AwsAuth/auth architecture (different constructor and method signatures, an Okta session cache, inline MFA), so nothing cherry-picked cleanly — this was reimplemented by hand on our current main and should be reviewed as new code, not a mechanical port. Credit to amplify-education/okta-awscli, source commits:

  • 02474e0 Add _locking module with locked() and atomic_write() helpers
  • 14897cc Lock and atomic-write write_sts_token
  • 0d27063 Lock and atomic-write copy_to_default (our set_default_profile)
  • 144f2ef Lock and merge-on-write _save_config_value (our config writers)
  • 33f9046 Handle filelock.Timeout with a friendly CLI error

Behavior changes (this is NOT a no-op refactor)

This PR intentionally changes observable behavior:

  1. Serialization — concurrent okta-awscli processes now block on a per-file advisory lock (~/.aws/credentials.lock, ~/.okta-aws.lock) while writing, instead of racing. Lock timeout is 60s.
  2. New failure mode — if the lock can't be acquired within 60s, the run prints Could not acquire lock on <file> - another okta-awscli process is holding it. Try again. and exits 1 (previously there was no lock, so no such path).
  3. Atomic replace instead of in-place truncate — the credentials/config files are now replaced via a temp file + os.replace, so the file inode changes on each write and the new file carries mkstemp's 0600 mode rather than the umask-derived mode of the old in-place open(..., 'w+') (typically 0644). For a credentials file 0600 is arguably desirable, but it is a change.
  4. Merge-on-write for ~/.okta-aws — the config writers now re-read the file under the lock before writing, so they no longer overwrite the whole file from a possibly-stale in-memory parser; keys/sections written by a concurrent process are preserved.
  5. New runtime dependency: filelock.

The written contents (keys/sections/values) are unchanged in the single-process case; only the write mechanism and concurrency semantics change.

What I deliberately did NOT port (out of scope / too entangled)

  • amplify's serialize-primary_auth lock and Okta rate-limit (E0000047) retry (31b7308, c66b1b7): these are woven into amplify's diverged okta_auth.py (session cache, _exit_on_okta_error/_okta_json_request helpers, inline MFA) that our base doesn't have. Porting them would mean dragging in that machinery or a risky reimplementation of the auth flow — not self-contained. Left for a separate, dedicated PR.
  • amplify's session-cache atomic write (2175a65): our base has no ~/.okta-token session cache, so it doesn't apply.
  • amplify's test scaffolding / tox / CI (102f37990): the repo's .gitignore excludes tests/*, so committing tests would need a .gitignore change (separate concern). I wrote and ran unit tests for the new _locking module locally (7 tests, all pass) but did not commit them, to respect the existing convention.
  • No version bump, no reformatting, no company-specific files, no unrelated cleanups.

Verification

  • python -m py_compile on all changed modules: pass.
  • import oktaawscli + all changed submodules: pass.
  • pylint --errors-only on the changed files: clean. (Note: pylint --errors-only oktaawscli reports a pre-existing E0606 in okta_auth.py:76 that exists on main and is untouched by this PR.)
  • Local unit tests for _locking (locked/atomic_write, timeout, atomic-rollback): 7/7 pass.
  • Functional smoke test with a temp $HOME: write_sts_token writes the profile + default mirror atomically with no leftover temp files; ~/.okta-aws writers preserve a pre-existing key (merge-on-write verified).

@nijave
nijave force-pushed the feature/amplify-locking-atomic-writes branch from 422d0fe to deaf8a3 Compare September 18, 2026 23:08
@nijave
nijave changed the base branch from main to feature/kvinck-patches September 18, 2026 23:08
@nijave
nijave force-pushed the feature/kvinck-patches branch from c677e9a to c1c8ba1 Compare September 18, 2026 23:50
Wrap the writes to ~/.aws/credentials (write_sts_token / set_default_profile)
and ~/.okta-aws (write_role_to_profile / write_applink_to_profile) with a
cross-process advisory lock (filelock) and an atomic temp-file + os.replace,
so parallel okta-awscli invocations can no longer corrupt or clobber each
other's config. The ~/.okta-aws writers now re-read under the lock and merge,
preserving keys written by a peer. A lock-acquisition timeout surfaces as a
friendly CLI error instead of a traceback.

Adds a small oktaawscli/_locking module (locked(), atomic_write()) and a
filelock runtime dependency.

Adapted by hand from amplify-education/okta-awscli (their base is 0.4.x, so
the change was reimplemented on our code rather than cherry-picked):
  02474e0 Add _locking module with locked() and atomic_write() helpers
  14897cc Lock and atomic-write write_sts_token
  0d27063 Lock and atomic-write copy_to_default
  144f2ef Lock and merge-on-write _save_config_value
  33f9046 Handle filelock.Timeout with a friendly CLI error
@nijave
nijave force-pushed the feature/amplify-locking-atomic-writes branch from deaf8a3 to ac34fa4 Compare September 18, 2026 23:50
Base automatically changed from feature/kvinck-patches to feature/aws-partitions-eusc-cn September 19, 2026 01:42
Base automatically changed from feature/aws-partitions-eusc-cn to chore/drop-python-3.9 September 19, 2026 01:43
Base automatically changed from chore/drop-python-3.9 to main September 19, 2026 01:44
@nijave
nijave merged commit b61d481 into main Sep 19, 2026
2 checks passed
@nijave
nijave deleted the feature/amplify-locking-atomic-writes branch September 19, 2026 01:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant