Skip to content

feat: add table-references command group - #22

Merged
andrewchester-dm merged 2 commits into
mainfrom
feat/DM-4183-table-references-cli
Aug 25, 2026
Merged

feat: add table-references command group#22
andrewchester-dm merged 2 commits into
mainfrom
feat/DM-4183-table-references-cli

Conversation

@andrewchester-dm

@andrewchester-dm andrewchester-dm commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Adds a table-references command group (list/get/create/update/delete) to dm, mirroring the existing connections command group, over `datamasque-python TableReferenceClient.

  • create accepts --file (JSON) or flags for the common case; source is a file path for file connections or schema.table for database connections, with format always explicit, never inferred.
  • update changes only the fields passed; any CSV/format flag replaces options wholesale on the resulting PUT.
  • A 404 from a server that predates table references surfaces as "not supported by this DataMasque version" rather than a generic failure.
    Bumps the datamasque-python floor to >=1.2.5,<2.

@ClassicMMT

Copy link
Copy Markdown
Contributor

Good job looks great!

Left a few small comments of things to look into.

Please make sure you add a PR description and bump dm-python to >=1.2.5.

Comment thread src/datamasque_cli/commands/table_references.py Outdated
Comment thread src/datamasque_cli/commands/table_references.py Outdated
Comment thread src/datamasque_cli/commands/table_references.py Outdated

@ClassicMMT ClassicMMT left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good. Almost there!

Comment thread src/datamasque_cli/commands/table_references.py Outdated
Comment thread src/datamasque_cli/commands/table_references.py Outdated
@andrewchester-dm
andrewchester-dm force-pushed the feat/DM-4183-table-references-cli branch from 7bea2fb to 792ad95 Compare August 24, 2026 01:54

@ClassicMMT ClassicMMT left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. Just two last things left to make it look a bit better.

Comment thread src/datamasque_cli/commands/table_references.py Outdated
Comment thread src/datamasque_cli/commands/table_references.py Outdated
Comment thread README.md
Comment thread CHANGELOG.md Outdated
@andrewchester-dm
andrewchester-dm force-pushed the feat/DM-4183-table-references-cli branch from 792ad95 to 8435e54 Compare August 24, 2026 04:45
@andrewchester-dm

Copy link
Copy Markdown
Contributor Author

Tests I ran

  • create --name customers with missing flags → confirmed clean usage error

  • create --name customers --connection test_files --source ... → success

  • get customers --json → confirmed all defaults filled in correctly

  • list → confirmed table rendering

  • create --file /tmp/orders.json → success, get orders --json → confirmed

  • delete orders / delete customers (with y/N prompt) → both correct

  • Repeated create --name test_files ... three times → confirmed create-or-update (idempotent overwrite), no "duplicate name" error

  • update test_files (no fields) → confirmed "pass at least one field" error

  • connections create throwaway → connections delete throwaway → table-references create --connection throwaway → got "not found" (the flag-path limitation we then investigated and logged)

  • update test_files --delimiter ';' → confirmed full-replace-not-merge behavior

  • Malformed --file ({"name": "bad"}) → confirmed clean error, no traceback

  • get/update/delete on a nonexistent name → confirmed clean "not found" on all three

  • --connection by UUID (flag form) → confirmed equivalent to name form

  • --file combined with another flag → confirmed rejection, confirmed target record unchanged

  • Archived-connection rejection via --file (raw ID, throwaway2) → confirmed real server message: "Cannot reference an archived connection."

  • Cleaned up all test clutter, then real Postgres connection (library db) + dm connections test → confirmed live reachability

  • Table reference against real schema.table source (public.details), then update to public.drivers license → confirmed DB-connection path works

  • Unsupported connection type (real MongoDB connection object) → confirmed clean rejection

  • "Not supported by this DataMasque version" (temporarily disabled the URL route, confirmed 404→clean CLI message, reverted) → confirmed

  • TREF011 JSON-side masking run — first attempt failed (later traced to a stale build cache on my end, not a real bug; rebuilt fresh and re-ran successfully)

  • TREF011 DB-side masking run against a real local Postgres I set up — first comparison against the JSON side mismatched (traced to independent per-run random salts, not a bug); re-ran both sides with a shared run_secret → confirmed byte-identical masked output across both engines (the actual cross-system consistency guarantee)

  • TREF012 (NDJSON, independent per-record resolution) → confirmed same-canonical records mask alike, others don't collide

  • TREF013 (JSON on_missing: error) → confirmed exact error text, whole-task failure

  • TREF014 (JSON on_missing: self) → confirmed deterministic self-seed, warning logged exactly once

  • TREF017 (table_reference + force_consistency: true) → confirmed rejected at prepare time, exact error text

  • TREF019 (null source_key canonicalization) → confirmed nulls mask alike, non-null miss doesn't collide

  • TREF021 (string "800" vs number 800 key equivalence) → confirmed identical resolution

  • TREF022 (object source_key) → confirmed rejection, exact error text

  • TREF023 (array source_key) → confirmed rejection, exact error text

  • Verified all 6 GitHub Actions CI checks green on the final pushed commit

@ClassicMMT ClassicMMT left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. @kw-datamasque all good?

@ClassicMMT ClassicMMT left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. @kw-datamasque all good?

@kw-datamasque

Copy link
Copy Markdown
Collaborator

Feel free to create a new release once merged in!

@andrewchester-dm
andrewchester-dm merged commit d9bd888 into main Aug 25, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants