Skip to content

fix(files): physical-generation snapshots reject database-loaded observation timestamps - #902

Open
kevinheneveld wants to merge 2 commits into
Listenarrs:canaryfrom
kevinheneveld:fix/physical-identity-utc-kind
Open

fix(files): physical-generation snapshots reject database-loaded observation timestamps#902
kevinheneveld wants to merge 2 commits into
Listenarrs:canaryfrom
kevinheneveld:fix/physical-identity-utc-kind

Conversation

@kevinheneveld

Copy link
Copy Markdown
Contributor

The bug

AudiobookFile.ApplyPhysicalObjectIdentity guards that its observedAtUtc argument has DateTimeKind.Utc — but SQLite/EF materializes the stored PhysicalIdentityObservedAtUtc column as Unspecified. Two paths re-apply that database-loaded value verbatim and throw:

  • AudiobookFileService.ClonePhysicalGenerationhit live on my instance: any rescan of a book that has an existing physical identity fails its scan job with System.ArgumentException: Physical identity observation time must be UTC. (stack: RefreshPhysicalGenerationCoreAsyncClonePhysicalGeneration). This bites more often after upgrading across the v1.3.0→v1.3.1 identity-scheme change, since stale identities now commonly trigger generation refreshes.
  • EfAudiobookFileRepository.ApplyPhysicalGeneration — same exposure when its source row came from the database.

The fix

Normalize with DateTime.SpecifyKind(value, DateTimeKind.Utc) at both re-apply sites — the same treatment RootFolderRelocationService.MetadataPlanning already applies to this exact column, so this just closes the two spots that were missed.

🤖 Generated with Claude Code

@kevinheneveld
kevinheneveld requested a review from a team August 26, 2026 00:25
kevinheneveld added a commit to kevinheneveld/Listenarr that referenced this pull request Aug 26, 2026
…s normalize to UTC kind (PR Listenarrs#902)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rvation timestamps

ApplyPhysicalObjectIdentity requires DateTimeKind.Utc, but SQLite
materializes the stored PhysicalIdentityObservedAtUtc as Unspecified.
Two paths re-apply that database-loaded value verbatim and throw
'Physical identity observation time must be UTC', failing the scan or
generation-refresh that triggered them: ClonePhysicalGeneration (hit
live on a rescan of a book with an existing physical identity) and
EfAudiobookFileRepository.ApplyPhysicalGeneration. Normalize with
DateTime.SpecifyKind, mirroring the treatment already in
RootFolderRelocationService.MetadataPlanning.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@therobbiedavis
therobbiedavis force-pushed the fix/physical-identity-utc-kind branch from 0c31a42 to c13420d Compare August 27, 2026 17:50
@therobbiedavis therobbiedavis added the patch patch version bump - backward compatible bug fixes label Aug 27, 2026
Comment on lines +436 to +438
DateTime.SpecifyKind(
source.PhysicalIdentityObservedAtUtc.Value,
DateTimeKind.Utc));

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we're repairing this one layer too late. PhysicalIdentityObservedAtUtc is UTC-by-contract — AudiobookFile.ApplyPhysicalObjectIdentity rejects anything else — but SQLite materialization currently allows an AudiobookFile to leave persistence with this property set to DateTimeKind.Unspecified.

Rather than requiring each consumer to know that persisted AudiobookFile timestamps need repairing, can we normalize this when EF materializes PhysicalIdentityObservedAtUtc, ideally in AudiobookFileConfiguration with a value conversion?

That would restore the entity invariant at the persistence boundary and keep ClonePhysicalGeneration persistence-agnostic. Otherwise the next consumer that reuses this property can reproduce the same failure.

Comment on lines +182 to +184
DateTime.SpecifyKind(
source.PhysicalIdentityObservedAtUtc.Value,
DateTimeKind.Utc));

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you clarify the production path this is intended to fix?

ApplyPhysicalGeneration appears to be used only by the non-relational fallback. SQLite takes the IsRelational() / ExecuteUpdateAsync path, and the source passed here is the caller-provided replacement rather than the database-loaded existing row.

So I don't think this method represents the SQLite failure described in the PR body. If we normalize PhysicalIdentityObservedAtUtc at EF materialization instead, I think this workaround becomes unnecessary as well.

If there is a path where a database-materialized source reaches this method, could we add a regression test demonstrating it?

@therobbiedavis therobbiedavis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The underlying bug looks valid, but I'd like regression coverage before we merge this.

Our normal BaseTests configuration uses EF InMemory, which preserves the DateTime.Kind and therefore cannot reproduce the SQLite behavior that caused this failure. The regression test needs to cross an actual SQLite persistence boundary:

Create an AudiobookFile with a UTC physical-identity observation timestamp.
Save it through SQLite.
Dispose that context.
Load the row through a fresh context.
Verify the loaded entity still satisfies the UTC contract.

I'd also like a test that drives the reported failure through RefreshPhysicalGenerationAsync using a persisted/reloaded file, since that proves the original ClonePhysicalGeneration exception cannot recur.

The fresh context is important here: an existing SQLite repository test saves and reloads through the same context, so EF can return the already-tracked UTC object instead of materializing the value back from SQLite. That's why the existing test suite doesn't catch this.

…edAtUtc at EF materialization

Review rework: instead of repairing DateTimeKind in each consumer, add a
materialization value conversion in AudiobookFileConfiguration so a loaded
AudiobookFile always satisfies its own UTC contract, and revert both
consumer-side SpecifyKind patches (the EfAudiobookFileRepository one also
turned out to be the non-relational fallback with a caller-provided source,
not the SQLite failure path).

Regression coverage crosses a real SQLite persistence boundary:
- fresh-context round-trip asserting the loaded timestamp keeps Utc kind
  and re-applies through the guarded domain method
- RefreshPhysicalGenerationAsync driven against a SQLite-backed provider
  with registration and refresh in separate resolutions, reproducing the
  original ClonePhysicalGeneration exception when the conversion is absent
  (verified red without it)

ServiceCollectionBuilder gains WithSqliteDatabase for tests that depend on
real SQLite materialization semantics, which EF InMemory cannot reproduce.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@kevinheneveld

Copy link
Copy Markdown
Contributor Author

Thanks for the review — all three points addressed in the follow-up commit:

  • Persistence-boundary fix: the repair now lives in AudiobookFileConfiguration as a materialization value conversion on PhysicalIdentityObservedAtUtc, so a loaded AudiobookFile always satisfies its own UTC contract and ClonePhysicalGeneration stays persistence-agnostic. Both consumer-side SpecifyKind patches are reverted.
  • EfAudiobookFileRepository.ApplyPhysicalGeneration: you're right — that's the non-relational fallback and its source is the caller-provided replacement, not a database-loaded row. Reverted; no production path there matches the reported failure.
  • Regression coverage (both crossing a real SQLite boundary):
    • AudiobookFilePhysicalIdentityTimestampTests saves through one context, disposes it, reloads through a fresh context on the same connection, and asserts the loaded timestamp keeps Utc kind and re-applies through the guarded domain method.
    • RefreshPhysicalGenerationAsync_SqlitePersistedFile_SnapshotsPredecessorWithoutUtcKindViolation drives the reported failure end-to-end against a SQLite-backed provider, with registration and refresh in separate resolutions so the refresh materializes the persisted row rather than reusing a tracked entity. Verified red without the conversion (it throws the exact original ArgumentException) and green with it.

Test infra note: ServiceCollectionBuilder gained an opt-in WithSqliteDatabase(connection) so service-level tests can run over real SQLite materialization when a regression depends on it — default remains InMemory.

kevinheneveld added a commit to kevinheneveld/Listenarr that referenced this pull request Aug 31, 2026
… EF materialization (PR Listenarrs#902)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
kevinheneveld added a commit to kevinheneveld/Listenarr that referenced this pull request Aug 31, 2026
…verter-only shape

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

patch patch version bump - backward compatible bug fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants