Add a safe compatibility path for opaque FUSE imports - #905
Draft
dktrekkie wants to merge 2 commits into
Draft
Conversation
Classify filesystem-object handle trust once at the Linux generation-candidate funnel. Auto mode keeps known non-FUSE behavior and inode-generation evidence, while refusing opaque handles on FUSE or when filesystem type is unavailable.
Read an optional named client setting once per processing job and carry it through the import boundary. Existing clients default to Auto, while mixed archive or staging inputs conservatively fall back to whole-set Auto.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This is a compact proof of concept for #836. It addresses two independent
filesystem boundaries exposed by an unattended download import on an Unraid
user share:
filesystem provides the persistence guarantees required for durable object
identity.
explicit case semantics when read-only automatic detection is inconclusive.
The implementation deliberately adds generic policy and carrier seams instead
of special-casing Unraid,
shfs, a download-client vendor, or a mount path. Itmakes unattended imports safe and functional through Listenarr's existing
compatibility copy-and-retain path.
This is proposed as an interim compatibility path, not as the final hardlink
solution for Unraid. Its purpose is to unblock affected imports without granting
destructive authority from identity evidence that was reproduced as unstable.
It gives the project room to design verified hardlink publication for weak
storage separately. Until then, affected
HardlinkCopyimports complete asfull copies and retain their sources, with the time and storage cost described
below.
Reproduced failure sequence
The download client completed successfully, and Listenarr could enumerate and
probe every source file. Import then failed before publication because the
library root no longer matched its persisted physical identity. Confirming the
root only helped temporarily: the opaque
fuse.shfshandle changed after thesame directory was looked up again, so Listenarr treated the unchanged path as
a different directory.
Making FUSE identity fail closed allowed the root to enter Listenarr's existing
limited-storage contract, but exposed a second boundary. The translated local
download source is outside the library root and still resolved case semantics
with
Auto. Onfuse.shfs, the flags ioctl was unsupported and the read-onlyexisting-entry probe was inconclusive, so import stopped before publication.
With the final PoC image, the library root and translated source were configured
explicitly as
Sensitive. The same unattended workflow then completed: fourrelease artifacts were compatibility-published, the audio file was registered,
the follow-up scan completed, and all source files were retained.
Why this design
The initial working prototype inferred case behavior from the exact
fuse.shfsmount subtype. That solved the reproduced system but coupled genericfilesystem code to one Unraid implementation. It was removed.
The final design instead separates two questions:
Both decisions are named and independently replaceable. The identity decision
is enforced once at the existing Linux generation-candidate funnel, and the
source setting is parsed once per processing job. No per-file probing loop,
trust cache, vendor detection, or downloader-specific branch is introduced.
Changes
Filesystem-object identity trust
Auto/Trusted/Untrustedtrust policy.every consumer gets the same result without call-site plumbing.
Auto, keep opaque handles on known non-FUSE filesystems, but do notprobe or persist them on FUSE or when filesystem type is unavailable.
zero, and retain the target branch's existing FID/fallback behavior.
fstatfshelper with the existing filesystem-semanticscode. No trust decision is cached.
Download-source case semantics
sourceCaseSensitivityMode, defaulting toAutofor existing clients.resolution of the translated local source path.
Autowhen archive extraction or staging means a fileset no longer has one trustworthy source endpoint.
Hardlink behavior and trade-off
Listenarr already contains native Linux hardlink support, and this PoC does not
modify that implementation. The existing publication planner only selects its
durable hardlink path when source object identity is durable and the destination
authorizes filesystem mutation.
The configured action is named
HardlinkCopy, and Listenarr already treats copyas its fallback when a hardlink cannot be created. This PoC therefore uses an
existing product fallback rather than inventing a new publication behavior.
However, the distinction still matters: on the affected weak FUSE path the
fallback becomes systematic, so the user's performance and storage goal is not
met even though the import itself succeeds.
Under this PoC,
Autodoes not grant that authority from an opaque FUSE handle.The planner therefore selects
AdditiveCopyRetainSource, even when the requestedcompleted-file action is
HardlinkCopy. Compatibility publication creates anew file, copies and flushes all bytes, verifies the destination, registers it,
and retains the source.
The end-to-end validation source and destination were exposed through one
container mount and were physically located on the same backing ZFS pool, so
cross-device layout was not the reason for the fallback. A post-import check
showed different inodes with link count one: the successful result was a full
copy, not a hardlink.
The fallback is not represented internally as a successful hardlink. Each
import result preserves
RequestedAction=HardlinkCopyandEffectiveAction=Copy, together withdurable_identity_unavailableand acompatibility-publication message. Those details are recorded in history,
although the current UI may not make the distinction prominent.
This is a deliberate safety trade-off, not a claim that Unraid cannot support
hardlinks. It means imports take copy time and consume duplicate space while the
source remains present. The unresolved product work is to support verified
hardlink publication on weak storage without treating link success as proof
that later destructive cleanup is safe. That likely requires additional
journaling, verification, policy, and UI work, but not necessarily a rewrite of
the native hardlink implementation. That extension is intentionally outside
this PoC.
Intentional PoC boundary
This PR demonstrates the architecture and provides a usable mitigation for the
reproduced import without claiming to finish the product design. It can be
reviewed either as a small compatibility change that is safe to ship while the
hardlink path is designed, or as a concrete reference implementation for that
later work. It does not add UI, API schema, migration, automatic Unraid
detection, downloader-specific behavior, or a final choice of where the
identity-trust setting should live.
For validation,
sourceCaseSensitivityMode=Sensitivewas added manually to theexisting download-client settings JSON; no database column was introduced. The
current UI does not carry this unknown property and can remove it when a client
is edited. Likewise, the identity policy currently uses
Autoat the seam anddoes not expose a product-level carrier. These are explicit PoC limitations,
not finished configuration UX.
Maintainer decisions intentionally left open are:
from destructive move/source-cleanup authority.
Testing
dotnet format listenarr.slnx --no-restore --verify-no-changes: passed.same four unrelated baseline failures remained.
the same four established baseline failures remained.
48 passed, 15 platform skips, and zero failures.
control expose the same raw handle shape. Production policy rejected the
FUSE handle while retaining usable identity on XFS and ZFS controls.
and case-sensitive on the affected system.
and returned HTTP 200 in an isolated smoke container.
validates the generic path with the client used to reproduce FUSE mounts pass the durable-identity probe:
name_to_handle_atreturns an ephemeral nodeid, so the root folder isHealthywith mutation until the inode is evicted #836; the codecontains no SABnzbd-specific branch. Four release artifacts were
compatibility-published, the audio file was registered and scanned, and all
four source files were retained. Destructive filesystem authority remained
disabled.
Notes
copy-and-retain path; no source-deletion authority was enabled.
already disclosed in the issue discussion.