Skip to content

NIFI-16159 Add support for FileResourceService in PutFile - #11495

Merged
exceptionfactory merged 1 commit into
apache:mainfrom
shubhluck:NIFI-16159
Aug 30, 2026
Merged

NIFI-16159 Add support for FileResourceService in PutFile#11495
exceptionfactory merged 1 commit into
apache:mainfrom
shubhluck:NIFI-16159

Conversation

@shubhluck

Copy link
Copy Markdown
Contributor

Summary

NIFI-16159

Add support for FileResourceService in PutFile, matching the Resource Transfer Source / File Resource Service pattern already available in PutS3Object (NIFI-12642), PutGCSObject (NIFI-12643), and PutHDFS (NIFI-12801).

PutFile currently writes only the incoming FlowFile's content to the destination directory. Users who want to stage a file already on the local filesystem (or reachable via a custom FileResourceService implementation) have to first pull it into a FlowFile with FetchFile/GetFile, which is wasteful for large local files.

Changes

  • Add Resource Transfer Source and File Resource Service property descriptors to PutFile, reusing ResourceTransferProperties from nifi-resource-transfer.
  • When Resource Transfer Source = File Resource Service, obtain a FileResource via ResourceTransferUtils.getFileResource(...) and stream its InputStream to the .<filename> temp file with StandardCopyOption.REPLACE_EXISTING (preserving the overwrite semantics of the original session.exportTo(...) call), then proceed with the existing rename / permissions / owner / group / lastModified logic.
  • When Resource Transfer Source = FlowFile Content (default) the behavior is byte-identical to today.
  • Add nifi-resource-transfer and nifi-file-resource-service-api (compile) and nifi-file-resource-service (test) to nifi-standard-processors/pom.xml.
  • Add TestPutFile#testPutFileFromLocalFile covering the FileResourceService path using StandardFileResourceService.

Backward compatibility

Resource Transfer Source defaults to FlowFile Content, so existing flows are unaffected. File Resource Service only appears in the UI when the source is set to File Resource Service (declared via dependsOn).

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

Pull Request Tracking

  • Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
  • Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000
  • Pull request contains commits signed with a registered key indicating Verified status

Pull Request Formatting

  • Pull Request based on current revision of the main branch
  • Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

  • Build completed using ./mvnw clean install -P contrib-check
    • JDK 21
    • JDK 25

Licensing

  • New dependencies are compatible with the Apache License 2.0 according to the License Policy
  • New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

  • Documentation formatting appears as expected in rendered files

@shubhluck

Copy link
Copy Markdown
Contributor Author

The TestSocketLoadBalancedFlowFileQueue.testGetQueueSnapshotIncludesRebalancingPartition failure on the JDK 21 EN leg is unrelated to this change (which only touches nifi-standard-processors/PutFile). The other three matrix legs passed. Looks like a flaky test.

@exceptionfactory exceptionfactory left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for proposing this addition @shubhluck. Although using the File Resource Service for local transfer is a bit of an edge case, it is minimal enough that it looks straightforward. I noted a few minor recommendations, then this should be ready to go.

Comment thread nifi-extension-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml Outdated

@exceptionfactory exceptionfactory left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @shubhluck, the latest version looks good! +1 merging

@exceptionfactory
exceptionfactory merged commit 5c9093a into apache:main Aug 30, 2026
8 checks passed
@shubhluck
shubhluck deleted the NIFI-16159 branch August 30, 2026 02:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants