Skip to content

feat: Add --purge option to delete command to physically delete data files#192

Merged
subkanthi merged 1 commit into
masterfrom
191-delete-partition-add-option-to-physically-delete-data-files
Jul 21, 2026
Merged

feat: Add --purge option to delete command to physically delete data files#192
subkanthi merged 1 commit into
masterfrom
191-delete-partition-add-option-to-physically-delete-data-files

Conversation

@subkanthi

@subkanthi subkanthi commented Jul 20, 2026

Copy link
Copy Markdown
Collaborator

closes: #191

@subkanthi subkanthi linked an issue Jul 20, 2026 that may be closed by this pull request
Delete partition , add option to physically delete data files #191
Closed
@subkanthi

subkanthi commented Jul 20, 2026

Copy link
Copy Markdown
Collaborator Author 
./../ice/target/ice-0.0.0-SNAPSHOT-shaded.jar delete nyc.taxis_p_by_day  \
  --partition '[{"name": "tpep_pickup_datetime_day", "values": ["2025-01-01"]}]' --dry-run=false --purge
2026-07-20 16:41:04 [main/37045] INFO o.a.i.r.auth.AuthManagers > Loading AuthManager implementation: org.apache.iceberg.rest.auth.NoopAuthManager
                                   2026-07-20 16:41:05 [main/37045] INFO c.a.i.c.internal.cmd.Delete > Deleting s3a://bucket1/nyc/taxis_p_by_day/data/tpep_pickup_datetime_day=2025-01-01/payment_type=1/1784572748678-5526da589bc474f953669f343505ec94f0074bfc701eb046184de439bfe0485f-part.parquet
                                                                      2026-07-20 16:41:05 [main/37045] INFO c.a.i.c.internal.cmd.Delete > Deleting s3a://bucket1/nyc/taxis_p_by_day/data/tpep_pickup_datetime_day=2025-01-01/payment_type=0/1784572748678-5526da589bc474f953669f343505ec94f0074bfc701eb046184de439bfe0485f-part.parquet
                                                                                                         2026-07-20 16:41:05 [main/37045] INFO c.a.i.c.internal.cmd.Delete > Deleting s3a://bucket1/nyc/taxis_p_by_day/data/tpep_pickup_datetime_day=2025-01-01/payment_type=3/1784572748678-5526da589bc474f953669f343505ec94f0074bfc701eb046184de439bfe0485f-part.parquet
                             2026-07-20 16:41:05 [main/37045] INFO c.a.i.c.internal.cmd.Delete > Deleting s3a://bucket1/nyc/taxis_p_by_day/data/tpep_pickup_datetime_day=2025-01-01/payment_type=2/1784572748678-5526da589bc474f953669f343505ec94f0074bfc701eb046184de439bfe0485f-part.parquet
                                                                2026-07-20 16:41:05 [main/37045] INFO c.a.i.c.internal.cmd.Delete > Deleting s3a://bucket1/nyc/taxis_p_by_day/data/tpep_pickup_datetime_day=2025-01-01/payment_type=4/1784572748678-5526da589bc474f953669f343505ec94f0074bfc701eb046184de439bfe0485f-part.parquet
                                                                                                   2026-07-20 16:41:05 [main/37045] INFO o.a.i.SnapshotProducer > Committed snapshot 5770877521385373195 (BaseRewriteFiles)
                                                                                                            2026-07-20 16:41:05 [main/37045] INFO c.a.i.c.internal.cmd.Delete > Purging s3a://bucket1/nyc/taxis_p_by_day/data/tpep_pickup_datetime_day=2025-01-01/payment_type=1/1784572748678-5526da589bc474f953669f343505ec94f0074bfc701eb046184de439bfe0485f-part.parquet
                               2026-07-20 16:41:05 [main/37045] INFO c.a.i.c.internal.cmd.Delete > Purging s3a://bucket1/nyc/taxis_p_by_day/data/tpep_pickup_datetime_day=2025-01-01/payment_type=0/1784572748678-5526da589bc474f953669f343505ec94f0074bfc701eb046184de439bfe0485f-part.parquet
                                                                 2026-07-20 16:41:05 [main/37045] INFO c.a.i.c.internal.cmd.Delete > Purging s3a://bucket1/nyc/taxis_p_by_day/data/tpep_pickup_datetime_day=2025-01-01/payment_type=3/1784572748678-5526da589bc474f953669f343505ec94f0074bfc701eb046184de439bfe0485f-part.parquet
                                                                                                   2026-07-20 16:41:05 [main/37045] INFO c.a.i.c.internal.cmd.Delete > Purging s3a://bucket1/nyc/taxis_p_by_day/data/tpep_pickup_datetime_day=2025-01-01/payment_type=2/1784572748678-5526da589bc474f953669f343505ec94f0074bfc701eb046184de439bfe0485f-part.parquet
                      2026-07-20 16:41:05 [main/37045] INFO c.a.i.c.internal.cmd.Delete > Purging s3a://bucket1/nyc/taxis_p_by_day/data/tpep_pickup_datetime_day=2025-01-01/payment_type=4/1784572748678-5526da589bc474f953669f343505ec94f0074bfc701eb046184de439bfe0485f-part.parquet

@subkanthi
subkanthi marked this pull request as ready for review July 20, 2026 21:04
@subkanthi
subkanthi requested a review from xieandrew July 21, 2026 16:38
@subkanthi subkanthi added the ice Relates to ice label Jul 21, 2026
@xieandrew xieandrew changed the title Added logic to physically data files on deletion of partition. feat: Add --purge option to delete command to physically delete data files Jul 21, 2026
xieandrew
xieandrew approved these changes Jul 21, 2026
View reviewed changes

@xieandrew xieandrew left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@subkanthi
subkanthi merged commit 5d264d8 into master Jul 21, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@xieandrew xieandrew xieandrew approved these changes

Assignees

No one assigned

Labels

ice Relates to ice

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Delete partition , add option to physically delete data files

2 participants

@subkanthi @xieandrew