Differentiate garbage marks - #4122
Conversation
e838c6c to
ad06444
Compare
2e836cc to
6a7846d
Compare
ad06444 to
f12d78c
Compare
6a7846d to
fd45d09
Compare
carpawell
left a comment
carpawell
left a comment
There was a problem hiding this comment.
We have implemented nspcc-dev/neofs-api#348 but not used anywhere as its usage was questionable. Then we have an optimistic #4125 that always tries to GET without indexes. Then we have this PR that drops indexes only as a fallback when normal placement does not work. So what do we do? @roman-khimov
There was a problem hiding this comment.
what is "an availability error"?
| // | ||
| // Returns [apistatus.ErrObjectAlreadyRemoved] if the object was marked for | ||
| // removal. | ||
| func (s *Service) restoreFromECPartsByRule(ctx context.Context, cnr cid.ID, parent oid.ID, rule iec.Rule, ruleIdx int, sortedNodes []netmap.NodeInfo) (object.Object, error) { |
There was a problem hiding this comment.
is it expected? we requested EC part but got a non-regular object
There was a problem hiding this comment.
received object ID and other err context may be useful. it is just part index is set, rule index is not or smth like that now
There was a problem hiding this comment.
i think it is the second time you decode attributes for the remote nodes case (getAnyECPartFromNode). can it be done in a single place?
There was a problem hiding this comment.
as i see, in the end there is always dst ObjectWriter but even for the happy path we are collecting all the EC parts in memory without streaming them, right? and after collecting them, we are even concatenating them into a single slice?
There was a problem hiding this comment.
what this part do? i think it is better to extend/change checkECPartInfoGetRequest to get what you need there without repeating
|
|
||
| // MarkGarbage marks objects to be physically removed from shard. Redundant | ||
| // objects remain readable until they are physically removed by GC. | ||
| func (db *DB) MarkGarbage(cnr cid.ID, addrs []oid.ID, mark GarbageMark) (ContainerGarbageDiff, error) { |
There was a problem hiding this comment.
i would check that the mark is known. otherwise, we may end up with a broken metabase with deleted objects no one knows why
|
Conflicts btw. |
|
GC marks are separate from #4119 to me. And #4119 itself has conceptual problems, think of REP handling wrt the same issue and object moved from B to A while fetcher goes from A to B. Nothing can help in this case, only some larger window of availability on B relative to GC doing its job. And it's exactly the same problem as with EC, so EC should not do any retries. Then #4125 is a different beast, it tries a different strategy and if it works better --- it should just be used as a replacement for current one. |
f12d78c to
ca245dd
Compare
ca245dd to
e493b7a
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #4122 +/- ##
==========================================
- Coverage 27.18% 27.12% -0.07%
==========================================
Files 677 678 +1
Lines 46210 46109 -101
==========================================
- Hits 12562 12505 -57
+ Misses 32447 32418 -29
+ Partials 1201 1186 -15 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
033a01d to
bf02a0f
Compare
Differentiate garbage marks for redundant policer copies from logical object removal. Keep such copies readable until GC removes them. Closes #4019. Signed-off-by: Andrey Butusov <andrey@nspcc.io>
bf02a0f to
cc1291b
Compare
3 participants
Closes #4019.