Details
-
New Feature
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
Description
On big repositories compaction can take quite a while to run as it needs to create a full deep copy of the current root node state. For such cases it could be beneficial if we could partially compact the repository thus splitting full compaction over multiple cycles.
Partial compaction would run compaction on a sub-tree just like we now run it on the full tree. Afterwards it would create a new root node state by referencing the previous root node state replacing said sub-tree with the compacted one.
Todo: Asses feasibility and impact, implement prototype.
Attachments
Attachments
Issue Links
- blocks
-
OAK-6399 Re-establish full offline compaction functionality
- Closed
- depends upon
-
OAK-6009 Simplify cancellation of compaction by timeout
- Resolved
-
OAK-5996 Revisions.setHead(Function) should return the new head or null instead of boolean
- Closed
-
OAK-6005 Add record id of the compacted root to the GC journal
- Closed
- is blocked by
-
OAK-4122 Replace the commit semaphore in the segment node store with a scheduler
- Closed
- is related to
-
OAK-6456 Include the tail generation in the TAR index
- Closed
-
OAK-6457 Increment the segment version number
- Closed
-
OAK-6458 Increment the manifest version number
- Closed
-
OAK-6468 Include the tail generation in the binary references index
- Closed
-
OAK-6484 Include tail generation in the GC journal
- Closed
-
OAK-6485 Expose tail and full compaction via different methods
- Closed
- relates to
-
OAK-6520 Improve tail compactions resilience when base state cannot be determined
- Closed
-
OAK-6519 Properly handle tail compactions in deduplication caches
- Closed
-
OAK-6522 Implement unit tests for OnlineCompactor
- Closed
- links to
See my comment on
OAK-4122on how such an approach could look like.