Uploaded image for project: 'Jackrabbit Oak'
  1. Jackrabbit Oak
  2. OAK-4649

Move index files outside of the TAR files

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Reopened
    • Major
    • Resolution: Unresolved
    • None
    • None
    • segment-tar

    Description

      TAR files currently embed three indexes: an index of the segments contained in the TAR files, a graph index and an index of external binary references.

      Index files are checked for consistency purposes at the startup of the system. Normally, if an index file is corrupted it is recreated. Since the index file is contained inside the TAR file, recreating them implies rewriting the whole TAR file and appending the new index.

      This process creates unnecessary backups, since the biggest part of the TAR file is effectively immutable. Moreover, because index files are stored in the TAR files, we can't treat TAR files as true read-only files. There is always the possibility that they have to be opened again in write mode for the recovery of the index file.

      I propose to move those index files outside of the TAR files. TAR files will end up being truly read-only files containing immutable data, and index files will be granted their own physical files on the file system. Being index files derived data, they now can be recreated at will without impacting the read-only part of the segment store.

      Attachments

        Activity

          People

            Unassigned Unassigned
            frm Francesco Mari
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: