Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-5547

Don't delete HFiles when in "backup mode"

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.95.0
    • None
    • None
    • Reviewed
    • Hide
      All HFiles are now automatically archived to the configured "hbase.table.archive.directory". HFiles are cleaned similarly to HLogs - a cleaner delegate chain that is instantiated as per the configured classes. Similar to hlog cleaners, a TimeToLiveHFileCleaner is used by default, and specified under the "hbase.master.hfilecleaner.plugins" configuration key.

      In unifying the two cleaner interfaces, the new configuration parameter to use the TimeToLiveHLogCleaner is:
        <property>
          <name>hbase.master.logcleaner.plugins</name>
          <value>org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner</value>
       </property>

      It is still enabled by default, so nothing needs to be changed if you are not modifying the logcleaners.
      Show
      All HFiles are now automatically archived to the configured "hbase.table.archive.directory". HFiles are cleaned similarly to HLogs - a cleaner delegate chain that is instantiated as per the configured classes. Similar to hlog cleaners, a TimeToLiveHFileCleaner is used by default, and specified under the "hbase.master.hfilecleaner.plugins" configuration key. In unifying the two cleaner interfaces, the new configuration parameter to use the TimeToLiveHLogCleaner is:   <property>     <name>hbase.master.logcleaner.plugins</name>     <value>org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner</value>  </property> It is still enabled by default, so nothing needs to be changed if you are not modifying the logcleaners.

    Description

      This came up in a discussion I had with Stack.
      It would be nice if HBase could be notified that a backup is in progress (via a znode for example) and in that case either:
      1. rename HFiles to be delete to <file>.bck
      2. rename the HFiles into a special directory
      3. rename them to a general trash directory (which would not need to be tied to backup mode).

      That way it should be able to get a consistent backup based on HFiles (HDFS snapshots or hard links would be better options here, but we do not have those).

      #1 makes cleanup a bit harder.

      Attachments

        1. 5547.addendum-v3
          7 kB
          Ted Yu
        2. 5547-addendum-v4.txt
          0.7 kB
          Ted Yu
        3. 5547-v12.txt
          184 kB
          Ted Yu
        4. 5547-v16.txt
          196 kB
          Ted Yu
        5. hbase-5447-v8.patch
          168 kB
          Jesse Yates
        6. hbase-5447-v8.patch
          168 kB
          Ted Yu
        7. hbase-5547-0.94-backport-v0.patch
          147 kB
          Jesse Yates
        8. hbase-5547-v9.patch
          185 kB
          Jesse Yates
        9. java_HBASE-5547_v13.patch
          190 kB
          Jesse Yates
        10. java_HBASE-5547_v14.patch
          194 kB
          Jesse Yates
        11. java_HBASE-5547_v15.patch
          196 kB
          Ted Yu
        12. java_HBASE-5547_v4.patch
          195 kB
          Jesse Yates
        13. java_HBASE-5547_v5.patch
          195 kB
          Jesse Yates
        14. java_HBASE-5547_v6.patch
          195 kB
          Jesse Yates
        15. java_HBASE-5547_v7.patch
          162 kB
          Jesse Yates
        16. java_HBASE-5547.addendum
          7 kB
          Jesse Yates
        17. java_HBASE-5547.addendum-v1
          13 kB
          Jesse Yates
        18. java_HBASE-5547.addendum-v2
          7 kB
          Jesse Yates

        Issue Links

          blocks

          New Feature - A new feature of the product, which has yet to be developed. HBASE-6055 Offline Snapshots in HBase 0.96

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HBASE-6796 Backport HBASE-5547, Don't delete HFiles in backup mode.

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. HBASE-50 Snapshot of table

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          is required by

          Bug - A problem which impairs or prevents the functions of the product. HBASE-26718 HFileArchiver can remove referenced StoreFiles from the archive

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HBASE-6441 MasterFS doesn't set scheme for internal FileSystem

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. HBASE-6055 Offline Snapshots in HBase 0.96

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. HBASE-6205 Support an option to keep data of dropped table for some time

          • Major - Major loss of function.
          • Closed
          There are no Sub-Tasks for this issue.

          Activity

            People

              jesse_yates Jesse Yates
              larsh Lars Hofhansl
              Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: