Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-14350 Procedure V2 Phase 2: Assignment Manager
  3. HBASE-18108

Procedure WALs are archived but not cleaned; fix

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 2.0.0
    • 2.0.0
    • proc-v2
    • None
    • Reviewed
    • Hide
      The archived Procedure WALs are moved to <hbase_root>/oldWALs/masterProcedureWALs
      directory. TimeToLiveProcedureWALCleaner class was added which regularly cleans the Procedure WAL files from there.

      The TimeToLiveProcedureWALCleaner is added to hbase.master.logcleaner.plugins configuration value.

      A new config parameter is added: hbase.master.procedurewalcleaner.ttl, which specifies how long a Procedure WAL should stay in the archive directory.
      Show
      The archived Procedure WALs are moved to <hbase_root>/oldWALs/masterProcedureWALs directory. TimeToLiveProcedureWALCleaner class was added which regularly cleans the Procedure WAL files from there. The TimeToLiveProcedureWALCleaner is added to hbase.master.logcleaner.plugins configuration value. A new config parameter is added: hbase.master.procedurewalcleaner.ttl, which specifies how long a Procedure WAL should stay in the archive directory.

    Description

      The Procedure WAL files used to be deleted when done. HBASE-14614 keeps them around in case issue but what is missing is a GC for no-longer-needed WAL files. This one is pretty important.

      From WALProcedureStore Cleaner TODO in https://docs.google.com/document/d/1eVKa7FHdeoJ1-9o8yZcOTAQbv0u0bblBlCCzVSIn69g/edit#heading=h.r2pc835nb7vi

      Attachments

        1. HBASE-18108.master.001.patch
          21 kB
          Peter Somogyi
        2. HBASE-18108.master.002.patch
          22 kB
          Peter Somogyi
        3. HBASE-18108.master.003.patch
          27 kB
          Peter Somogyi
        4. HBASE-18108.master.004.patch
          27 kB
          Peter Somogyi
        5. HBASE-18108.master.004.patch
          27 kB
          Peter Somogyi
        6. HBASE-18108.master.005.patch
          27 kB
          Peter Somogyi

        Issue Links

          links to

          Web Link Review Board (master)

          Activity

            People

              psomogyi Peter Somogyi
              stack Michael Stack
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: