HBase Backup/Restore Phase 3: Filter WALs on backup to include only edits from backed up tables

High level design overview

• When incremental backup request comes for tables {t} we select all the tables already registered in a backup system - {T} and union them with {t}

  , which results in a new table set - U(t, T)

• For every table K from U(t,T) we perform the following:
  • Convert new WAL files into HFile applying table filter K (only edits for table T will pass the filter)
  • Move these HFile(s) to backup destination

During restore (incremental):

• We run full restore first
• Then collect all HFiles from intermediate incremental images and run them through HFileSplitterJob, which splits files into a current tables region boundaries
• Load these files using LoadIncrementalHFiles tool