Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
2.6.0, 3.0.0
Description
The logic in BackupAdminImpl#finalizeDelete does not properly clean up tables from the incrementalBackupTableSet (= the set of backups to include in every incremental backup).
This can lead to backups failing.
Minimal example to reproduce from source:
- Add following to `conf/hbase-site.xml` to enable backups:
<property>
<name>hbase.backup.enable</name>
<value>true</value>
</property>
<property>
<name>hbase.master.logcleaner.plugins</name>
<value>org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner,org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner,org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner,org.apache.hadoop.hbase.backup.master.BackupLogCleaner</value>
</property>
<property>
<name>hbase.procedure.master.classes</name>
<value>org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager</value>
</property>
<property>
<name>hbase.procedure.regionserver.classes</name>
<value>org.apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureManager</value>
</property>
<property>
<name>hbase.coprocessor.region.classes</name>
<value>org.apache.hadoop.hbase.backup.BackupObserver</value>
</property>
<property>
<name>hbase.fs.tmp.dir</name>
<value>file:/tmp/hbase-tmp</value>
</property>
- Start HBase: bin/start-hbase.sh
echo "create 'table1', 'cf'" | bin/hbase shell -n echo "create 'table2', 'cf'" | bin/hbase shell -nbin/hbase backup create full file:/tmp/hbasebackups -t table1 bin/hbase backup create full file:/tmp/hbasebackups -t table2 bin/hbase backup create incremental file:/tmp/hbasebackups # Deletes the 2 most recent backups bin/hbase backup delete -l $(bin/hbase backup history | head -n1 | tail -n -1 | grep -o -P "backup_\d+"),$(bin/hbase backup history | head -n2 | tail -n -1 | grep -o -P "backup_\d+") bin/hbase backup create incremental file:/tmp/hbasebackups -t table1 bin/hbase backup history
- Output shows the incremental backup still includes table2, this should only be table1:
{ID=backup_1715000053763,Type=INCREMENTAL,Tables={table2,table1},State=COMPLETE,Start time=Mon May 06 14:54:14 CEST 2024,End time=Mon May 06 14:54:16 CEST 2024,Progress=100%}
{ID=backup_1715000031407,Type=FULL,Tables={table1},State=COMPLETE,Start time=Mon May 06 14:53:52 CEST 2024,End time=Mon May 06 14:53:54 CEST 2024,Progress=100%}
PR will follow soon.
(Edited: my original ticket included a stacktrace of an IllegalStateException from a PR for HBASE-28562)