Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-7593

Supporting HSync and lease recovery

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.4.0
    • 2.0.0
    • None
    • None

    Description

      This is the umbrella jira encompassing the design and implementation of hflush API and lease recovery support in Ozone. This feature enables new use cases such as HBase and Solr where the Write Ahead Log and Transaction Logs are flushed to Ozone constantly.

      A design doc will be added shortly.

      Attachments

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. HDDS-9146 Potential data loss with HSync due to deletedTable entry having the same block as keyTable entry's

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDDS-8198 Hbase on Ozone has very high latency while running IntegrationTestIngestWithTags

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDDS-8199 "Directory is not empty" while running Hbase(configured to run on Ozone) LoadTestTool

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDDS-8200 [Hbase-Ozone] "RegionTooBusyException: Failed to obtain lock" while running IntegrationTestIngestWithTags

          • Major - Major loss of function.
          • Resolved
          Discovered while testing

          Bug - A problem which impairs or prevents the functions of the product. HDDS-10536 [Hbase Ozone] BLOCK_TOKEN_VERIFICATION failed while long running YCSB

          • Major - Major loss of function.
          • Open
          is a parent of

          Bug - A problem which impairs or prevents the functions of the product. HDDS-9164 [Hsync] moves blocks to deleted table on final commit

          • Major - Major loss of function.
          • Resolved
          is duplicated by

          New Feature - A new feature of the product, which has yet to be developed. HDDS-4353 Ozone supports hflush operation

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDDS-633 HBase service fails to start with OzoneFS as defaultFS

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HDDS-11012 [Hbase-Ozone] HMaster down with NO_REPLICA_FOUND causing "CorruptHFileException: Problem reading HFile Trailer"

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDDS-11445 HBase support Phase II

          • Major - Major loss of function.
          • Open
          is required by

          New Feature - A new feature of the product, which has yet to be developed. HBASE-27740 Support Ozone as a WAL backing storage

          • Major - Major loss of function.
          • Open
          relates to

          Sub-task - The sub-task of the issue HDDS-7722 FSO buckets fail to invalidate open file table cache when committing a key

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HDDS-8047 [hsync] Make Putblock performance acceptable

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. HDDS-4353 Ozone supports hflush operation

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDDS-10465 Change ozone.client.bytes.per.checksum default to 16KB

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. RATIS-1886 AppendLog sleep fixed time cause significant drop in write throughput

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. RATIS-2089 Add CommitInfoProto in NotReplicatedException

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Task - A task that needs to be done. HDDS-8825 Use apache/hadoop 3.3.5 docker image

          • Major - Major loss of function.
          • Resolved
          supercedes

          New Feature - A new feature of the product, which has yet to be developed. HDDS-3589 Support running HBase on Ozone.

          • Major - Major loss of function.
          • Resolved
          Testing discovered

          Sub-task - The sub-task of the issue HDDS-8024 When readChunk from a datanode fails, retry other datanodes.

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HDDS-10860 Intermittent failure in TestLeaseRecovery.testFinalizeBlockFailure

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDDS-8969 [Phoenix-Ozone] Failed to identify the fs No FileSystem for scheme "ofs"

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. HDDS-10298 ozone admin om lof, total open files estimation is ambigious

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. HDDS-8292 Inconsistent key name handling for FSO bucket files

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDDS-8267 getOMDBUpdates requests crashed Ozone Manager

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDDS-8198 Hbase on Ozone has very high latency while running IntegrationTestIngestWithTags

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDDS-8537 [Hbase-Ozone] OMKeyRenameRequestWithFSO: Rename key failed

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDDS-8932 [Hbase-Ozone] Hbase on Ozone EC Should gracefully exit

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDDS-10190 [hsync] Lease recovery fails when file has just created without blocks

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDDS-10242 [hsync] Lease recovery failed as "Unable to find the block"

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDDS-10256 Block allocation should retry if SCM is in safe mode

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDDS-8090 When getBlock from a datanode fails, retry other datanodes.

          • Major - Major loss of function.
          • Resolved
          There are no Sub-Tasks for this issue.

          Activity

            People

              weichiu Wei-Chiu Chuang
              weichiu Wei-Chiu Chuang
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 72h
                  72h
                  Remaining:
                  Remaining Estimate - 72h
                  72h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified