Step 1 of 4: Choose Issues

Cancel

T Patch Info Key Summary Assignee Reporter P Status Resolution Created Updated Due Development
Sub-task HDDS-11407

HDDS-7593 Use OMLayoutFeature.HBASE_SUPPORT for HSYNC

Hemant Kumar Hemant Kumar Major Resolved Fixed  
Sub-task HDDS-11390

HDDS-7593 [hsync] Remove hsync and hflush capability check in ContentGenerator

Hemant Kumar Hemant Kumar Major Resolved Fixed  
Sub-task HDDS-11369

HDDS-7593 [hsync] Remove KeyOutputStreamSemaphore logs

Chung En Lee Wei-Chiu Chuang Trivial Resolved Fixed  
Sub-task HDDS-11342

HDDS-7593 [hsync] Add a config as HBase-related features master switch

Siyao Meng Siyao Meng Major Resolved Fixed  
Sub-task HDDS-11328

HDDS-7593 ContainerStateMachine should not crash because of CHUNK_FILE_INCONSISTENCY

Duong Duong Major Resolved Implemented  
Sub-task HDDS-11327

HDDS-7593 [hsync] Revert config default ozone.fs.hsync.enabled to false

Siyao Meng Siyao Meng Major Resolved Done  
Sub-task HDDS-11322

HDDS-7593 [hsync] Block ECKeyOutputStream from calling hsync and hflush

Siyao Meng Siyao Meng Major Resolved Fixed  
Sub-task HDDS-11312

HDDS-7593 [hsync] Add upgrade tests

Hemant Kumar Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-11292

HDDS-7593 [hsync] Move HBASE_SUPPORT layout upgrade test into its own test

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-11280

HDDS-7593 TestBlockOutputStream.testWriteMoreThanFlushSize is flaky

Ashish Kumar Ashish Kumar Major Resolved Fixed  
Sub-task HDDS-11275

HDDS-7593 Merge recent commits from master to HDDS-7593

Ashish Kumar Ashish Kumar Major Resolved Won't Do  
Sub-task HDDS-11270

HDDS-7593 [hsync] Add DN layout version (HBASE_SUPPORT/version 8) upgrade test

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-11260

HDDS-7593 [hsync] Add Ozone Manager protocol version

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-11259

HDDS-7593 [hsync] DataNode should verify HBASE_SUPPORT layout version for every PutBlock

Wei-Chiu Chuang Wei-Chiu Chuang Blocker Resolved Fixed  
Sub-task HDDS-11258

HDDS-7593 [hsync] Add new OM layout version

Wei-Chiu Chuang Wei-Chiu Chuang Blocker Resolved Fixed  
Sub-task HDDS-11239

HDDS-7593 Flakiness in KeyOutputStream exception handling

Duong Duong Blocker Resolved Fixed  
Sub-task HDDS-11225

HDDS-7593 Increase ipc.server.read.threadpool.size

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-11224

HDDS-7593 Increase hdds.datanode.handler.count

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-11208

HDDS-7593 Change RatisBlockOutputStream to use HDDS-11174

Tsz-wo Sze Tsz-wo Sze Major Resolved Fixed  
Sub-task HDDS-11193

HDDS-7593 KeyOutputStream flakiness when running write and hsync concurrently

Duong Duong Major Resolved Fixed  
Sub-task HDDS-11185

HDDS-7593 Fix ContainerOpsLatencies metrics

Duong Duong Major Resolved Fixed  
Sub-task HDDS-11184

HDDS-7593 [hsync] Add a client config to limit write concurrency on the same key

Siyao Meng Siyao Meng Major Resolved Fixed  
Sub-task HDDS-11174

HDDS-7593 [hsync] Change XceiverClientRatis.watchForCommit to async

Tsz-wo Sze Tsz-wo Sze Major Resolved Fixed  
Sub-task HDDS-11170

HDDS-7593 [hsync] 9th merge from master

Siyao Meng Siyao Meng Major Resolved Fixed  
Sub-task HDDS-11168

HDDS-7593 [hsync] Instantiates audit parameter lazily in DataNode dispatch handler

Wei-Chiu Chuang Wei-Chiu Chuang Minor Resolved Fixed  
Sub-task HDDS-11135

HDDS-7593 [hsync] Replace expensive VolumeUsage.getMinVolumeFreeSpace()

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-11073

HDDS-7593 [hsync] Improve BlockOutputStream's BufferPool to support variable buffer allocation from concurrent hsync

Duong Duong Blocker Resolved Implemented  
Sub-task HDDS-11071

HDDS-7593 [hsync] 8th merge from master

Siyao Meng Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-11016

HDDS-7593 [hsync] 7th merge from master

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-11014

HDDS-7593 [hsync] Block finalization should also merge last chunk to blockDataTable

Wei-Chiu Chuang Wei-Chiu Chuang Blocker Resolved Fixed  
Sub-task HDDS-10955

HDDS-7593 [hsync] Adopt Ratis 3.1.0 when it's released

Chung En Lee Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-10941

HDDS-7593 Add a few interesting ContainerStateMachine metrics in CSMMetrics

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-10925

HDDS-7593 [hsync] Parameterize TestBlockOutputStream on ozone.client.stream.putblock.piggybacking

Wei-Chiu Chuang Wei-Chiu Chuang Minor Resolved Fixed  
Sub-task HDDS-10922

HDDS-7593 Merge master branch 611066a to HDDS-7593 dev branch

Siyao Meng Siyao Meng Major Resolved Duplicate  
Sub-task HDDS-10906

HDDS-7593 [hsync] 6th merge from master

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-10904

HDDS-7593 [hsync] Enable PutBlock piggybacking and incremental chunk list by default

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-10890

HDDS-7593 [hsync] Increase default value for hdds.container.ratis.log.appender.queue.num-elements

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Done  
Sub-task HDDS-10860

HDDS-7593 Intermittent failure in TestLeaseRecovery.testFinalizeBlockFailure

Ashish Kumar Ashish Kumar Major Resolved Fixed  
Sub-task HDDS-10838

HDDS-7593 Merge recent commits from master to HDDS-7593

Ashish Kumar Ashish Kumar Major Resolved Done  
Sub-task HDDS-10836

HDDS-7593 Wrong count is displaying in listOpenFile CLI

Unassigned Ashish Kumar Major Resolved Duplicate  
Sub-task HDDS-10835

HDDS-7593 Show overwritten hsync keys in ListOpenFile CLI

Sammi Chen Sammi Chen Major Resolved Fixed  
Sub-task HDDS-10820

HDDS-7593 Freon tool DN-Echo to support GRPC and Ratis read/write mode

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-10797

HDDS-7593 Remove unused UserGroupInformation object in DataNode token verifier

Wei-Chiu Chuang Wei-Chiu Chuang Minor Resolved Implemented  
Sub-task HDDS-10774

HDDS-7593 [hsync] Show deleted hsync keys in ListOpenFile CLI

Ashish Kumar Ashish Kumar Minor Resolved Fixed  
Sub-task HDDS-10755

HDDS-7593 [hsync] Investigate why DataNode Echo throughput is so low

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Done  
Sub-task HDDS-10754

HDDS-7593 [hsync] lease recovery contract test class not substantiated

Chung En Lee Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-10736

HDDS-7593 [hsync] OMKeyCommitRequest should reject if client id doesn't match

Chung En Lee Wei-Chiu Chuang Major Resolved Duplicate  
Sub-task HDDS-10733

HDDS-7593 [hsync] Merge HDDS-7593 feature branch into master

Ashish Kumar Wei-Chiu Chuang Blocker Resolved Fixed 31/May/24
Sub-task HDDS-10678

HDDS-7593 [hsync] Merge recent commits from master #4

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-10651

HDDS-7593 HBase WAL splitting fails due to lease recovery

Sammi Chen Wei-Chiu Chuang Major Resolved Not A Bug  
Sub-task HDDS-10626

HDDS-7593 [LeaseRecovery] OM shuts down with "SecretKey client must have been initialized already"

Sammi Chen Pratyush Bhatt Blocker Resolved Fixed  
Sub-task HDDS-10592

HDDS-7593 [hsync] Output stream lastChunkBuffer should use direct buffer

Ashish Kumar Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-10591

HDDS-7593 [hsync] improve block token refresh message

Wei-Chiu Chuang Wei-Chiu Chuang Minor Resolved Fixed  
Sub-task HDDS-10573

HDDS-7593 [hsync] Add a freon tool to benchmark hsync/write concurrency

Duong Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-10565

HDDS-7593 Merge master 97038ef to feature branch HDDS-7593

Siyao Meng Siyao Meng Major Resolved Fixed  
Sub-task HDDS-10511

HDDS-7593 OzoneFSInputStream to support ByteBufferPositionedReadable

Ashish Kumar Ashish Kumar Major Resolved Fixed  
Sub-task HDDS-10497

HDDS-7593 [hsync] Refresh block token immediately if block token expires

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-10471

HDDS-7593 [hsync] MockDatanodeStorage.writeChunk should make a copy of byte string

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-10442

HDDS-7593 [hsync] Add a Freon tool to measure client to DataNode round-trip latency

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-10431

HDDS-7593 Merge recent commits from master (7c8160fe) to HDDS-7593

Siyao Meng Siyao Meng Major Resolved Fixed  
Sub-task HDDS-10361

HDDS-7593 [hsync] Output stream should support direct byte buffer

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-10321

HDDS-7593 Merge recent commits from master to HDDS-7593

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-10268

HDDS-7593 [hsync] Add OpenTracing traces to client side read path

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-10252

HDDS-7593 [hsync] Revisit configuration keys for incremental chunk list after HDDS-9884

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-10147

HDDS-7593 Migrate tests to JUnit5

Ashish Kumar Attila Doroszlai Major Resolved Implemented  
Sub-task HDDS-10141

HDDS-7593 [hsync]Support hard limit and auto recovery for hsync file

Ashish Kumar Ashish Kumar Major Resolved Fixed  
Sub-task HDDS-10140

HDDS-7593 [hsync] Rebase HDDS-7593 branch onto master

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Done  
Sub-task HDDS-10104

HDDS-7593 Introduce soft limit support for lease recovery

Ashish Kumar Ashish Kumar Major Resolved Fixed  
Sub-task HDDS-10093

HDDS-7593 Make recoverLease call idempotent

Sammi Chen Sammi Chen Major Resolved Fixed  
Sub-task HDDS-10077

HDDS-7593 Add hsync metadata to hsync'ed keys in OpenKeyTable as well

Siyao Meng Siyao Meng Major Resolved Fixed  
Sub-task HDDS-10044

HDDS-7593 [hsync] File recovery support in Client

Sammi Chen Sammi Chen Major Resolved Fixed  
Sub-task HDDS-10003

HDDS-7593 [hsync] Cache serialized block token in output stream to reduce heap consumption

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Implemented  
Sub-task HDDS-9987

HDDS-7593 [hsync] Client side metrics

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-9930

HDDS-7593 Deleted file reappears after HSync

Siyao Meng Pratyush Bhatt Critical Resolved Fixed  
Sub-task HDDS-9918

HDDS-7593 [hsync] Remove block token from Ratis log once verified

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-9915

HDDS-7593 hsync: Interface to retrieve block info and finalize block in DN through ratis

Ashish Kumar Ashish Kumar Major Resolved Fixed  
Sub-task HDDS-9884

HDDS-7593 DataNode doesn't set proper DatanodeVersion when registering with SCM

Siyao Meng Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-9844

HDDS-7593 [hsync] De-synchronize hsync API

Duong Wei-Chiu Chuang Blocker Resolved Fixed  
Sub-task HDDS-9842

HDDS-7593 [hsync] Checking disk capacity at every write request is expensive for HBase

Attila Doroszlai Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-9753

HDDS-7593 [hsync] Make Putblock performance acceptable - Tool cleanup, guardrails

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Duplicate  
Sub-task HDDS-9752

HDDS-7593 [hsync] Make Putblock performance acceptable - Client side

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-9751

HDDS-7593 [hsync] Make Putblock performance acceptable - DataNode side

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-9750

HDDS-7593 [hsync] Make Putblock performance acceptable - Skeleton code

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-9734

HDDS-7593 ChunkInputStream should use new token after pipeline refresh

Attila Doroszlai Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-9638

HDDS-7593 [hsync] File recovery support in OM

Sammi Chen Sammi Chen Major Resolved Fixed  
Sub-task HDDS-9483

HDDS-7593 ozone freon --server is broken by HDDS-6176

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-9387

HDDS-7593 [hsync] Reduce updating block length times at OM during hsync

Sammi Chen Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-9386

HDDS-7593 [hsync] Redesign the lease recovery protocol so block length is updated correctly at OM

Unassigned Wei-Chiu Chuang Major Resolved Done  
Sub-task HDDS-9385

HDDS-7593 [hsync] Flush to only wait for majority of DataNodes

Unassigned Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-9365

HDDS-7593 [hsync] DataNode to deserialize Ratis transaction only once

Tsz-wo Sze Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-9358

HDDS-7593 recoverLease should close underlying streams

Ashish Kumar Attila Doroszlai Major Resolved Won't Fix  
Sub-task HDDS-9176

HDDS-7593 two client parallel perform commit with Hsync can cause dataloss

Siyao Meng Sumit Agrawal Major Resolved Done  
Sub-task HDDS-9148

HDDS-7593 OM to reject hsync if ozone.fs.hsync.enabled is false

Wei-Chiu Chuang Siyao Meng Critical Resolved Fixed  
Sub-task HDDS-9131

HDDS-7593 [hsync] A freon tool to focus on hsync/hflush performance

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Implemented  
Sub-task HDDS-9130

HDDS-7593 [hsync] Combine WriteData and PutBlock requests into one

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-8830

HDDS-7593 Add admin CLI to list open files

Siyao Meng Pratyush Bhatt Major Resolved Fixed  
Sub-task HDDS-8797

HDDS-7593 [hsync] HBase RegionServer input stream not shut down properly

Unassigned Wei-Chiu Chuang Major Resolved Duplicate  
Sub-task HDDS-8769

HDDS-7593 [hsync] disk usage thread aborts if ratis log rolls very quickly

Unassigned Wei-Chiu Chuang Major Resolved Cannot Reproduce  
Sub-task HDDS-8602

HDDS-7593 [File Lease] OM adds FileLeaseManager

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Later  
Sub-task HDDS-8601

HDDS-7593 [File Lease] OM adds request message handler

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Later  
Sub-task HDDS-8600

HDDS-7593 [File Lease] Client side lease renewer thread and request message

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Later  
Sub-task HDDS-8545

HDDS-7593 [hsync] reject renaming open file

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Implemented  
Sub-task HDDS-8506

HDDS-7593 LeaseRecovery failing with NullPointer exception

Wei-Chiu Chuang Pratyush Bhatt Major Resolved Fixed  
Sub-task HDDS-8439

HDDS-7593 [hsync] write after lease recovery does not fail

Sammi Chen Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-8436

HDDS-7593 Support setSafeMode(), isFileClosed() FileSystem API

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-8352

HDDS-7593 OM crash with NPE in OMKeyCommitRequest due to missing user info

Sumit Agrawal Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-8302

HDDS-7593 Add a flag to disable hsync by default

Wei-Chiu Chuang Wei-Chiu Chuang Blocker Resolved Implemented  
Sub-task HDDS-8295

HDDS-7593 [hsync] Support renaming open files

Unassigned Wei-Chiu Chuang Major Resolved Later  
Sub-task HDDS-8278

HDDS-7593 O3fs/ofs to support setTimes() API

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-8259

HDDS-7593 [hsync] OMKeyRequest: Detect allocated but uncommitted blocks

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-8243

HDDS-7593 [hsync] Add a CLI to recover lease

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-8193

HDDS-7593 Ozone file systems to support Hadoop's PathCapabilities interface

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-8146

HDDS-7593 TestHSync is no longer flaky

Tsz-wo Sze Tsz-wo Sze Major Resolved Fixed  
Sub-task HDDS-8047

HDDS-7593 [hsync] Make Putblock performance acceptable

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Implemented  
Sub-task HDDS-8029

HDDS-7593 [hsync] Outputstream in encrypted buckets do not return the correct stream capabilities

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-7965

HDDS-7593 Quota needs to be updated correctly for Hsync

Sumit Agrawal Sumit Agrawal Major Resolved Fixed  
Sub-task HDDS-7947

HDDS-7593 Disallow overwriting a hsync'ed key

Wei-Chiu Chuang Tsz-wo Sze Blocker Resolved Fixed  
Sub-task HDDS-7930

HDDS-7593 input stream does not refresh expired block token

Wei-Chiu Chuang Wei-Chiu Chuang Blocker Resolved Fixed  
Sub-task HDDS-7926

HDDS-7593 [hsync] Recon throws ClassCastException

Arafat Khan Wei-Chiu Chuang Blocker Resolved Fixed  
Sub-task HDDS-7907

HDDS-7593 [hsync] KeyOutputStream is not thread safe

Wei-Chiu Chuang Wei-Chiu Chuang Blocker Resolved Fixed  
Sub-task HDDS-7855

HDDS-7593 Add hsync metrics in OM

Tsz-wo Sze Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-7826

HDDS-7593 Add support for application to probe output stream capability

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Implemented  
Sub-task HDDS-7782

HDDS-7593 OM lease recovery for hsync'ed files

Tsz-wo Sze Tsz-wo Sze Major Resolved Fixed  
Sub-task HDDS-7781

HDDS-7593 Change Get Key Info to return HSync info

Wei-Chiu Chuang Tsz-wo Sze Major Resolved Fixed  
Sub-task HDDS-7769

HDDS-7593 Implement client initiated lease recovery

Wei-Chiu Chuang Wei-Chiu Chuang Blocker Resolved Fixed  
Sub-task HDDS-7764

HDDS-7593 Potential discrepancy of key creation time may cause premature open key clean up

Tsz-wo Sze Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-7724

HDDS-7593 hsync: Change KeyOutputStream to update length in OM

Unassigned Tsz-wo Sze Major Resolved Duplicate  
Sub-task HDDS-7688

HDDS-7593 Ozone Manager change to support HSync

Wei-Chiu Chuang Wei-Chiu Chuang Major Resolved Fixed  
Sub-task HDDS-7638

HDDS-7593 Ozone client change to support HSync

Tsz-wo Sze Tsz-wo Sze Major Resolved Fixed  

Cancel