Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
1.3.0
Description
I have a bucket with erasure coding enabled
$ ozone sh bucket info impala/test-warehouse { "metadata" : { }, "volumeName" : "impala", "name" : "test-warehouse", "storageType" : "DISK", "versioning" : false, "usedBytes" : 16966237353, "usedNamespace" : 28550, "creationTime" : "2023-04-11T16:59:38.827Z", "modificationTime" : "2023-04-11T17:22:22.328Z", "encryptionKeyName" : "testkey", "quotaInBytes" : -1, "quotaInNamespace" : -1, "bucketLayout" : "FILE_SYSTEM_OPTIMIZED", "owner" : "michael", "replicationConfig" : { "data" : 3, "parity" : 2, "ecChunkSize" : 1048576, "codec" : "RS", "replicationType" : "EC", "requiredNodes" : 5 }, "link" : false }
I’m going to create a key at foo/bar in that bucket. It doesn’t exist yet.
$ ozone sh key info impala/test-warehouse/foo KEY_NOT_FOUND Key:foo not found
I create the key in that bucket
$ ozone sh key cp impala/test-warehouse alltypes_avro_snap/year=2009/month=1/000000_0 foo/bar 23/04/11 11:13:41 WARN erasurecode.ErasureCodeNative: ISA-L support is not available in your platform... using builtin-java codec where applicable 23/04/11 11:13:41 WARN impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-xceiverclientmetrics.properties,hadoop-metrics2.properties 23/04/11 11:13:41 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s). 23/04/11 11:13:41 INFO impl.MetricsSystemImpl: XceiverClientMetrics metrics system started
The key reports EC replication
$ ozone sh key info impala/test-warehouse/foo/bar { "volumeName" : "impala", "bucketName" : "test-warehouse", "name" : "foo/bar", "dataSize" : 6937, "creationTime" : "2023-04-11T18:13:41.081Z", "modificationTime" : "2023-04-11T18:13:41.660Z", "replicationConfig" : { "data" : 3, "parity" : 2, "ecChunkSize" : 1048576, "codec" : "RS", "replicationType" : "EC", "requiredNodes" : 5 }, ... }
However the prefix reports RATIS
$ ozone sh key info impala/test-warehouse/foo { "volumeName" : "impala", "bucketName" : "test-warehouse", "name" : "foo/", "dataSize" : 0, "creationTime" : "2023-04-11T18:13:41.081Z", "modificationTime" : "2023-04-11T18:13:41.081Z", "replicationConfig" : { "replicationFactor" : "ONE", "requiredNodes" : 1, "replicationType" : "RATIS" }, "metadata" : { }, "ozoneKeyLocations" : [ ] }
It used to report EC for the prefix (maybe with LEGACY layout), not RATIS.
Attachments
Issue Links
- causes
-
IMPALA-12132 Directories/tables in Ozone FSO EC bucket incorrectly report no EC
- Open
- is related to
-
IMPALA-12052 Update tests for new Ozone EC chunk size format
- Resolved
- links to