Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-8543

New key prefix in EC bucket with FSO reports RATIS replication

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.3.0
    • 1.4.0
    • EC

    Description

      I have a bucket with erasure coding enabled

      $ ozone sh bucket info impala/test-warehouse                                                                                                                   {
        "metadata" : { },
        "volumeName" : "impala",
        "name" : "test-warehouse",
        "storageType" : "DISK",
        "versioning" : false,
        "usedBytes" : 16966237353,
        "usedNamespace" : 28550,
        "creationTime" : "2023-04-11T16:59:38.827Z",
        "modificationTime" : "2023-04-11T17:22:22.328Z",
        "encryptionKeyName" : "testkey",
        "quotaInBytes" : -1,
        "quotaInNamespace" : -1,
        "bucketLayout" : "FILE_SYSTEM_OPTIMIZED",
        "owner" : "michael",
        "replicationConfig" : {
          "data" : 3,
          "parity" : 2,
          "ecChunkSize" : 1048576,
          "codec" : "RS",
          "replicationType" : "EC",
          "requiredNodes" : 5
        },
        "link" : false
      }
      

      I’m going to create a key at foo/bar in that bucket. It doesn’t exist yet.

      $ ozone sh key info impala/test-warehouse/foo
      KEY_NOT_FOUND Key:foo not found
      

      I create the key in that bucket

      $ ozone sh key cp impala/test-warehouse alltypes_avro_snap/year=2009/month=1/000000_0 foo/bar                                                                  23/04/11 11:13:41 WARN erasurecode.ErasureCodeNative: ISA-L support is not available in your platform... using builtin-java codec where applicable
      23/04/11 11:13:41 WARN impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-xceiverclientmetrics.properties,hadoop-metrics2.properties
      23/04/11 11:13:41 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
      23/04/11 11:13:41 INFO impl.MetricsSystemImpl: XceiverClientMetrics metrics system started
      

      The key reports EC replication

      $ ozone sh key info impala/test-warehouse/foo/bar
      {
        "volumeName" : "impala",
        "bucketName" : "test-warehouse",
        "name" : "foo/bar",
        "dataSize" : 6937,
        "creationTime" : "2023-04-11T18:13:41.081Z",
        "modificationTime" : "2023-04-11T18:13:41.660Z",
        "replicationConfig" : {
          "data" : 3,
          "parity" : 2,
          "ecChunkSize" : 1048576,
          "codec" : "RS",
          "replicationType" : "EC",
          "requiredNodes" : 5
        },
        ...
      }
      

      However the prefix reports RATIS

      $ ozone sh key info impala/test-warehouse/foo
      {
        "volumeName" : "impala",
        "bucketName" : "test-warehouse",
        "name" : "foo/",
        "dataSize" : 0,
        "creationTime" : "2023-04-11T18:13:41.081Z",
        "modificationTime" : "2023-04-11T18:13:41.081Z",
        "replicationConfig" : {
          "replicationFactor" : "ONE",
          "requiredNodes" : 1,
          "replicationType" : "RATIS"
        },
        "metadata" : { },
        "ozoneKeyLocations" : [ ]
      }
      

      It used to report EC for the prefix (maybe with LEGACY layout), not RATIS.

      Attachments

        Issue Links

          Activity

            People

              adoroszlai Attila Doroszlai
              MikaelSmith Michael Smith
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: