Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-4852

acceptance (secure) fails frequently at `ozone freon ockg -n1 -t1 -p container` in container.robot

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • None
    • None
    • test
    • None

    Description

      It's quite annoying this fails often in CIs for a while. Very flaky.

      https://github.com/apache/ozone/blob/bc3e3e5f62a871c50f6261534424c67a53713d22/hadoop-ozone/dist/src/main/smoketest/admincli/container.robot#L29

      2021-02-20 00:24:19,978 [Thread-5] INFO freon.ProgressBar: Progress: 0.00 % (0 out of 1)
      2021-02-20 00:24:20,294 [main] INFO rpc.RpcClient: Creating Volume: vol1, with testuser/scm@EXAMPLE.COM as owner and space quota set to -1 bytes, counts quota set to -1
      2021-02-20 00:24:20,338 [main] INFO rpc.RpcClient: Creating Bucket: vol1/bucket1, with Versioning false and Storage Type set to DISK and Encryption set to false 
      2021-02-20 00:24:20,416 [pool-1-thread-1] ERROR freon.BaseFreonGenerator: Error on executing task 0
      INTERNAL_ERROR org.apache.hadoop.ozone.om.exceptions.OMException: Allocated 0 blocks. Requested 1 blocks
      	at org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.handleError(OzoneManagerProtocolClientSideTranslatorPB.java:604)
      	at org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.openKey(OzoneManagerProtocolClientSideTranslatorPB.java:595)
      	at org.apache.hadoop.ozone.client.rpc.RpcClient.createKey(RpcClient.java:716)
      	at org.apache.hadoop.ozone.client.OzoneBucket.createKey(OzoneBucket.java:491)
      	at org.apache.hadoop.ozone.freon.OzoneClientKeyGenerator.lambda$createKey$0(OzoneClientKeyGenerator.java:114)
      	at com.codahale.metrics.Timer.time(Timer.java:101)
      	at org.apache.hadoop.ozone.freon.OzoneClientKeyGenerator.createKey(OzoneClientKeyGenerator.java:113)
      	at org.apache.hadoop.ozone.freon.BaseFreonGenerator.tryNextTask(BaseFreonGenerator.java:188)
      	at org.apache.hadoop.ozone.freon.BaseFreonGenerator.taskLoop(BaseFreonGenerator.java:168)
      	at org.apache.hadoop.ozone.freon.BaseFreonGenerator.lambda$startTaskRunners$0(BaseFreonGenerator.java:151)
      	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
      	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
      	at java.base/java.lang.Thread.run(Thread.java:834)
      2021-02-20 00:24:20,979 [Thread-5] INFO freon.ProgressBar: Progress: 0.00 % (0 out of 1)
      2021-02-20 00:24:21,979 [Thread-5] INFO freon.ProgressBar: Progress: 100.00 % (1 out of 1)
      One ore more freon test is failed.
      2021-02-20 00:24:22,002 [Thread-4] INFO metrics: type=TIMER, name=key-create, count=1, min=26.521721, max=26.521721, mean=26.521721, stddev=0.0, median=26.521721, p75=26.521721, p95=26.521721, p98=26.521721, p99=26.521721, p999=26.521721, mean_rate=0.6144826916810641, m1=0.0, m5=0.0, m15=0.0, rate_unit=events/second, duration_unit=milliseconds
      2021-02-20 00:24:22,006 [Thread-4] INFO freon.BaseFreonGenerator: Total execution time (sec): 3
      2021-02-20 00:24:22,006 [Thread-4] INFO freon.BaseFreonGenerator: Failures: 1
      2021-02-20 00:24:22,006 [Thread-4] INFO freon.BaseFreonGenerator: Successful executions: 0	
      

      The OMException: Allocated 0 blocks. Requested 1 blocks seems to be the culprit.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              smeng Siyao Meng
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: