Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-25924

KDF Integration tests intermittently fails

    XMLWordPrintableJSON

Details

    Description

      Intermittent failures introduced as part of merge (PR#18314: FLINK-24228[connectors/firehose] - Unified Async Sink for Kinesis Firehose):

      1. Failures are intermittent and affecting c. 1 in 7 of builds- on flink-ci.flink and flink-ci.flink-master-mirror .
      2. The issue looks identical to the KinesaliteContainer startup issue (Appendix 1).
      3. I have managed to reproduce the issue locally - if I start some parallel containers and keep them running - and then run KinesisFirehoseSinkITCase  then c. 1 in 6 gives the error.
      4. The errors have a slightly different appearance on flink-ci.flink-master-mirror vs flink-ci.flink which has the same appearance as local. I only hope it is a difference in logging/killing environment variables. (and that there aren’t 2 distinct issues)

      Appendix 1:

      org.testcontainers.containers.ContainerLaunchException: Container startup failed

at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:336)
at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:317)
at org.testcontainers.containers.GenericContainer.starting(GenericContainer.java:1066)
at 
... 11 more
Caused by: org.testcontainers.containers.ContainerLaunchException: Could not create/start container
at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:525)
at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:331)
at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81)
... 12 more
Caused by: org.rnorth.ducttape.TimeoutException: Timeout waiting for result with exception
at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:54)
at

      Attachments

        Issue Links

          is cloned by

          Bug - A problem which impairs or prevents the functions of the product. FLINK-25948 KDS / KDF Sink should call .close() to clean up resources

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. FLINK-25922 KinesisFirehoseSinkITCase hangs on AZP

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          is related to

          New Feature - A new feature of the product, which has yet to be developed. FLINK-24228 [FLIP-171] Firehose implementation of Async Sink

          • Major - Major loss of function.
          • Closed
          is superceded by

          Bug - A problem which impairs or prevents the functions of the product. FLINK-25944 Intermittent Failures on KDF AZP

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. FLINK-25945 Intermittent Failures on KDF AZP 2

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Technical Debt - FLINK-24433 "No space left on device" in Azure e2e tests

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed
          supercedes

          Bug - A problem which impairs or prevents the functions of the product. FLINK-25922 KinesisFirehoseSinkITCase hangs on AZP

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          links to

          Web Link GitHub Pull Request #18606

          Web Link GitHub Pull Request #18633

          Activity

            People

              chalixar Ahmed Hamdy
              CrynetLogistics Zichen Liu
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: