Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-17560

testCurrentLag flaky if enabled for new consumer

    XMLWordPrintableJSON

Details

    Description

      There was an initial attempt to enable testCurrentLag for the new consumer with https://github.com/apache/kafka/pull/16982. Even though it included several changes addressing flakiness, it seems not enough, given that the test is still flaky when running for the new consumer. Fails with:

       
      org.opentest4j.AssertionFailedError: 
      Expected :OptionalLong[40]
      Actual   :OptionalLong.empty
      <Click to see difference>
      at org.apache.kafka.clients.consumer.KafkaConsumerTest.testCurrentLag(KafkaConsumerTest.java:2529)

       

      I wonder if this flakiness is because after receiving a response to a list offset request, maybe a single call to currentLag may not be enough (offsets received in the response not updated yet in the background). Note that after receiving a response to the ListOffsets request, the test moves right away into a single call to currentLag that it expects to be 40, but in the background we only update the subscription state when we get to:

      https://github.com/apache/kafka/blob/aaf3fc05f8e7cb6b8c79f8471f68faa3a994e1e5/clients/src/main/java/org/apache/kafka/clients/consumer/internals/OffsetsRequestManager.java#L565

       

      So we could end up with that single call to currentLag not finding the endOffsets we got in the response (it will find them "eventually" with the new consumer I expect)  

       

      We should review this test to be able to enable it for the new consumer without flakiness.

      Attachments

        Activity

          People

            frankvicky TengYao Chi
            lianetm Lianet Magrans
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: