Uploaded image for project: 'Camel'
  1. Camel
  2. CAMEL-18717

camel-kafka: investigate offset not increasing

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.18.3, 3.19.0
    • 3.18.4, 3.20.0
    • camel-kafka
    • None
    • Unknown

    Description

      Quoting the report from an user on the users mailing list:

       

      Hi All,

      I'm upgrading Camel from 3.16 to 3.17 and found a problem that OffsetRepository didn't get update after manual commit.
      It is a Spring boot project and MemoryStateRepository as follows.

              @Bean
              public MemoryStateRepository offsetRepo() {
                      MemoryStateRepository stateRepository = new MemoryStateRepository();
                      stateRepository.setState(topic + "/0", "");
                      return stateRepository;
              }

      Here is the method I use when commit:

              private void commitOffsetToKafka(Exchange exchange) {
                      KafkaManualCommit manual = exchange.getIn().getHeader(KafkaConstants.MANUAL_COMMIT, KafkaManualCommit.class);
                      manual.commit();
              }

      Here is my Kafka endpoint

      kafka:integration_test _topic?brokers=127.0.0.1:41478&autoCommitEnable=false&allowManualCommit=true&seekTo=beginning&maxPollRecords=2&groupId=ANE&autoOffsetReset=earliest&offsetRepository=#offsetRepo&kafkaManualCommitFactory=#class:org.apache.camel.component.kafka.consumer.DefaultKafkaManualCommitFactory

      It is working fine in 3.16. When DefaultkafkaManualSyncCommit.commit() is invoked. The offsetRepository.setState() get invoked so the offset increased.
      However, in 3.17, DefaultkafkaManualSyncCommit.commit() doesn't have a reference to offsetRepository so the offset in offsetRepository won't get increased.

      Is this a bug or there is a different way to get the offset after committing. Could you please suggest what I should do to test if the offset get increased after manual commit?

       

      This happened even after upgrading to Camel 3.18.3 as requested on the thread.

       

      This needs to be investigated further.

      Attachments

        Issue Links

          Activity

            People

              orpiske Otavio Rodolfo Piske
              orpiske Otavio Rodolfo Piske
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: