Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-1689

Add validations before state transitions in ZkBarrierForVersionUpgrade.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.14.1
    • None
    • None

    Description

      Problem:
      A barrier in zookeeper can be in either DONE or TimedOut state. Existing ZkBarrier implementation doesn't have any state validations before acting upon the barrier events received from zookeeper. This can corrupt barrier states when any of the following occur:

      • When a stream processor joins a barrier marked as timed out by the leader of the group.
      • When the leader of a group expires a barrier marked as Done. This can happen when a leader plays out a stale events from its in-memory buffer.

      This will cause the processors group to go through unnecessary rebalancing phase(worse, corrupt the barrier state).

      Changes:

      • Introduce a barrier state: NEW which will be the beginning state of a barrier(In existing implementation, barrier is created with empty/null value). Barrier will be marked as Done or TimedOut only if the current barrier state is NEW. 
      • Fix the disabled barrier test cases. Add new test cases to validate the changed functionality.
      • Improve logging to aid debugging.

       

      Attachments

        Issue Links

          Activity

            People

              spvenkat Shanthoosh Venkataraman
              spvenkat Shanthoosh Venkataraman
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: