Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-20373

Fix IO threading model

    XMLWordPrintableJSON

Details

    Description

      Currently IO is resubmitted to inboundExecutor for further processing (even there are corner cases then a message handler called in IO thread)

      It makes message processing essentially single threaded and introduces additional lag due to message transition to additional queue.

      addMessageHandler should be extended with a 3-d argument: a pool for submitting a callback for execution, or an executorSelector like in jraft.

      inboundExecutor should be changed to striped, use more than one thread, and serve messages without explicit executor. Delivery guaranties should be preserved: if a message A is send before B, B can't be processed on a receiver before A. A stripe is defined by sender-receiver pair (or can be user defined - TBD)

      outboundExecutor also looks like a contention point - need to be addressed as well

      Attachments

        Issue Links

          blocks

          Improvement - An improvement or enhancement to an existing feature or task. IGNITE-21541 Avoid partition-operations pool when it not lead to starvation

          • Major - Major loss of function.
          • Resolved
          is blocked by

          Improvement - An improvement or enhancement to an existing feature or task. IGNITE-21601 Avoid watch thread hijack due to schema sync in KV/record views

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. IGNITE-21458 Up to 25% performance drop in key-value operations (embedded node)

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. IGNITE-21600 Make partition-operations a plain executor

          • Major - Major loss of function.
          • Resolved
          split to

          Improvement - An improvement or enhancement to an existing feature or task. IGNITE-21627 Make Outbound pool striped

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. IGNITE-21626 Make each component of the Inbound pool striped

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #3300

          Activity

            People

              rpuch Roman Puchkovskiy
              ascherbakov Alexey Scherbakov
              Vladislav Pyatkov Vladislav Pyatkov
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h
                  1h