Uploaded image for project: 'ActiveMQ Artemis'
  1. ActiveMQ Artemis
  2. ARTEMIS-4314

Queue Federation, support consumerWindowSize zero and federate in batches only when the local queue is has excess capacity

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.29.0
    • 2.30.0
    • Federation
    • None

    Description

      Dual queue federation, where clusters federate in both direction can suffer from message flip flopping once the priority adjustment kicks in.

      If there is a large backlog, the lower priority federation consumer is in play once all of the local consumer credit is exhausted and the backlog can drain to the other cluster.

      If demand is low there, the process can repeat. limiting the rate of the federation consumer can help but it is not ideal b/c when there is no local demand, we want to have a high rate of migration.

       

      A possible solution is to have the federation consumer manage its own credit and only flow messages when the local queue has capacity. Then flow a batch of messages, and await again that the local queue has capacity. In this way, there is no thundering herd effect, but there is also fast migration of messages once there is demand.

      the consumerWindowSize=0 is already in play for consumer.receive calls and there is already a defaultConsumerWindowSize for an address. These can be combined to realise batchFederationOnCapacity semantics.

      Attachments

        Issue Links

          Activity

            People

              gtully Gary Tully
              gtully Gary Tully
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 40m
                  2h 40m