Uploaded image for project: 'ActiveMQ Artemis'
  1. ActiveMQ Artemis
  2. ARTEMIS-4065

Improving Page Counting by using real records, and not use the journal extensive for every message sent

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.28.0
    • Broker
    • None

    Description

      Paging will now keep a single record in the journal for every destination paged. We used to have a record, and then one update for every message sent or acked. We would also flush the counter with a new record once you reached 1000 pending updates.

      This was creating a great pressure in the system forcing a lot of journal compacting to defragment the records in the journal.

      After this change , once the system is paging, the system will update the record every minute (hard coded timeout), and once the server is shutdown. It will be a single record per Queue, which is a much lighter approach than the previous generating thousands of records.

      Also, Every time the broker starts, we will swipe the paged messages in parallel. While the page-counters are being rebuild we will serve the counter based on the previously recorded journal (before the restart).

      One reason we keep the record in the journal is also for compatibility. So if you downgrade to a previous version of the broker the data would still be valid on the previous version. (Only recommendation is a clean shutdown before a downgrade as we will update the page counter before shutdown).

      The rebuild counter will shortly disable removing paged files while messages are being counted. We will save a snapshot position up to a positionN, and read all messages until that positionN.

      we will then merge the read counters with a snapshot between positionN and newly sent messages.

      This will allow using less records in the journal to keep the message counters boosting paging performance.

      Attachments

        Issue Links

          Activity

            People

              clebertsuconic Clebert Suconic
              clebertsuconic Clebert Suconic
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 4h 20m
                  4h 20m