Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-7886

Coordinator should not wait for read timeouts when replicas hit Exceptions

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Low
    • Resolution: Fixed
    • 2.2.0 beta 1
    • None

    • Tested with Cassandra 2.0.8

    • Clients

    Description

      Issue
      When you have TombstoneOverwhelmingExceptions occuring in queries, this will cause the query to be simply dropped on every data-node, but no response is sent back to the coordinator. Instead the coordinator waits for the specified read_request_timeout_in_ms.

      On the application side this can cause memory issues, since the application is waiting for the timeout interval for every request.Therefore, if our application runs into TombstoneOverwhelmingExceptions, then (sooner or later) our entire application cluster goes down

      Proposed solution
      I think the data nodes should send a error message to the coordinator when they run into a TombstoneOverwhelmingException. Then the coordinator does not have to wait for the timeout-interval.

      Attachments

        1. 7886-final.txt
          53 kB
          Tom Hobbs
        2. 7886_v6_trunk.txt
          55 kB
          Christian Spriegel
        3. 7886_v5_trunk.txt
          48 kB
          Christian Spriegel
        4. 7886_v4_trunk.txt
          44 kB
          Christian Spriegel
        5. 7886_v3_trunk.txt
          36 kB
          Christian Spriegel
        6. 7886_v2_trunk.txt
          35 kB
          Christian Spriegel
        7. 7886_v1.txt
          26 kB
          Christian Spriegel

        Issue Links

          is related to

          New Feature - A new feature of the product, which has yet to be developed. CASSANDRA-8592 Add WriteFailureException

          • Normal -
          • Resolved
          relates to

          New Feature - A new feature of the product, which has yet to be developed. CASSANDRA-6117 Avoid death-by-tombstone by default

          • Normal -
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. CASSANDRA-8561 Tombstone log warning does not log partition key

          • Normal -
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. CASSANDRA-6747 MessagingService should handle failures on remote nodes.

          • Low -
          • Resolved
          requires

          Task - A task that needs to be done. CASSANDRA-8043 Native Protocol V4

          • Normal -
          • Resolved

          Activity

            People

              christianmovi Christian Spriegel
              christianmovi Christian Spriegel
              Christian Spriegel
              Tom Hobbs
              Alan Boudreault Alan Boudreault
              Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: