Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-8559

OOM caused by large tombstone warning.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Normal
    • Resolution: Fixed
    • 2.0.15, 2.1.5
    • None
    • 2.0.11 / 2.1

    • Normal

    Description

      When running with high amount of tombstones the error message generation from CASSANDRA-6117 can lead to out of memory situation with the default setting.

      Attached a heapdump viewed in visualvm showing how this construct created two 777mb strings to print the error message for a read query and then crashed OOM.

              if (respectTombstoneThresholds() && columnCounter.ignored() > DatabaseDescriptor.getTombstoneWarnThreshold())
              {
                  StringBuilder sb = new StringBuilder();
                  CellNameType type = container.metadata().comparator;
                  for (ColumnSlice sl : slices)
                  {
                      assert sl != null;
      
                      sb.append('[');
                      sb.append(type.getString(sl.start));
                      sb.append('-');
                      sb.append(type.getString(sl.finish));
                      sb.append(']');
                  }
      
                  logger.warn("Read {} live and {} tombstoned cells in {}.{} (see tombstone_warn_threshold). {} columns was requested, slices={}, delInfo={}",
                              columnCounter.live(), columnCounter.ignored(), container.metadata().ksName, container.metadata().cfName, count, sb, container.deletionInfo());
              }
      

      Attachments

        1. Selection_048.png
          103 kB
          Dominic Letz
        2. stacktrace.log
          3 kB
          Dominic Letz
        3. cassandra-2.0.11-8559.txt
          5 kB
          Dominic Letz
        4. 8559.txt
          1 kB
          Aleksey Yeschenko

        Issue Links

          Activity

            People

              aleksey Aleksey Yeschenko
              dominicletz Dominic Letz
              Aleksey Yeschenko
              Sam Tunnicliffe
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: