Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-1308

compressed text read can OOM the query in the scanner threads

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • Impala 2.0
    • Impala 2.0
    • None
    • None

    Description

      We don't start up additional scanner threads if there is memory pressure. This is based on the per scanner thread mem estimate, which can be orders of magnitude too low for compressed text.

      SCANNER_THREAD_MEM_USAGE in
      https://github.com/cloudera/Impala/blob/master/be/src/exec/hdfs-scan-node.cc

      Attachments

        Activity

          People

            skye Skye Wanderman-Milne
            nong_impala_60e1 Nong Li
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: