Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-13161

impalad crash -- impala::DelimitedTextParser<true>::ParseFieldLocations

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • Impala 4.0.0, Impala 4.4.0
    • Impala 4.5.0, Impala 4.4.1
    • Backend
    • None

    Description

      Impala version: 4.0.0

      Problem:
      impalad crash, by operating a text table, which has a 3GB data file that only contains '\x00' char

      Steps:
      python -c 'f=open("impala_0_3gb.data.csv", "wb");tmp="\x00"*1024*1024*3; [f.write(tmp) for i in range(1024)] ;f.close()'
      create table impala_0_3gb (id int)
      hdfs dfs -put impala_0_3gb.data.csv /user/hive/warehouse/impala_0_3gb/
      refresh impala_0_3gb
      select count(1) from impala_0_3gb

      Errors:
      Wrote minidump to 1dcf110f-5a2e-49a2-be4eb7a5-4709ed19.dmp
      #

      1. A fatal error has been detected by the Java Runtime Environment:
        #
      2. SIGSEGV (0xb) at pc=0x000000000181861c, pid=956182, tid=0x00007fc6b340e700
        #
      3. JRE version: OpenJDK Runtime Environment (8.0) (build 1.8.0)
      4. Java VM: OpenJDK 64-Bit Server VM
      5. Problematic frame:
      6. C [impalad+0x141861c] impala::DelimitedTextParser<true>::ParseFieldLocations(int, long, char*, char, impala::FieldLocation, int*, int*, char**)+0x7cc
        #
      7. Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
        #
      8. An error report file with more information is saved as:
      9. /tmp/hs_err_pid956182.log
        #
        #

      C [impalad+0x141861c] impala::DelimitedTextParser<true>::ParseFieldLocations(int, long, char*, char, impala::FieldLocation, int*, int*, char**)+0x7cc
      C [impalad+0x136fe11] impala::HdfsTextScanner::ProcessRange(impala::RowBatch*, int*)+0x1a1
      C [impalad+0x137100e] impala::HdfsTextScanner::FinishScanRange(impala::RowBatch*)+0x3be
      C [impalad+0x13721ac] impala::HdfsTextScanner::GetNextInternal(impala::RowBatch*)+0x12c
      C [impalad+0x131cdfc] impala::HdfsScanner::ProcessSplit()+0x19c
      C [impalad+0x1443e17] impala::HdfsScanNode::ProcessSplit(std::vector<impala::FilterContext, std::allocator<impala::FilterContext> > const&, impala::MemPool*, impala::io::ScanRange*, long*)+0x7e7
      C [impalad+0x1447001] impala::HdfsScanNode::ScannerThread(bool, long)+0x541

      Attachments

        Issue Links

          Activity

            People

              stigahuang Quanlong Huang
              nyq nyq
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: