Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-2480

Exception when closing output is ignored

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • TEZ-2003
    • 0.8.2
    • None
    • None

    Description

      Happens a lot in some queries:

      sershe_20150522112029_d0863b33-8d2f-4b4c-b013-9ef70a2bc586:1_Map 1_8_0)] WARN org.apache.tez.runtime.LogicalIOProcessorRuntimeTask: Ignoring exception when closing output Reducer 2(cleanup). Exception class=java.lang.NullPointerException, message=null
      java.lang.NullPointerException
              at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:618)
              at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:81)
              at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:613)
              at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:831)
              at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:608)
              at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1425)
              at org.apache.hadoop.fs.RawLocalFileSystem.open(RawLocalFileSystem.java:198)
              at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:768)
              at org.apache.tez.runtime.library.common.sort.impl.TezSpillRecord.<init>(TezSpillRecord.java:64)
              at org.apache.tez.runtime.library.common.sort.impl.TezSpillRecord.<init>(TezSpillRecord.java:56)
              at org.apache.tez.runtime.library.common.sort.impl.TezSpillRecord.<init>(TezSpillRecord.java:51)
              at org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput.generateEvents(OrderedPartitionedKVOutput.java:209)
              at org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput.close(OrderedPartitionedKVOutput.java:186)
              at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.cleanup(LogicalIOProcessorRuntimeTask.java:849)
              at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:104)
              at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:35)
              at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
      

      Can this be fixed or not logged?

      Attachments

        1. TEZ-2480.1.txt
          2 kB
          Siddharth Seth

        Activity

          People

            sseth Siddharth Seth
            sershe Sergey Shelukhin
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: