Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-682

TezGroupedSplits fails with empty (zero length) file

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.2.0, 0.3.0
    • 0.3.0
    • None
    • None

    Description

      Running hive on a directory with some 0 length files in it:

      2013-12-17 23:01:35,868 ERROR [AsyncDispatcher event handler] org.apache.tez.dag.app.dag.impl.VertexImpl: Vertex Input: bucket1_1 initializer failed
      java.lang.NullPointerException
      at org.apache.hadoop.io.Text.encode(Text.java:443)
      at org.apache.hadoop.io.Text.encode(Text.java:424)
      at org.apache.hadoop.io.Text.writeString(Text.java:476)
      at org.apache.hadoop.mapred.split.TezGroupedSplit.write(TezGroupedSplit.java:87)
      at org.apache.tez.mapreduce.hadoop.MRHelpers.createSplitProto(MRHelpers.java:446)
      at org.apache.tez.mapreduce.common.MRInputAMSplitGenerator.initialize(MRInputAMSplitGenerator.java:129)
      at org.apache.tez.dag.app.dag.RootInputInitializerRunner$InputInitializerCallable.call(RootInputInitializerRunner.java:121)
      at org.apache.tez.dag.app.dag.RootInputInitializerRunner$InputInitializerCallable.call(RootInputInitializerRunner.java:97)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
      at java.util.concurrent.FutureTask.run(FutureTask.java:138)
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
      at java.lang.Thread.run(Thread.java:695)

      Deleting the empty files makes the error disappear.

      Attachments

        1. TEZ-682.1.patch
          7 kB
          Bikas Saha

        Issue Links

          Activity

            People

              bikassaha Bikas Saha
              hagleitn Gunther Hagleitner
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: