Blocker Description
TestCombiner.testMultiCombinerUse() in pig test suite sets io.sort.mb to 1 for testing purposes. Tez is stuck at a loop while running the task with 0.4.0-incubating and 0.5.0-incubating-SNAPSHOT in org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.computeBytesInMerges. It runs fine with 0.3.0-incubating.
at java.lang.Long.valueOf(Long.java:557) at org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.computeBytesInMerges(TezMerger.java:814) at org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.merge(TezMerger.java:571) at org.apache.tez.runtime.library.common.sort.impl.TezMerger$MergeQueue.merge(TezMerger.java:541) at org.apache.tez.runtime.library.common.sort.impl.TezMerger.merge(TezMerger.java:164) at org.apache.tez.runtime.library.common.sort.impl.dflt.DefaultSorter.mergeParts(DefaultSorter.java:1078) at org.apache.tez.runtime.library.common.sort.impl.dflt.DefaultSorter.flush(DefaultSorter.java:633) at org.apache.tez.runtime.library.output.OnFileSortedOutput.close(OnFileSortedOutput.java:134) - locked <7f03a5b90> (a org.apache.tez.runtime.library.output.OnFileSortedOutput) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.close(LogicalIOProcessorRuntimeTask.java:331) at org.apache.hadoop.mapred.YarnTezDagChild$5.run(YarnTezDagChild.java:567) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:394) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) at org.apache.hadoop.mapred.YarnTezDagChild.main(YarnTezDagChild.java:553)
To reproduce:
ant clean test -Dtestcase=TestCombiner -Dexectype=tez -Dhadoopversion=23 -Dtest.output=true -logfile /tmp/log