Details
-
Bug
-
Status: Resolved
-
Blocker
-
Resolution: Fixed
-
None
-
None
Description
java.lang.IllegalArgumentException at java.nio.Buffer.limit(Buffer.java:275) at org.apache.hadoop.io.compress.lz4.Lz4Compressor.compress(Lz4Compressor.java:240) at org.apache.hadoop.io.compress.BlockCompressorStream.compress(BlockCompressorStream.java:149) at org.apache.hadoop.io.compress.BlockCompressorStream.write(BlockCompressorStream.java:131) at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:58) at java.io.DataOutputStream.write(DataOutputStream.java:107) at org.apache.tez.runtime.library.common.sort.impl.IFile$Writer.writeKVPair(IFile.java:423) at org.apache.tez.runtime.library.common.sort.impl.IFile$Writer.append(IFile.java:392) at org.apache.tez.runtime.library.common.sort.impl.dflt.DefaultSorter.spill(DefaultSorter.java:927) at org.apache.tez.runtime.library.common.sort.impl.dflt.DefaultSorter.sortAndSpill(DefaultSorter.java:865) at org.apache.tez.runtime.library.common.sort.impl.dflt.DefaultSorter.flush(DefaultSorter.java:729) at org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput.close(OrderedPartitionedKVOutput.java:191) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.close(LogicalIOProcessorRuntimeTask.java:398) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:83) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:2035) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Attachments
Attachments
Issue Links
- is caused by
-
TEZ-4135 Improve memory allocation when executing in-memory reads
- Resolved
- is related to
-
TEZ-4135 Improve memory allocation when executing in-memory reads
- Resolved
-
TEZ-4199 MergeManager::finalMerge should make use of compression
- Resolved
- relates to
-
TEZ-4239 Optimize compression/decompression buffer allocation
- Open