Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
0.3.2
-
None
-
None
-
all, but especially on big busy clusters
Description
When a task writes to DFS file, depending on how busy the cluster is, it can timeout after 10 minutes by default, because the progress towards writing a DFS file does not count as progress of the task. The solution (patch is forthcoming) is to provide a way to callback reporter to report task progress from DFSOutputStream.