Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
2.7.1
-
None
-
None
Description
As discussed on MAPREDUCE-6302, we could improve the reducer preemption as follows:
- preempt enough reducers so there are enough mappers to match slowstart
- prioritize preempting reducers that are still in SHUFFLE phase
- add an option to not preempt reducers that are past SHUFFLE phase irrespective of slowstart as long as one mapper can run
We could do it all in one patch or create subtasks as necessary.
Attachments
Issue Links
- is related to
-
MAPREDUCE-6302 Preempt reducers after a configurable timeout irrespective of headroom
- Closed