Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
The scheduler's Stage class has many fields which are only applicable to result stages or shuffle map stages. As a result, I think that it makes sense to make Stage into an abstract base class with two subclasses, ResultStage and ShuffleMapStage. This would improve the understandability of the DAGScheduler code.