[SPARK-4653] DAGScheduler refactoring and cleanup - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Resolved
Priority: Major
Resolution: Incomplete
Affects Version/s: None
Fix Version/s: None
Component/s: Scheduler, Spark Core
Labels:
- bulk-closed

Description

This is an umbrella JIRA for DAGScheduler refactoring and cleanup. Please comment or open sub-issues if you have refactoring suggestions that should fall under this umbrella.

Attachments

Sub-Tasks

1.	Clean up DAGScheduler's getMissingParentStages() and stageDependsOn() methods	Resolved	Josh Rosen
2.	Split Stage into ShuffleMapStage and ResultStage subclasses	Resolved	Ilya Ganelin
3.	abstract RDD's DAG graph iteration in DAGScheduler	Closed	Unassigned
4.	Refactor large functions in DAGScheduler to improve readibility	Closed	Unassigned
5.	Improve readability of DAGScheduler stage creation methods	Resolved	Kay Ousterhout
6.	Eliminate redundant code in DAGScheduler's getParentStages and getAncestorShuffleDependencies methods.	Resolved	Kay Ousterhout
7.	Remove task-level functionality from the DAGScheduler	Resolved	Kay Ousterhout

Activity

People

Assignee:: Josh Rosen

Reporter:: Josh Rosen

Votes:: 1 Vote for this issue

Watchers:: 6 Start watching this issue

Dates

Created:: 29/Nov/14 19:38

Updated:: 17/May/20 17:48

Resolved:: 21/May/19 05:36