Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-4653 DAGScheduler refactoring and cleanup
  3. SPARK-6746

Refactor large functions in DAGScheduler to improve readibility

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Won't Fix
    • None
    • None
    • Scheduler, Spark Core
    • None

    Description

      The DAGScheduler class contains two huge functions that make it
      very hard to understand what's going on in the code. These are:

      1) The monolithic handleTaskCompletion
      2) The cleanupStateForJobAndIndependentStages function

      These can be simply modularized to eliminate some awkward type casting and improve code readability.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              ilganeli Ilya Ganelin
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: