Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-4653 DAGScheduler refactoring and cleanup
  3. SPARK-15926

Improve readability of DAGScheduler stage creation methods

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.0.0
    • 2.1.0
    • Scheduler, Spark Core
    • None

    Description

      The current code to create new stages is very confusing: it's difficult to reason about which functions actually create new stages versus just looking up existing stages, and there are many similarly-named functions that do very different things. The goal of this JIRA is to clean some of that up.

      Attachments

        Issue Links

          Activity

            People

              kayousterhout Kay Ousterhout
              kayousterhout Kay Ousterhout
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: