Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-4280

In dynamic allocation, add option to never kill executors with cached blocks

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 1.2.0
    • None
    • Spark Core
    • None

    Description

      Even with the external shuffle service, this is useful in situations like Hive on Spark where a query might require caching some data. We want to be able to give back executors after the job ends, but not during the job if it would delete intermediate results.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              sandyr Sandy Ryza
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: