Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-35751

Support Joint eviction strategies for cached RDD partitions

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: In Progress
    • Minor
    • Resolution: Unresolved
    • 3.1.2
    • None
    • Block Manager
    • None

    Description

      As of now, there's as it were ousting procedure for cached RDD segment in Spark. The default RDD removal methodology is LRU .When memory space not adequate for RDD caching, a few allotments will be ousted, on the off chance that these segments are utilized once more latterly, they will be replicated by the Ancestry data and cached in memory once more. The replicate stage will bring in extra taken a toll. Be that as it may, LRU has no ensure for the most reduced duplicate cost. The to begin with RDD that required to be cached is ordinarily created by perusing from HDFS and doing a few changes. The perusing operation ordinarily fetched longer time than other Start transformations. 

      Attachments

        Activity

          People

            Unassigned Unassigned
            qfoxzjd JindongZhang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: