Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-24667

If folders managed by DiskBlockManager are deleted manually, shell throws FileNotFoundException

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Not A Problem
    • 2.3.1
    • None
    • Block Manager, Spark Core
    • None

    Description

      1. In spark shell
      2. Run a join query, which writes some data to folders managed by DiskBlockManager.
      3. Delete some of the folders, e.g.tmp\blockmgr-7145fe5a-757c-4640-a032-047c7f1d4080\2d
      4. run the same query again, it throws FileNotFoundException
      5. Suggest to check the folder existence when found a folder in subDirs

      Attachments

        Activity

          People

            Unassigned Unassigned
            wuyh1007 wuyonghua
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: