Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-746

DAG and Vertex commit should ensure that all tasks have completed before committing

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • None
    • None
    • None
    • None

    Description

      There may be a race when a task (say a speculative task) is still running. Then the committer commits (say lists & cleans some files in the output dir). Then that task writes some temporary files in the output dir and gets killed. Those temp files may remain behind.

      Attachments

        Issue Links

          Activity

            People

              bikassaha Bikas Saha
              bikassaha Bikas Saha
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: