Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-14631

Distcp should add a default atomicWorkPath properties when using atomic

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.7.3, 3.0.0-alpha3
    • 2.7.3
    • None
    • None

    Description

      Distcp should add a default AtomicWorkPath properties when using atomic

      Distcp#configureOutputFormat using code below to generate atomic work path,

          if (context.shouldAtomicCommit()) {
            Path workDir = context.getAtomicWorkPath();
            if (workDir == null) {
              workDir = targetPath.getParent();
            }
            workDir = new Path(workDir, WIP_PREFIX + targetPath.getName()
                                      + rand.nextInt());
      

      When atomic is set and AtomicWorkPath == null, distcp will get the parent of current WorkDir. In this case, if workdir is "/", the parent will be null, wich means
      workDir = new Path(workDir, WIP_PREFIX + targetPath.getName() + rand.nextInt()); will throw a nullpoint exception.

      Attachments

        Issue Links

          Activity

            People

              Hongyuan Li Hongyuan Li
              Hongyuan Li Hongyuan Li
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: