Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1744

DistributedCache creates its own FileSytem instance when adding a file/archive to the path

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.23.1, 1.0.0
    • None
    • None
    • Reviewed

    Description

      According to the contract of UserGroupInformation.doAs() the only required operations within the doAs() block are the
      creation of a JobClient or getting a FileSystem .

      The DistributedCache.add(File/Archive)ToClasspath() methods create a FileSystem instance outside of the doAs() block,
      this FileSystem instance is not in the scope of the proxy user but of the superuser and permissions may make the method
      fail.

      One option is to overload the methods above to receive a filesystem.

      Another option is to do obtain the FileSystem within a doAs() block, for this it would be required to have the proxy
      user set in the passed configuration.

      The second option seems nicer, but I don't know if the proxy user is as a property in the jobconf.

      Attachments

        1. MAPREDUCE-1744-0.23-trunk.patch
          6 kB
          Alejandro Abdelnur
        2. MAPREDUCE-1744-0.23-trunk.patch
          5 kB
          Alejandro Abdelnur
        3. MAPREDUCE-1744.patch
          7 kB
          Arun Murthy
        4. mapred-1744-3.patch
          7 kB
          Krishna Ramachandran
        5. mapred-1744-2.patch
          7 kB
          Krishna Ramachandran
        6. mapred-1744-1.patch
          7 kB
          Krishna Ramachandran
        7. mapred-1744.patch
          8 kB
          Krishna Ramachandran
        8. h1744.patch
          7 kB
          Krishna Ramachandran
        9. BZ-3503564--2010-05-06.patch
          14 kB
          Dick King

        Issue Links

          Activity

            People

              dking Dick King
              dking Dick King
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: