Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-17032

Handle an internal dir in viewfs having multiple children mount points pointing to different filesystems

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.2.2, 3.3.1, 3.4.0
    • fs, viewfs
    • None
    • Reviewed

    Description

      In case the viefs mount table is configured in a way where multiple child mount points are pointing to different file systems, the getContentSummary or getStatus don't return the expected result

      mount link /a/b/ → hdfs://nn1/a/b
 mount link /a/d/ → file:///nn2/c/d

      b has two files and d has 1 file. So getContentSummary on / should return 3 files.

      It also fails for the following scenario:

      mount link  /internalDir -> /internalDir/linternalDir2
mount link  /internalDir -> /internalDir/linkToDir2 -> hdfs://nn1/dir2

      Exception:

      java.io.IOException: Internal implementation error: expected file name to be /java.io.IOException: Internal implementation error: expected file name to be /
 at org.apache.hadoop.fs.viewfs.InternalDirOfViewFs.checkPathIsSlash(InternalDirOfViewFs.java:88) at org.apache.hadoop.fs.viewfs.InternalDirOfViewFs.getFileStatus(InternalDirOfViewFs.java:154) at org.apache.hadoop.fs.FileSystem.getContentSummary(FileSystem.java:1684) at org.apache.hadoop.fs.FileSystem.getContentSummary(FileSystem.java:1695) at org.apache.hadoop.fs.viewfs.ViewFileSystem.getContentSummary(ViewFileSystem.java:918) at org.apache.hadoop.fs.viewfs.ViewFileSystemBaseTest.testGetContentSummary(ViewFileSystemBaseTest.java:1106)

      Attachments

        Issue Links

          is duplicated by

          Sub-task - The sub-task of the issue HDFS-15426 Fix ContentSummary for mount links in ViewFileSystemOverloadScheme

          • Major - Major loss of function.
          • Resolved
          relates to

          New Feature - A new feature of the product, which has yet to be developed. HDFS-15289 Allow viewfs mounts with HDFS/HCFS scheme and centralized mount table

          • Major - Major loss of function.
          • Open

          Activity

            People

              abhishekd Abhishek Das
              abhishekd Abhishek Das
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: