Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-8495

Change ownership of the directory to bulk load

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Trivial
    • Resolution: Fixed
    • 0.94.7, 0.95.0
    • 0.99.0, 0.94.21, 0.98.4
    • mapreduce
    • None

    Description

      To bulk load something you need to change the ownership of the data directory to allow the hbase user to read and move the files, also in the split case you must use the hbase user to run the LoadIncrementalHFiles tool, since internally some directories "_tmp" are created to add the split reference files.

      In a secure cluster, the SecureBulkLoadEndPoint will take care of this problem by doing a chmod 777 on the directory to bulk load.

      NOTE that a chown is not possible since you must be a super user to change the ownership, a change group may be possible but the user must be in the hbase group... and anyway it will require a chmod to allow the group to perform the move.

      Caused by: org.apache.hadoop.security.AccessControlException: Permission denied: user=hbase, access=WRITE, inode="/test/cf":th30z:supergroup:drwxr-xr-x
      	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:205)
      Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(java.io.IOException): java.io.IOException: Exception in rename
      	at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.rename(HRegionFileSystem.java:928)
      	at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.commitStoreFile(HRegionFileSystem.java:340)
      	at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.bulkLoadStoreFile(HRegionFileSystem.java:414)
      

      Attachments

        1. HBASE-8495-0.94-v0.patch
          1 kB
          Matteo Bertozzi

        Activity

          People

            mbertozzi Matteo Bertozzi
            mbertozzi Matteo Bertozzi
            Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: