Details
-
Sub-task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
3.4.0, 3.3.6
-
None
-
None
Description
x3a xattr list needs a prefix compatible with hdfs or existing code which tries to copy attributes between stores can break
we need a prefix of
{user/trusted/security/system/raw}.
now, problem: currently xattrs are used by the magic committer to propagate file size progress; renaming the prefix will break existing code. But as it's read only we could modify spark to look for both old and new values.
org.apache.hadoop.HadoopIllegalArgumentException: An XAttr name must be prefixed with user/trusted/security/system/raw, followed by a '.'
at org.apache.hadoop.hdfs.XAttrHelper.buildXAttr(XAttrHelper.java:77)
at org.apache.hadoop.hdfs.DFSClient.setXAttr(DFSClient.java:2835)
at org.apache.hadoop.hdfs.DistributedFileSystem$59.doCall(DistributedFileSystem.java:3106)
at org.apache.hadoop.hdfs.DistributedFileSystem$59.doCall(DistributedFileSystem.java:3102)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.setXAttr(DistributedFileSystem.java:3115)
at org.apache.hadoop.fs.FileSystem.setXAttr(FileSystem.java:3097)
Attachments
Issue Links
- relates to
-
HIVE-25609 Preserve XAttrs in normal file copy case.
-
- Resolved
-