Details
-
Bug
-
Status: Closed
-
Blocker
-
Resolution: Fixed
-
None
-
None
-
None
Description
Current certain S3 path patterns break with Hudi, for ex: paths with <string>.<integer> pattern in S3 bucket name. This error stems from https://github.com/apache/hudi/blob/master/hudi-common/src/main/java/org/apache/hudi/common/fs/HoodieWrapperFileSystem.java#L148 where it uses getHost() API while converting to new Scheme. While this works well for HDFS where it always has a host, it does not work well with this specific pattern which is a valid S3 bucket name.
For example:
s3://bucket.1234/table1 gets incorrectly converted to s3:/table1
Instead we should use getAuthority() which correctly gets the S3 bucket name as the authority.
Attachments
Issue Links
- links to