Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Duplicate
-
None
-
None
-
None
-
None
Description
Currently oozie looks to make sure the URL prefix for where job information is stored is "hdfs://...". There are API compatible alternatives to hdfs now. Work on OOZIE-477 made this configurable but was resolved as won't fix. This specific issue, however, needs to be addressed.
As I understand it, the only reason for having that hardcoded constraint to begin with was because there was a concern that people would accidentally use the local filesystem rather than hdfs and it was a way to easily fail fast. One way to address this would be to do what was done with the code on OOZIE-477 and make it possible to configure it. However, I wonder if another way would be to just check to make sure that it wasn't the local filesystem. So perhaps unless the URL begins with "file://..." just accept it and let it fail if it's not available. That way, no special casing or configuration has to happen for API compatible alternatives and the original problem is still addressed. What do people think of that solution? If it's workable, I can submit a patch to that effect.
Attachments
Issue Links
- relates to
-
OOZIE-477 Adding configurable filesystem support instead of hardcoded "hdfs"
- Closed