Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-695

Oozie should not require that job information is stored in locations beginning with "hdfs://"

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • None
    • None
    • None
    • None

    Description

      Currently oozie looks to make sure the URL prefix for where job information is stored is "hdfs://...". There are API compatible alternatives to hdfs now. Work on OOZIE-477 made this configurable but was resolved as won't fix. This specific issue, however, needs to be addressed.

      As I understand it, the only reason for having that hardcoded constraint to begin with was because there was a concern that people would accidentally use the local filesystem rather than hdfs and it was a way to easily fail fast. One way to address this would be to do what was done with the code on OOZIE-477 and make it possible to configure it. However, I wonder if another way would be to just check to make sure that it wasn't the local filesystem. So perhaps unless the URL begins with "file://..." just accept it and let it fail if it's not available. That way, no special casing or configuration has to happen for API compatible alternatives and the original problem is still addressed. What do people think of that solution? If it's workable, I can submit a patch to that effect.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              jeromatron Jeremy Hanna
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: