Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15691

Add PathCapabilities to FS and FC to complement StreamCapabilities

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.3.0, 3.2.2, 3.2.3
    • None
    • None

    Description

      This complements the StreamCapabilities interface by allowing applications
      to probe for a specific path on a specific instance of a FileSystem
      or FileContext offering a specific feature.

      This is intended to allow applications to determine

      • Whether a method is implemented before calling it and dealing with
        any subsequent UnsupportedOperationException.
      • Whether a specific feature is believed to be available in the remote store.

      As well as a common set of capabilities defined in CommonPathCapabilities,
      file systems are free to add their own capabilities, prefixed with
      fs. + schema + .

      The plan is to identify and document more capabilities -and for file systems
      which add new features, for a declaration of the availability of the feature to
      always be available.

      The interface may be offered by other classes too; there is no restriction here.

      Note

      • The remote store is not expected to be checked for the feature;
        It is more a check of client API and the client's configuration/knowledge
        of the state of the remote system.
      • Permissions are not checked.

      This is needed for

      • HADOOP-14707: declare that a dest FS supports permissions
      • object stores to declare that they offer PUT-in-place alongside (slow-rename)
      • Anything else where the implementation semantics of an FS is so different caller apps would benefit from probing for the underlying semantics

      I know, we want all filesystem to work exactly the same. But it doesn't hold, especially for object stores —and to efficiently use them, callers need to be able to ask for specific features.

      Attachments

        1. HADOOP-15691-001.patch
          34 kB
          Steve Loughran
        2. HADOOP-15691-002.patch
          33 kB
          Steve Loughran
        3. HADOOP-15691-003.patch
          48 kB
          Steve Loughran
        4. HADOOP-15691-004.patch
          56 kB
          Steve Loughran

        Issue Links

          Activity

            People

              stevel@apache.org Steve Loughran
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: