Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15226 Über-JIRA: S3Guard Phase III: Hadoop 3.2 features
  3. HADOOP-13756

LocalMetadataStore#put(DirListingMetadata) should also put file metadata into fileHash.

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0-beta1
    • 3.2.0
    • fs/s3, test
    • None

    Description

      LocalMetadataStore#put(DirListingMetadata) only puts the metadata into dirHash, thus all FileStatus s are missing from LocalMedataStore#fileHash(), which makes it confuse to use.

      So in the current way, to correctly put file status into the store (and also set authoriative flag), you need to run

      List<PathMetadata> metas = new ArrayList<PathMetadata>();
      boolean authorizative = true;
      for (S3AFileStatus status : files) {
         PathMetadata meta = new PathMetadata(status);
         store.put(meta);
      }
      DirListingMetadata dirMeta = new DirMeta(parent, metas, authorizative);
      store.put(dirMeta);
      

      Since solely calling store.put(dirMeta) is not correct, and calling store.put(dirMeta); after putting all sub-file FileStatuss does the repetitive jobs. Can we just use a put(PathMetadata) and a get/setAuthorative() in the MetadataStore interface instead?

      Attachments

        1. HADOOP-13756.001.patch
          3 kB
          Gabor Bota

        Activity

          People

            gabor.bota Gabor Bota
            eddyxu Lei (Eddy) Xu
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: