Description
Currently, when S3Guard is enabled, getFileStatus() will skip S3 if it gets a result from the MetadataStore (e.g. dynamodb) first.
I would like to add a new parameter fs.s3a.metadatastore.getfilestatus.authoritative which, when true, keeps the current behavior. When false, S3AFileSystem will check both S3 and the MetadataStore.
I'm not sure yet if we want to have this behavior the same for all callers of getFileStatus(), or if we only want to check both S3 and MetadataStore for some internal callers such as open().
Attachments
Issue Links
- is related to
-
HADOOP-16279 S3Guard: Implement time-based (TTL) expiry for entries (and tombstones)
- Resolved
- relates to
-
HADOOP-14467 S3Guard: Improve FNFE message when opening a stream
- Resolved