Details
-
Documentation
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
3.0.0
-
None
Description
Sometimes directory listing could become a bottleneck when user jobs have large number of input directories. This is especially true when against object store like S3.
There are a few parameters to tune this. This proposes to add some info in the tuning guide so that the knowledge can be better shared.