Description
I was looking at the code and it was a bit difficult to see the life cycle of InMemoryFileIndex passed into getOrInferFileFormatSchema, because once it is passed in, and another time it was created in getOrInferFileFormatSchema. It'd be easier to understand the life cycle if we move the creation of it out.
Attachments
Issue Links
- is related to
-
SPARK-26222 Scan: track file listing time
- Resolved
- links to