[HUDI-7244] Ensure ClosableIterator is propagated all the way to FileScanRDD - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Critical
Resolution: Fixed
Affects Version/s: None
Fix Version/s: None
Component/s: spark, spark-sql
Labels:
- pull-request-available

Epic Link:
Redesign Hudi Spark Integration

Description

CI tests are OOMing. One cause is that resources are not being freed from the new filegroup reader. After some code inspection, it was found that close is not being called in the HoodieFileGroupReaderIterator

Attachments

Issue Links

links to

GitHub Pull Request #10381

Activity

People

Assignee:: Jonathan Vexler

Reporter:: Jonathan Vexler

Votes:: 0 Vote for this issue

Watchers:: 1 Start watching this issue

Dates

Created:: 20/Dec/23 22:55

Updated:: 25/Jan/24 04:13

Resolved:: 25/Jan/24 04:13