Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
1.13.1
-
None
-
None
Description
I try to use the `appendFile` method of `ParquetFileWriter` to merge some smaller Parquet files into one large parquet file. After I finished the merge, I tried deleting the smaller files, but couldn't actually delete them.
After reading the ParquetFileWriter source code, I found that the file he opened was not closed in the end method, which seemed unreasonable.
When testing on `Windows`, it was found that these implicitly opened files cannot be deleted or renamed. This seems to be related to Windows file handles. But it's worked fine on macOS.
Anyway, I will submit a patch to ensure that files opened through the appendFile function are closed correctly.