Details
-
Bug
-
Status: Triage Needed
-
P2
-
Resolution: Unresolved
-
2.38.0
-
None
-
None
Description
When a snowflake string column contains new-line, the snowflakeIO fails to read it as a record. Instead two invalid records gets created.
I see the issue due to the usage of TextIO; as the exported extract from the snowflake contains the column data enclosed with quoted string, making it a valid CSV.
Possibly be solved if FileIO(with custom splitable dofn to handle big file) is used instead of TextIO.