Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-14526

SnowflakeIO fails to recognize new-line char in a column

Details

    • Bug
    • Status: Triage Needed
    • P2
    • Resolution: Unresolved
    • 2.38.0
    • None
    • io-java-snowflake
    • None

    Description

      When a  snowflake string column contains new-line, the snowflakeIO fails to read it as a record. Instead two invalid records gets created.

      I see the issue due to the usage of TextIO; as the exported extract from the snowflake contains the column data enclosed with quoted string, making it a valid CSV.

      Possibly be solved if FileIO(with custom splitable dofn to handle big file) is used instead of TextIO.

      Attachments

        Activity

          People

            Unassigned Unassigned
            Varun Rauthan Varun
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: