Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-3921

StringParser not specifying encoding to use

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Implemented
    • 1.0.3
    • 1.2.0
    • None
    • None

    Description

      Class `flink.types.parser.StringParser` has javadocs indicating that contents are expected to be Ascii, similar to `StringValueParser`. That makes sense, but when constructing actual instance, no encoding is specified; on line 66 f.ex:

      this.result = new String(bytes, startPos+1, i - startPos - 2);

      which leads to using whatever default platform encoding is. If contents really are always Ascii (would not count on that as parser is used from CSV reader), not a big deal, but it can lead to the usual Latin-1-VS-UTF-8 issues.

      So I think that encoding should be explicitly specified, whatever is to be used: javadocs claim ascii, so could be "us-ascii", but could well be UTF-8 or even ISO-8859-1.

      Attachments

        Issue Links

          Activity

            People

              rekhajoshm Rekha Joshi
              cowtowncoder Tatu Saloranta
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: